A Brief History of Open Source Intelligence

An article with the above title appeared on the bellingcat site.

It is an excellent article even if I don’t agree that OSINT went into hibernation after WW2. For example, from after WW2 until the Cold War ended, in the US, the Foreign Broadcast Information Service, which is now the Open Source Enterprise and in the UK the BBC Monitoring Service trawled the airwaves, and other open sources regularly publishing transcripts and analysis of what they heard, starting after the war and continues today. There are many other examples.

On the other hand, today’s OSINT is highly influenced by a convergence of technologies. The market penetration of smartphones with 3G connections and the popularity of social media sites is one such convergence of technologies that produces raw data. The other convergence of technology is the availability of inexpensive software and computer hardware to process the raw data for analysis.

The Darknet & Freenet

Freenet is like BitTorrent with web sites. Freenet is an anonymous peer-to-peer data-sharing network where uploaded data is assigned a unique key then broken-up into small, encrypted chunks which are then scattered across multiple computers on the network.

When someone wants a document, photograph or some other data, they “fetch” it from the network using the unique key assigned to that data. The fetch requests get routed through intermediary computers that don’t house the requested data, This ensures that no single computer on the network knows the contents of any individual data file.

With the Freenet client running on your PC, you can use most Web browsers to browse files and websites (AKA freesites) on the Freenet. The client allows you to access the Freenet welcome page ( using your normal browser. From this welcome page, you can move on to browse Freenet, chat on Freenet forums, and communicate with other Freenet users.

Freenet has a darknet mode (AKA friends-only mode) for maximum privacy. In darknet mode, you connect to Freenet through trusted associates with whom you exchange encryption keys, which makes it difficult for anyone to track your movements on Freenet or even that you’re using Freenet. Of course, funneling your Freenet access through a handful of trusted associates may create a traffic bottleneck that slows response times. To avoid this, get five or ten friends to join up with you so you can fetch Freenet websites and files at greater speed.

Don’t expect this to provide total anonymity if you are doing something that is illegal or a risk to national security. Freenet has been infiltrated by police agencies that have created their own Freenet nodes to deanonymize users. You can be certain that national intelligence agencies have done the same.

Chrome is Listening

So you want to use Chrome as your browser. Are you aware that it has recently been reported that a Chrome Bug Allows Sites to Listen to Your Private Conversations?

The best way to avoid this threat is as follows:

  • Go to chrome://settings/content
  • Scroll down to Media
  • Select “Do not allow any sites to access my camera and microphone.

This will disable Google’s Conversational Search, etc. but security will be increased.

I never liked the way Chrome ‘phoned home’ to Google with user tracking, bug tracking etc. I have also found extensions that had malware-filled updates. However, it is faster than Firefox, which over the course of a research project may save hours of extra time. I resisted using Chrome due to security & privacy issues.

I now use is Comodo Dragon, which is based on the open-source Chrome browser, however, it is more private and secure if used properly. I disable the camera & mic as SOP, so I haven’t investigated how Dragon responds to this exploit. The setting change that I outlined was in reference to the actual Chrome browser and this particular exploit, there may be more that I don’t know about.

I am very careful about exposing myself to the internet. My outward-facing computers don’t have cameras or mics to entirely circumvent malicious software like this and the likes of Finspy.

Windows Error Reporting Risk

Windows Error Reporting (WER) is a crash reporting technology introduced by Microsoft with Windows XP. However, we now know that it may send Microsoft unencrypted personally identifiable information contained in the memory and application data that may make you vulnerable to attack. WER is turned on by default. WER from Windows 8 may now use TLS encryption.

The Snowdon leaks described how the U.S. National Security Agency intercepts the unencrypted WER logs to fingerprint machines like some malware to identify potential system, network and application weaknesses to execute attacks that move through an enterprise network. WER reports on more than Windows crashes. It reports hardware changes, such as the first-time use of a new USB device and mobile devices. It sends time-stamp data, device manufacturer, identifier and revision, along with host computer information such as default language, operating system service pack and update version, hardware manufacturer, model and name, as well as BIOS version and unique machine identifier. This creates a blueprint of the applications running on a network to help an attacker develop or execute attacks with little chance of detection.

This is only one example of the OS, applications, browsers, etc. leaking information that the investigator must be aware of when conducting investigative internet research.

To shut-off WER in Windows 7 go to Control Panel>System and Security>Action Center>Change Action Center settings>Related settings>Problem reporting settings. The selections for “Each time a problem occurs, ask me before checking for solutions” and “Never check for solutions” disable WER. Choosing Never check for solutions will fully disable error reporting in Windows 7.


Asymmetric Warfare & Business Continuity

In a previous article, I wrote about a system that created a single point of failure. In a strategic sense, computers and IT as a whole have become a single point of failure in both government and industry.

Chinese military leaders call automation the great equalizer, since its enemies heavily depend upon computers. An effective attack upon their enemy’s IT infrastructure provides an immediate and disproportionate impact which is the core concept of asymmetric warfare.

This asymmetry benefits the attacker, regardless of his motives or methods.

DIY Intelligence Agency

This is how to built yourself a very robust personal Intelligence Agency. Every intelligence agency in the world tracks key words, information patterns, and news events from a central aggregated location.

  1. Create a Google account while being discrete with the information you put in the profile.
  2. Log on to your Google account and in the top left of the screen go to more then down to even more and select Alerts, which appears as the first link on the More Google Products page. Set up a number of these alerts for “news” and “blogs” based on your search terms. Set each of these to “as-it-happens” to e-mail you with a link to the article.
  3. Set up your smart phone to receive these alerts, and code the incoming messages with a special sound. You’ll then get a specific sound on your smart phone with each Google Alert.
  4. Establish a  Google Reader account. Subscribe to all the blogs you can find on your topics of interest. Google Reader includes a search bar to help search through the dross to find the good stuff. You now have an online central location from which you can manage your information intake.
  5. Over time, add more and more RSS feeds.  Intelligence agencies have them, and so should you.  You will be surprised Google Reader and the Google Alerts you will consistently outperform major news organizations in bring actionable intelligence to your attention. The may give you a competitive edge.
  6. Check your favorite blogs and and those that they are linking to consistently. Add these RSS feeds to your reader. Check the blog rolls of the blogs to which you subscribe  and add all of their RSS feeds to your reader. To vet these new sources, use the reader’s search facility.

CIA World Factbook

The CIA announced that their World Factbook Web site had been redesigned. I’m not the only person who constantly relies on this — over 3 million visitors access the online Factbook monthly. That’s not surprising as the World Factbook provides information about the background, geography, people, government, economy, communications, transportation, military, and transnational issues for 266 countries and other entities.

I really like the new features of reporting world rankings for data like life expectancy. Another new feature is the “Field Listing” icon that gives you an alphabetical listing of countries for that field so that you can do your own comparison of data that can’t be ranked.

This is a timely resource — it is updated every two weeks and the updates are logged on a special page. Though I wish either the country entries or data fields indicated the last update, but that might be asking too much.

If you want to avoid all the Flash content use the text-only version. I’m not a big fan of Flash, but this is a very well executed use of it that makes the World Factbook more useful.

China’s Espionage and Cyber Attack Strategy

An excellent article about the “recent discovery of Chinese cyber warfare attacks on foreign computers, on communication computers of visiting dignitaries, and espionage activities to assist a friendly country is building weapons of mass destruction (WMDI)” entitled China’s Silent Warfare at BLOg Source INTelligence reveals a lot about China’s espionage and cyber attack strategy.

Mobile Phones & Tin Foil Hats

Under certain circumstances, if you lose sight of your mobile telephone, then you may reasonably assume it has been compromised. These circumstances are more common than you might think. Here are two cases of this that I have encountered over the last year or so.

Read more

An Intellegence Revolution

I’m sometimes referred to dismissively as the Investigator who searches databases, or the guy who gets other people to do research and just manages the report writing (they can’t grasp the concept of a Project Manager). These people dinosaurs just don’t understand that the conduct of knowledge work has changed and that it will continue to change. Industrial technology brought about the Industrial Revolution, now information  technology is bringing about an Intelligence Revolution.

For example, the  news media acknowledged that Wikipedia was the clearing house for information about the Virginia Tech shooting. Over 8000 amendments to the Wikipedia article were posted in 2 weeks. A former director of the National Security Agency told  congress  in 2002, “Al-Qaida did not need to develop a telecommunication system. All it had to do was harvest the products of a three trillion dollar a year telecommunications industry; an industry that had made communications signals varied, global, instantaneous, complex, and encrypted.”

Open sources, open systems, and advanced telecommunications technology are changing how any form of intelligence collection and reporting is done. These developments have also changed how we have to look at the Intelligence Cycle. The decision-makers and intelligence professionals must now come together within the same space and time to focus on the target in a collaborative model using easily configurable open systems. ( An open system, in management science, is a system that is capable of self-maintenance on the basis of throughput of resources from the environment and usually operated on  a computer system that provides a  combination of interoperability, portability, and open software standards.)  In effect, contributors, analysts, and end-users must employ every tool available simultaneously. There is no time for the traditional Intelligence Cycle to function. Clark’s Intelligence Analysis: A Target-Centric Approach, 2nd Edition and  the “fusion cells” in Iraq may offer models for this more focused, collaborative, and time-compressed intelligence process.

This presents management difficulties associated with the resistance to change, training, organisational structure, the introduction of new technology, and outsourcing. Contractors will collect and fact-check data before entering it into an open system for further processing. Portions of intelligence projects will be managed by outside contractors who compile data from many sources and then feed the results into the open system. Contractors will create chronologies, social network maps, link diagrams, and databases, all of which will be available through an open system. Everybody involved with collection will have some contact with the end user or project manager.

These changes are starting to happen in the public sector. If you do competitive intelligence or complex investigations in the private sector, then you need to start changing your work processes or be left behind by your competitors.  Adapt, or become a fossil.

Terrorists in Internet Virtual Worlds

Massively Multiplayer Online Role-Playing Games (MMORPG) like World of Warcraft are little more than communication tools and some terrorism and intelligence experts are concerned that it is theoretically possible that such platforms as MMORPG’s and Second Life might be used to plan terrorist attacks. For an overview of this topic see The Ongoing Debate About The Possibility of Terrorists in Virtual Worlds  at The Evince Blog.

U.S. Policy of Seizing Data at the Border

The U.S. government has published its policy regarding seizing laptops and other devices capable of storing data.

Federal agents may take a traveler’s laptop or other electronic device to an off-site location for an unspecified period of time without any suspicion of wrongdoing, as part of border search policies the Department of Homeland Security recently disclosed. Also, officials may share copies of the laptop’s contents with other agencies and private entities for language translation, data decryption, or other reasons, according to the policies, dated July 16 and issued by two DHS agencies, US Customs and Border Protection and US Immigration and Customs Enforcement… DHS officials said that the newly disclosed policies — which apply to anyone entering the country, including US citizens — are reasonable and necessary to prevent terrorism… The policies cover ‘any device capable of storing information in digital or analog form,’ including hard drives, flash drives, cell phones, iPods, pagers, beepers, and video and audio tapes. They also cover ‘all papers and other written documentation,’ including books, pamphlets and ‘written materials commonly referred to as “pocket trash…”

It seems the best thing is to keep encrypted files on a network drive at home, and download the needed encrypted data  after crossing the border.

Secrets are Secret, unless you work in the UK Cabinet Office

By now you have heard of the secret intelligence files left on a commuter train in England.

Keith Vaz MP, chairman of the powerful Home Affairs select committee told the BBC: “Such confidential documents should be locked away…they should not be read on trains.”

This should be a reminder to the private sector regarding trade secrets.

Trade Secrets

A trade secret is not protected by a Patent, Trademark, or Industrial Design. A trade secret is confidential and proprietary information that you protect because of its commercial value and the competitive advantage that it produces for your company.

Competitive Intelligence

Exposing a trade secret in public by working on a critical document on an airplane, leaving a trade secret on a commuter train, or exposing it in an proposal, may eliminate the confidential nature of the data, and once you do that, you have, by definition, given up protecting it, therefore, it is not a trade secret that you can claim as proprietary — your former trade secret moves into the public domain for all to see and use.

As a competitive intelligence practitioner, I often find former trade secrets loose in the public domain due to irresponsible security practices. If the owner does not protect the trade secret, it ceases to be confidential and proprietary data, and is likely to become somebody else’s competitive advantage, or worse still, it might become a standard practice for an entire industry.