Information Vs. Actionable Intelligence & the PI

I see many courses for Private Investigators (PIs) about using the Internet for Open Source Intelligence (OSINT). These courses are predominately about Internet sites that might yield useful information. These courses don’t teach how to process and analyse the captured data or how to properly report what was found. The OSINT concept usually misses the “intelligence” part, and it’s more about gathering raw information, not the production of intelligence.

As an example, I just captured a FB account with about 1000 posts, thousands of friends and pictures, along with about 20 videos. How would anyone search through all of this and link it to relevant people, places, things, or companies? Even if the PI can identify some useful linkages and other data, how does he report it in a timely and cost-effective manner? All these courses conveniently omit the fact that a senior decision-maker needs an accurate and concise report that illustrates the linkages between relevant data.

Unfortunately, many of the course providers don’t create investigative or intelligence product, they teach courses about Internet sources.

According to Justin Seitz, the creator of Hunchly, a Chrome browser extension for collecting OSINT material from the Internet, “the greatest limiting factor of the OSINT concept is budgets that don’t recognise the time, resources, and training needed to complete the research, or the complexity of creating a true intelligence product. The budget provided to the PI leaves no choice but to simply provide screenshots and captured raw data to clients who don’t want to pay the premium required to deconstruct a network, or to chase-down the best breadcrumbs.” In the information industry, we call this ‘rip & ship’. Nobody expects other professionals to work like this.

In a recent discussion with Mark Northwood of Northwood & Associates, a large Canadian private investigation company, he easily summed-up the problem. “If a client retains a lawyer and the lawyer researches case law in order to determine which are the best methods to advance or defend a claim does the client simply say “give me the case law, I will interpret it”–No–the lawyer gives the client his opinion and supports it with the case law. Clients pay lawyers substantial fees for their analysis of case law, not the collected the case law itself.” Clients need the PI doing OSINT to work in the same manner.

Northwood believes that PIs need to educate their clients into understanding that someone needs to analyze the raw OSINT data and the only person that can do that is the PI because he collected the raw data and has it immediately at hand. The PI is in the best position to collate, analyse, and report on the data he has collected.

Chicken or Egg

As I see it, this is a chicken or egg problem.

The Egg

Without reasonable budgets on offer, clients won’t find PIs with the programming experience necessary to mine the collected data. Nor will clients find PIs experienced with the complex and expensive software to collect and report on the data in the first place.

Clients cannot find PIs to conduct OSINT and create actionable reports because there is no profit in it for the PI. No PI is going to acquire such skills if there is no profit in doing so. Without the prospect of reasonable wages, people with the above skills won’t become PIs; nor will people with the training in the logic, rhetoric, and argumentation needed to produce actionable reports. Existing PIs won’t be motivated to learn these skills without the prospect of financial benefit.

The Chicken

If the PI consistently has appropriate budgets to work within, then he will have or acquire proper tools and skills needed to collect, analyse, and then report on the significance of the collected data. Proper budgets also permit the PI to develop a viable reporting protocol for the type of data he collects. Proper budgets preserve the integrity of the collected data and allow for the creation of intelligence reports that include proper citations.

This chicken definitely grows from the budget egg. A large Canadian PI firm is currently advertising for someone to conduct ‘social media investigations’ at a pay rate of $15 per hour. One can only imagine the nature of the client’s expectations and the type of work produced for so little pay.


Today, any intelligence or investigative product requires a fusion of many types and sources of data. A complete report usually needs surveillance observations, content from interviews, public records, and government documents.

Again, the budget to collect and analyse public records and government documents creates the skills and knowledge needed to perform this task. This fusion of data sources allows the PI to establish relevant links between the people, places, things, and companies of interest to the client.

OSINT Tools & Skills

If the budgets come to truly represent a desire for a better product, then the following will be the tools and skills your PI should possess in the realm of OSINT. This is the rocket science behind real OSINT.


Hunchly is a Google Chrome extension that tracks and captures every page that you view during an investigation. This saves you from having to stop and take screenshots or from having to create handwritten logs of every URL that you have visited. It includes the ability to track names, phone numbers and other pieces of information. Hunchly builds a data rich case file from all of your investigative steps that helps you to preserve evidence.

Hunchly permits the use of “selectors,” such as a name, address, or phone number that save you from manually searching each page or the collected data for the terms. In my opinion, this feature alone is worth the purchase price. The other useful features include:

  • being able to add notes to what you find
  • you can download notes as a Word document
  • all collected data is stored, tracked and accessed on your local machine–no security or privacy concerns about cloud use


If your research requires graphing of the relationships between people, places, things, and companies, then CaseFile provides that at a much lower cost than other solutions if the dataset small enough to be managed manually and this is the case presently for most of the PI’s work.


Maltego is the favoured software of many intelligence analysts, researchers, and investigators for searching, and linking OSINT data. While it helps search through mountains of data and sort it in useful ways based on publicly available information that is currently sitting on the Internet, it has many limitations.

If you need to search FB by email address, Instagram by photo GPS, search people in social media sites, or search LinkedIn by company or college, then this is the tool to use. However, some these capabilities can cost $1000 per year on top of the Maltego yearly fees. Less costly alternatives exist.

Given its current state of development, I am not certain that Maltego warrants its cost for the PI. Most of the search capabilities of Maltego are in ‘transforms’, which are Python scripts that access a search site’s API[1].

The search functions of the most used ‘transforms’ can be created in Python for a lower cost. The graphing component of Maltego is available in CaseFile. Using Hunchly, CaseFile, Python scripts, Word, and PowerPoint together should produce on acceptable product if the collected data is properly summarised and then analysed.


Python is a programming language best described as a language used to create scripts that execute specific tasks, such as searching for a specific word in a sea of text.

Python automates time-consuming tasks. It allows you to parse raw data untouched by other tools and read information from databases. It aids in the generation of reports and moves files into folder structures based on their content type. From the PI’s perspective, Hunchly can handle these tasks.

Python scripts may also provide access to a search site’s API. A page of scripts enables searching a site for search terms in a variety of ways. In practice, this is the PI’s favored use of Python.

The High-end Tools

When the volume of collected data increases, so does its lack of organisation for investigative purposes. This fact has spawned many products designed to search and retrieve text strings in masses of data. This is usually called “free text retrieval” (FTR) software. The following are the current leaders in utility for investigative purposes.


The dtSearch[2] product line enables searches of terabytes of text across a desktop, network, Internet or Intranet site.


In the near future, PIs may resort to high-end tools like the Nuix suite to find connections in the vast seas of data that like the Panama Papers dataset. Nuix is a FTR software that enables searching through huge volumes of unsorted data for people, places, things, and companies. It also allows users to display connections between all these entities along with timelines is a manner similar to Maltego and CaseFile.

For more than a decade, FTR software has been the province of well-funded intelligence agencies, law firms, and businesses. Journalism has discovered this due to the donation of Nuix to the Panama Papers project[3].

Social Media Monitoring

Products like XI Social Discovery[4], Geofeedia[5], Dataminr[6], Dunami[7], and SocioSpyder[8], to name a few, are being purchased by Fortune 500 companies, and government to manage social media research. These products are now becoming necessary for the successful private investigator.

The Report

In broad strokes, the PI’s report creation process should look like the following:

  • The PI will assemble or collate all of the collected information from all the tools used, examine links, or shared information such as URLs, email addresses, etc.. From this collated material, a summary begins to take shape.
  • The investigator ensures that each piece of crucial information is put into its own section within the logical order of the summary; visuals (screenshots, text captures, tagged photos) are included as much as possible.
  • Relationship graphs exported from CaseFile or Maltego should be included in the report if they fit the page, if not, screen clips may be used or Powerpoint slides can be imported.
  • From the summary rises the true analysis of how the data relates to or affects the client’s objectives.
  • The report must describe the sources and methods used and describe all investigative activities. This is crucial when little information is uncovered about a subject. This level of detail is not included in the summary.
  • Evidence (captured images, videos, etc.) remains in a separate file from the report.


In conclusion, as with all new products, the price will drop and quality will improve as PIs adopt the necessary programming skills and software in an increasingly competitive market. Of course, this will not happen if clients are not willing to provide reasonable OSINT budgets today.

[1] Application program interface (API) specifies how software components should interact, ie. a search interface.








Self-defence in Jail

In July 2016, Ontario Superior court Judge Edward Morgan wrote an astounding judgement in favour of self-defence in R. vs. Michael Short.

Short shivved an assailant in a provincial jail. While Short is a violent gang member, the Judge understood that he had the right to defend himself with a weapon, even in a jail. This Judge understood the poorly managed jails offered no protection to inmates facing unprovoked attacks.

Let’s hope more judges exhibit this level of understanding when faced with prosecutions of ordinary citizens who are forced to defend themselves.

When the Security Guard’s Job Stops

AttackThe fifth annual Horizon Scan Report published by the Business Continuity Institute, in association with BSI illustrates that physical security and related issues are growing concerns amongst business leaders. This renewed interest appears in studies and surveys throughout the industrialized world.

My own recent experience in Canada includes many executives asking questions about what they can do to prevent and manage active shooters, gang violence in their facilities, and terrorist attacks. Of course, they demand secrecy to surround their queries and the answers they receive. If I were to summarise the questions, they would display a surprising lack of knowledge about violence and Canadian law. I know the answers surprise the enquirer due to his reaction upon learning how helpless he is in the face of such low-probability but high-consequence threats. What follows should help to explain the most fundamental causes of, and reasons for, our inability to deal with these threats.

The recent awards for bravery related to the October 2014 attack on parliament hill should highlight our society’s irrational approach to managing armed attacks in public and private work places.

The outcome of this attack informs us that we cannot stop attackers at the front door due to our irrational aversion to armed security guards. That is apparent from the utterly inept response to the attack on parliament hill.

Most Canadian security operations stop short of actually managing an armed attack. Once something violent or dangerous starts, the normal response entails calling on somebody else to do the heavy lifting. In this organisational culture, when an attack starts, the security guard’s job stops. However, calling the police is not an emergency response procedure; it is an act of desperation and an admission of incompetence.

With this entrenched mindset, it does not matter how many resources have been devoted to the security operation, when an armed attack begins, security guards, employees, or guests will suffer serious injury or death.

Sign-in procedures, searches, and metal detectors have limited utility when violent intruders come calling. Intruders like this will not calmly line-up and politely follow orders.

The notion that technology and security theatre can supplant incompetence is common in the chancelleries that extoll the virtues of their most recent purchasing decision, but those worthies never face armed terrorists, gangsters, or homicidal lunatics themselves. On the other hand, unarmed guards exposed to armed intruders have a limited number of responses: run, hide, attempt moving people away from the attacker, die in place, or confront the attacker. As illustrated by the attack on parliament hill, unarmed guards are utterly ineffective in the last response option.

Most Canadians do not understand that self-defence is not so much a right as it is a defence in law used to enrich lawyers through endless prosecution and litigation. As a result, the government has embraced the union-shop mentality that sees the preservation of life and self-defence as something only government bureaucrats may do under the supposed ‘social contract’ and nobody has the money, power, and the perseverance needed to change this mindset. Demonstrating this needless and restrictive attitude is the fact that security guards may not get a pistol permit to defend life and limb; they may only get one to protect money. This promotes the perverse belief that the private sector is more interested in money than lives. Even worse, it demonstrates that our government does not believe that any class of private citizen should actually have the right to defend themselves.

Explaining to a public official or company manager that this aversion to armed security guards is irrational does not change his viewpoint but rather creates an enemy. Decades of propaganda and indoctrination against firearms ownership and the right to self-defence has produced an ignorance and unreasoning terror of weapons, which also manifests itself in the firm belief that only government bureaucrats have some magical ability to use weapons. Explaining,  if that were the case, then management of the parliament hill attack would have been quite different does not make any friends either.

In the 2014 Ottawa attack, the police did not sit on their hands outside as they did at the École Polytechnique shooting in 1989. Instead, they advanced to contact rather than waiting outside for specialized response units. This is termed Immediate Action Rapid Deployment (IARD), which is a fancy acronym for common sense.

The IARD protocol is to swiftly locate and close in on the attacker(s) to neutralize the menace at the earliest opportunity, thereby preventing further mayhem. However, this protocol has one critical flaw—the time between recognising the problem and having someone come by to resolve it. This delay causes further casualties. Would it not be more effective to stop or disrupt the attackers plan at the door? Should the attackers make it past the front door, would it not be more effective if on-site security personnel immediately employed the IRAD protocol rather than wait for police to arrive?

The federal government is slowly addressing these issues on parliament hill but do not expect any provisions for the private sector to address the very same threats.

CPIC only reports indictable and hybrid offences

Canadian Police Information Centre

In Canada, a criminal record is a documented guilty conviction with registration of the offenders name in CPIC (Canadian Police Information Centre).

CPIC Content

“Canada’s repository of criminal records relates to individuals that have been charged with indictable and/or hybrid offences. Since the Identification of Criminals Act only allows the taking of fingerprints in relation to indictable or hybrid offences, and the RCMP National Repository of Criminal Records is fingerprint-based, the National Repository only contains information relating to these two categories of offences. Summary conviction offences are only included in the National Repository if submitted as part of an occurrence involving an indictable or hybrid offence.” [source:  (20 Jan 16)]

Hybrid Offences

Hybrid Offences or Dual-Procedure Offences may proceed as either summary conviction offences or indictable offences. The Crown chooses the mode of prosecution but usually prosecutes the less serious of these as summary conviction offences. The crown may proceed on the hybrid offences as more serious indictable offences when the  circumstances make the crime more serious.

Phone Numbers on the Web

The Phone Archive  says it searches USA based phone numbers usages and context snippets on webpages and documents found on the Web. This is operated by the same folks that run The Email Archive that I found less that useful earlier this week. This site is much more useful.

While they advertise this as searching US based phone numbers I found it useful for finding references to any phone number in the NA numbering plan. I found numbers in Canadian, Panama, and Caribbean islands.

I haven’t compared results to the large search engines, but this is a useful resource.

Canadian Criminal Court Documents

The following lists the court documents that you should order when reviewing an accused’s involvement in a criminal prosecution in Canada.

In Canada, the charges are contained in the ‘Information‘. A person must swear under oath that the information about the crimes committed is true. This document usually contains a list of appearances and a synopsis of the verdict. It also identifies the victim and any co-accused.

The Bail or Recognisance will explain the conditions of the accused’s pretrial release. This may identify the sureties, where the accused must live, and other conditions such a a prohibition of having weapons.

Search warrants are a treasure trove of useful information because the police will meticulously explain their need for the warrant. However, court staff often try to prevent you access to these, but they are public record unless sealed by a court order.

Probation orders are like the Bail document in that they set out conditions. However, they also may indicate where the subject lived. In some cases, the probation order will be sent to another province. In that case, you know that during his probation, he was living in that province and a search of criminal court records in that province is indicated to see if he abided by the conditions of his probation.

Exhibits also represent a valuable source of information. Once a case is concluded, you may view the exhibits. Like search warrants, the court staff often tries to deny your access to exhibits. Persevere and demand access to the exhibits and you will eventually get to view them.

King John’s Ontario

Ontario wants to launch the Administrative Monetary Penalty (AMP) system. It’s a cute name for an extortion racket.

AMP will treat Highway Traffic Act (HTA) offences as a tax that you must pay. The accused cannot contest the charge; only discuss the amount of the penalty or perhaps the number of demerit points. This discussion will occur online with an ‘independent arbiter’.

The arbiter isn’t there to provide justice. You’re already guilty—you can only discuss the amount of the penalty. The money goes to the municipality and the municipality employs the so-called ‘independent arbiter’. The independence is a fiction.

The entire thing is an effort to bilk drivers. The government knows we must drive vehicles to exist in Ontario. Economists call this an inelastic demand. In such a demand, the quantity demanded is the same at any price because we must have it, and therefore, it may be taxed at any rate. The provincial government creates this tax by replacing the judicial process with automatic convictions and arbiters with a quota to meet—true government efficiency at last!

In 2011, the Law Society of Upper Canada specifically told the Law Commission of Ontario that AMP was not appropriate for HTA offences. The Ontario Para Legal Association rightly calls this an egregious violation of our legal rights. In rebuttal, the Ontario government imperiously states that there was a six-week public consultation about AMP that ended a couple of months ago, but I never heard of it and I haven’t found anybody else who heard about it either–some public consultation that was.

This will cause a drastic increase in the cost of insurance for residents of rent-seeking municipalities, as they will acquire artificially bad driver’s records. The term rent-seeking isn’t typically applied to government but I don’t see any alternative. Rent-seeking is seeking to increase your share of existing wealth by using the political process while not creating any new wealth. A rent-seeking government uses its discretionary and legislated authority to extract ‘rent’ for its own benefit.

What economists might call ‘rent-seeking’ is a coercive extortion racket, plain and simple. King John would feel a deep kinship with today’s Ontario government, since this type of behaviour brought about the Magna Carta eight hundred years ago.

UPDATE–1 May 2016:

Ontario scraps idea to take traffic ticket system out of the courts

“Ontario has scrapped a proposal to have people pay traffic tickets online or dispute them outside of court.”


How to be a Facebook Spy

If you need access to someone’s Facebook profile this is how to accomplish that task.

Set up an appealing Facebook account, then request to be friends of some people friended by the subject. Wait until some of them accept your friend request. With mutual friends in hand, request to be the subject’s Facebook friend. The subject will see that you have mutual friends and he should accept you as a friend. Then you have access to his profile, photos, postings, and perhaps you may find what you need. However, there are a few legal issues to consider.

If you are an Investigator, and your subject is represented, then asking permission to see his or her page is contact with a represented litigant. In Canada, if the opposing litigant is represented by council, then you may not contact him or her in person, by telephone, or electronically. In most cases you have to ask to be listed as a friend to view the subject’s Facebook page. Doing this will be considered improperly making contact with the litigant and whatever you find will be deemed inadmissible.

However, what you find in Google, other search engines, and unrelated Facebook pages may be used as the basis for a motion for the production of the subject’s entire Facebook page as happened in KOURTESIS V. JORIS (2007) O.J. No. 2677 (Sup. Ct.).

Free Corporate Searches

In Canada, 10 provinces, 3 territories, and the federal government allow the formation of corporations. Only four of ten provinces and the federal government make corporate filings available  online at no cost, these sites are as follows:

Only the federal corporation site allows searching by a director name (use site: command in Google). Only Alberta and Quebec report share holders.

The only free search for officer and directors are OpenCorporates and Neither of these can be relied upon to have all Canadian corporations or up-to-date databases.

Drones and the PI

The use of an unmanned aircraft (UAV) or drone to conduct surveillance is contentious public issue when government does it. When the private sector does it, it is particularly contentious.

As a speaker at a training event in Toronto, Ontario, I was asked about using UAVs for surveillance. This surprised me, as these were experienced private investigators. What follows was my answer to these questions.

If a private investigator intrudes into an area where the subject has a reasonable expectation of privacy and takes pictures and video, then that material is likely to be excluded by any court in Canada. The investigator must respect the Criminal Code as well as all municipal, provincial, and territorial laws regarding trespassing and privacy. The investigator may also face criminal charges or civil suit. A civil suit will name everybody even remotely associated with the sordid affair. These consequences pale in the face of what will happen next.

When a UAV is used for work done for hire and reward, as in a private investigation, a Special Operation Flight Certificate (SFOC) from Transport Canada is required. Aeronautics Act defines hire and reward as “any payment, consideration, gratuity or benefit, directly or indirectly charged, demanded, received or collected by any person for the use of an aircraft.”

The Canadian Air Regulations (CAR) Section 602.41 states that no person shall operate an unmanned air vehicle in flight except in accordance with a Special Flight Operation Certificate. Any violation of the CAR may result in substantial penalties: up to $5000 for an individual and $25,000 for a corporation. The UAV operator bears civil liability if property damage or injury occurs. If the video or image evidence was gathered in contravention of CAR do you think any court would allow the material in evidence? If the court did allow it, would the rest of your evidence be credible?

It takes 20 days to get a SFOC for each flight. Do you think the Transport Canada would even consider giving a private investigator such a permit? Can you plan your surveillance 20 days in advance?

In the U.S.A., commercial operation of a UAV it is still illegal. The Federal Aviation Administration (FAA) is considering allowing commercial UAV use in 2015.

Alberta Court Stikes Down Trespass to Premises Act

An Alberta Court struck down the provincial Trespass to Premises Act (TPA) as “unconstitutional” as it relates to public property in R v S.A. This decision prevents Transit Authorities across Alberta from using the Trespass to Premises Act to ban individuals from using their facilities.

R v S.A was about a young woman who was banned from all Edmonton LRT stations due to her involvement in an altercation at a single station. This eliminated her ability use public transit in Edmonton.

This is a long and thoughtful decision addresses the Liberty interest found in S.7 of the Charter. On reading the decision, I believe this decision will, over time, extend to all public places where the Trespass to Premises Act might be used by any public authority in Alberta.

Since 1976, Canadian courts have been whittling away at the right of private property owners to keep out trespassers under provincial trespass legislation. The reasoning presented in this decision may become the norm throughout Canada and it may have unforeseen implications for private landowners.

Security and facility management should begin reviewing trespass policies, operating practices, and training in the light of the direction and standards outlined in this case. It seems that the prudent course is to ensure trespass bans are objectively defensible and proportionate to the inappropriate behaviour. Implementing an appeal process for a trespass ban also seems judicious.

Please note that this decision is currently under appeal. It is also from a Provincial Court and not binding. However, understand that landowners rarely get expanded rights from the courts; it usually goes in the other direction.

Black Suits & Dark Glasses

I went to a meeting with a client to help solve a problem one of his customers was having. Sitting in the reception area, I witnessed a wondrous spectacle. In struts a guy in a black suit wearing dark glasses and earwig. He looks around ominously and then talks into his sleeve. Next, the great man enters surrounded by a phalanx of black suits, dark glasses, and earwigs. This is Canada. Private bodyguards don’t exist here. They are just for show-offs who like to look important or for those tricked into hiring some feckless cannon fodder.

It turned out that this was the guy with the problem. My client realised that he was ineptly handling the symptoms rather than treating the disease. He had received threats. He had suffered vandalism to his home and car. He couldn’t in any way identify his persecutor. However, he was a senior executive of a company in an industry that sometimes attracts threats and acts of violence.

When the great man was asked how he had received the threat, he said that he received it on his unlisted cell phone, so it must be a serious threat perpetrated by dangerous people. I Googled the cell phone number. Lo and behold, there it was in a Kijiji ad for some stuff he was selling. The picture of the stuff included the front of his house and enough of his car to identify its make, model, and colour.

His name appeared on the title for his house along with that of his wife. Searching his name in social media sites, I was able to identify his children and wife. I found that his son went to hockey practice at the arena where his car was vandalised.

In half an hour I learned where he lived, his cell phone number, identified his family, where his children went to school, and learned his son’s hockey schedule. More importantly, the social media content related to his family members also identified him. This led me to conclude that it was possible that he was not the target. Of course, the wife and kids didn’t have bodyguards.

Each of his bodyguards was questioned regarding their training and experience. It wasn’t surprising to me that they were repurposed security guards with no training. The agency providing the bodyguards did not conduct any investigation nor did the client’s employer.

Without any idea who in the family was being targeted, new security arrangements were made. The house and office got uniformed security guards. The client and his wife got reliable security drivers. We put in place new security arrangements for the children. All social media content was expunged. I ensured that the police and telephone company became involved.

Further investigation produced a list of suspects. The police tied one of these to the vandalism of the client’s car. Police interrogation led to a confession. The offender turned out to be the teenage daughter’s jilted suitor who was also a player on a rival hockey team.