Storage Systems — A Must for Researchers

Storage Systems
In addition to two computers, you should have at least one and preferably two large drives to attach to the computer. You will want to save your collected materials somewhere other than on the search computer. A large disk drive is the ideal place. But since drives sometimes crash and the information on them cannot be recovered, having two drives which are mirrors to each other is the safest and most professional route. As Navy SEALS say about virtually every piece of gear: “one is none and two is one.”
Some people in the field believe storing the files in the cloud is a better answer. Cloud computing – storing your files on a mainframe somewhere else – probably is safe and the stored material is backed up so there is relatively little chance of loss.
However, any time you store material with someone else you are sending it through a system where it can be intercepted and traced back to you. If you are gathering sensitive information – whether for government or a private concern – you want to reduce the chance of interception to zero. Cloud computing, for Open Source researchers, is less secure than keeping your collection of downloaded material on your own disk drives. Of course, to prevent loss through fire, flood, theft or other disasters you will want to keep your disk drives stored apart from one another.
The size of the external drives you select will depend on how much material you plan to collect and store over how long a period.Your storage needs will depend in part on what kind of material you are collecting. You will usually need less storage capacity when collecting text articles than you will need if you are collecting multi-media files, for instance. For most current collection needs, professionals find that external drives of 500 Megabytes to 5 Terrabytes are sufficient, These allow for growth of the collection over time.

Reading What you Hear

Audio-to-text transcription and searching is becoming more necessary as the Internet – and some say society as a whole – moves away from the written word and toward other communication forms. This movement began a century ago with the phonograph and accelerated thereafter. Now more than half the digital traffic payload moving across the Internet falls into the audio-visual realm. Internet researchers have to increasingly use, and “translate” into text, this tidal wave of audio-visuals.

At TRS we are finding more uses for the pop up archive at .

This translates a wide variety of file types –  aac, aif, aiff alac, flac, m4a, m4p, mp2,
mp3, mp4, ogg, raw, spx, wav, wma – into searchable text.

Do you have other favorite multi-media-to-text engines and what uses do you find for them?

OSINT Nomenclature for the Intelligence Community

OSINT, as a term, is often badly misunderstood. The word can be, and is, used in two different contexts even by people in the intelligence field who should – and probably do – know better.

OSINT is used as a generic term for almost any type of open source report or information. OSINT is also a specific type of open source resource that is a sub-type of the generic OSINT.

The upshot is that what some refer to as OSINT really isn’t “OSINT.” There are several levels of information complexity. Of these levels, OSINT is only one.

Open Source Data, sometimes abbreviated as OSD, is simply the raw material from a primary source. It is the data received from the original source without any filtering, validation, or efforts at presentation. Usually OSD material will not have been widely disseminated. Typically OSD might be of a letter, photograph, or transcript of conversation.

Open Source Information, abbreviated as OSIF or OSINFO, is collected data that has been run through some type of editorial process to filter, validate, and make an attractive presentation of the material. OSIF will often be material up that has been widely disseminated. Newspapers, books, magazines, and broadcast tapes are usually considered OSIF. Some of the most useful material in the civilian OSINT world is OSIF. Much of what non-experts term OSINT is actually better-defined as OSIF.

Open Source Intelligence (OSINT) is “information that has been deliberately discovered, discriminated, distilled and disseminated to a select audience.” [1] This IS the true OSINT.

Validated OSINT, called OSINT-V is “information to which a very high degree of certainty can be attributed.” In government circles OSINT-V can be produced by an all-source intelligence professional with access to classified intelligence sources where the information can be cross-checked for correctness. It can also come from an assured open source to which no question can be raised concerning its validity (for example—images of an aircraft arriving at an airport that are broadcast over the media).”

This nomenclature comes from the military but is generally understood throughout the intelligence field. Knowledge of the individual types and the differences are vital in understanding the production and usage of OSINT by intelligence organizations.


[1] NATO OSINT Handbook, Page 10