Where does threat intelligence (TI) come from? Yes, we all know that little gnomes living in the clouds make threat intelligence out of seagulls’ poo. But apart from that, where DOES it come from?
Let’s delve into this, starting from the high level. Some types of TI are simply the results of somebody else’s security monitoring. They see it, they share it, you pay them to see it too. The proverbial “they” may have bigger networks, better tools or simply be in that business for hard cold cash. Other TI comes from people who go actively crawl for threats, browse sites for malware, run honeypots, spam traps, sinkholes, etc or even actively “engage” the threat actors and/or infiltrate their ranks. Different processes – different TI (all fitting our broad TI definition).
In general, categorizing the TI sources as “technical” and “human” is a bit artificial, since many of the sources are in essence produced by a tight collaboration of humans AND machines (e.g. a smart TI analyst armed with data visualization and entity linking software) with various fractions of contributions from each. Still, mostly technical sources include:
Network monitoring (NIDS, NBA, etc)
Server and client honeypots [some vendors want to say they use “next gen honeypots” which really means that they use … honeypots 
]
Spam (and phishing email) traps
Live botnet connections
Link crawling for malware and exploit code
Malware reversing and observation
Social network monitoring
BGP observation [ooooh…fun!]
Tor usage monitoring
However, some of the juicier intel does not come from automated tools, but from people (admittedly, equipped with tools). For example, you don’t simply wget a Russian cybercrime forum for future analysis. A human needs to actively work to get in. Similarly, you don’t simply sniff IRC traffic or intercept emails between attackers, a lot of “very human” work is involved before this task can be completed. Some TI vendors whisper about infiltration and “aggressive techniques” (as I suspect, some hacking across state borders aimed at malicious infrastructure may in fact be legal [or at least unlikely to ever be charged] in the source country).
This set of sources includes examples such as:
Public internet, IRC, newsgroup monitoring [crawlers + human analysts/translators]
Attacker online community infiltration (web forum, “private” social media, IRC, etc)
Leaked data about attacker’s infrastructure (such as leaks by other attackers)
Direct compromise of attackers systems.
Furthermore, if you are buying/receiving TI from somebody else (government, regional CERT, sharing club, security vendor, TI vendor) the intel you receive is ultimately produced by a blend of the above sources. Of course, you may also be producing your very own TI from local incident artifacts – a subject of the next post in this series…
Have I missed any major threat intel sources?
Or, better, can you think of a better way to classify and organize them?
Posts related to this research project:
How to Make Better Threat Intelligence Out of Threat Intelligence Data?
On Threat Intelligence Use Cases
On Broad Types of Threat Intelligence
Threat Intelligence is NOT Signatures!
The Conundrum of Two Intelligences!
On Comparing Threat Intelligence Feeds
Starting Threat Intelligence Research