The intelligence community gets social by Brian Fung, WSJ

Digital media is mostly about entertainment for some, while for others, the value lies in being able to spread messages to a large audience. But, as many news organizations are discovering, Web 2.0 technologies are as good for listening as they are for broadcasting. The notion of social media as a trend-monitoring tool is spreading — and now U.S. spy agencies are jumping on board.

Intelligence Advanced Research Projects Activity (IARPA), the intelligence community’s research arm, says it hopes to use data gathered from social media to predict political unrest and natural disasters. While the proposal may rankle privacy critics, it’s just the latest example of the way intelligence officials are turning to the social Web to collect policy-relevant information.

The CIA already monitors social networks manually. In 2010, agency analysts became aware of a YouTube account allegedly belonging to the propaganda service of North Korea. Pyongyang soon had other identities set up on Twitter and Facebook (the latter of which was abandoned). The CIA issued several reports later that year on the regime’s entry into social media, concluding that the new Web offensive was primarily aimed at influencing the population of South Korea, one of the world’s most digitally enabled societies. Both countries are engaged in a tenuous military truce and longstanding public relations war.

Even as it was watching North Korea’s evolving positions on social media, the CIA was conducting a study of the social media landscape in India (pdf). Beyond uncovering some fascinating details about the country’s Internet usage patterns, analysts discovered that many of India’s controversial separatist groups were taking advantage of social media tools to advocate their agendas.

Spy agencies’ growing interest in digital media is perhaps unsurprising given that it is an industry that trades in information.. But it also reflects broader, underlying trends in intelligence-gathering. Since the end of the Cold War, U.S. officials have embraced what are called “open sources” — non-classified information drawn from newspapers, radio broadcasts and other publicly accessible outlets. Open sources accounted for some 80 percent (pdf)of what the CIA knew about the Soviet Union’s downfall in the early 1990s, according to then-deputy director William Studeman. Sherman Kent, one of the agency’s first analysts, once estimated (pdf) that 80 percent of all U.S. intelligence needs could be met with open sources in peacetime.

The biggest victory for open-source proponents came in 2005, when the CIA launched a new center dedicated to gleaning intelligence from public information. The announcement signaled more of a rebranding than anything else — open source intelligence has always been a part of the mix to some degree — but the event finally lent recognition and credibility to a historically obscure tradition.

The open-source revolution has only accelerated with social media. Now, analysts can tap directly into millions of individual sources at the micro level, examining tweets, blog posts and videos for new information. They can also step back and survey entire social ecosystems, using vast amounts of metadata to identify significant patterns of behavior in the abstract. Or at the mid-range level, digital media can reveal important connections among small groups of users.

Whether government scrutiny of social media is problematic for civil society depends on your conception of public and private. But it raises other questions, too. What is the intelligence value of an individual tweet? How does the study of social media affect signal-to-noise ratios and, more importantly, how does it affect ways in which the intelligence community allocates its resources to adapt? Does social media change the meaning of open-source intelligence?