supercomputer predicts revolution through 'culturomics'

from singularity hub: A new type of software has been shown to predict revolutions by mining news reports around the world. Retrospectively mining the news for the past 30 years the software indicates points at which the likelihood for a revolution is high. When put to the test – bingo! – the software showed spikes just before the recent Egyptian and Libyan upheavals. It was also able to sift through world news to retrospectively pinpoint Osama Bin Ladin’s location to within 200 km. In the emerging science of ‘culturomics’ that tracks cultural trends through the written word, the software was the first to demonstrate that news coverage can be used to predict future events.

The software sifted through news reports from nearly every country in the world. Major sources were from the global news databases such the US government-run Open Source Center which provides foreign open source intelligence, Britain’s equivalent BBC Monitoring, as well as the New York Times’ archive that dates back to 1945. In all, the body of data included over 100 million news articles. The story elements were woven together into a mind-boggling web of 100 trillion relationships. To crunch the massive amount of data the SGI Altix supercomputer Nautilus was enlisted. Its 1024 Intel Nehalem cores give it a total processing power of 8.2 teraflops (trillions of floating point operations per second).

The strategy for pulling out relevant information included two main techniques. ‘Sentiment mining’ involves counting the number of words in a document that are categorized as positive, such as “good” or “nice,” or negative, such as “terrible” or “horrific.” Changes in the tone of a regions’ news documents over time correlates with the sentiment of the people in that region. A major dip in tone, as when the frequency of negative terms rapidly increases, means the natives are getting restless. An unprecedented dip could spell revolution. The second technique, called ‘full-text geocoding,’ matches sentiment to geographic locations.