3 September 2014 in Security Informatics
We demonstrate how one can generate predictions for several thousand incidents of Latin American civil unrest, often many days in advance, by surfacing informative public posts available on Twitter and Tumblr. The data mining system presented here runs daily and requires no manual intervention. Identification of informative posts is accomplished by applying multiple textual and geographic filters to a high-volume data feed consisting of tens of millions of posts per day which have been flagged as public by their authors. Predictions are built by annotating the filtered posts, typically a few dozen per day, with demographic, spatial, and temporal information. Key to our textual filters is the fact that social media posts are necessarily short, making it possible to easily infer topic by simply searching for comentions of typically unrelated terms within the same post (e.g. a future date comentioned with an unrest keyword). Additional textual filters then proceed by applying a logistic regression classifier trained to recognize accounts belonging to organizations who are likely to announce civil unrest. Geographic filtering is accomplished despite sparsely available GPS information and without relying on sophisticated natural language processing. A geocoding technique which infers non-GPS-known user locations via the locations of their GPS-known friends provides us with location estimates for 91,984,163 Twitter users at a median error of 6.65km. We show that announcements of upcoming events tend to localize within a small geographic region, allowing us to forecast event locations which are not explicitly mentioned in text. We annotate our forecasts with demographic information by searching the collected posts for demographic specific keywords generated by hand as well as with the aid of DBpedia. Our system has been in production since December 2012 and, at the time of this writing, has produced 4,771 distinct forecasts for events across ten Latin American nations. Manual examination of 2,859 posts surfaced by our method revealed that only 108 were discussing topics unrelated to civil unrest. Examination of 2,596 forecasts generated between 2013-07-01 and 2013-11-30 found 1,192 (45.9%) matched exactly the date and within a 100 km radius of a civil unrest event reported in traditional news media.
30 March 2015 in Scientific Reports
Critical events in society or biological systems can be understood as large-scale self-emergent phenomena due to deteriorating stability. We often observe peculiar patterns preceding these events, posing a question of—how to interpret the self-organized patterns to know more about the imminent crisis. We start with a very general description — of interacting population giving rise to large-scale emergent behaviors that constitute critical events. Then we pose a key question: is there a quantifiable relation between the network of interactions and the emergent patterns? Our investigation leads to a fundamental understanding to: 1. Detect the system's transition based on the principal mode of the pattern dynamics; 2. Identify its evolving structure based on the observed patterns. The main finding of this study is that while the pattern is distorted by the network of interactions, its principal mode is invariant to the distortion even when the network constantly evolves. Our analysis on real-world markets show common self-organized behavior near the critical transitions, such as housing market collapse and stock market crashes, thus detection of critical events before they are in full effect is possible.
4 November 2014 in EPJ Data Science
25 September 2011 in Annual Conference of the PHM Society
As electrical and electronic systems (EES) steadfastly increase their functional complexity and connectedness, they pose ever-growing challenges in fault analysis and prevention. Many EES faults are intermittent, emerging (new faults), or cascading, and cannot be addressed by the traditional component-level diagnostic design. Leveraging the latest advancements in Network Science, we take the holistic approach to model and analyze the highly interrelated in-vehicle EES as layered sub-networks of hardware components, software components, and communication links. We develop multi-partite, multi-attribute betweenness centrality measures to quantify the complexity and maintainability of the layered EES network. We then use the betweenness centrality distribution to identify fault analysis monitoring points and fault-mitigation strategies. The promising results obtained by our initial empirical study of an example in-vehicle EES presents a first step toward network-theory based IVHM.