Predictive Analysis for Social Processes II: Predictability and Warning Analysis

This two-part paper presents a new approach to predictive analysis for social processes. Part I identifies a class of social processes, called positive externality processes, which are both important and difficult to predict, and introduces a multi-scale, stochastic hybrid system modeling framework for these systems. In Part II of the paper we develop a systems theory-based, computationally tractable approach to predictive analysis for these systems. Among other capabilities, this analytic methodology enables assessment of process predictability, identification of measurables which have predictive power, discovery of reliable early indicators for events of interest, and robust, scalable prediction. The potential of the proposed approach is illustrated through case studies involving online markets, social movements, and protest behavior

Predictive Analysis for Social Processes I: Multi-Scale Hybrid System Modeling

This two-part paper presents a new approach to predictive analysis for social processes. In Part I, we begin by identifying a class of social processes which are simultaneously important in applications and difficult to predict using existing methods. It is shown that these processes can be modeled within a multi-scale, stochastic hybrid system framework that is sociologically sensible, expressive, illuminating, and amenable to formal analysis. Among other advantages, the proposed modeling framework enables proper characterization of the interplay between the intrinsic aspects of a social process (e.g., the appeal of a political movement) and the social dynamics which are its realization; this characterization is key to successful social process prediction. The utility of the modeling methodology is illustrated through a case study involving the global SARS epidemic of 2002-2003. Part II of the paper then leverages this modeling framework to develop a rigorous, computationally tractable approach to social process predictive analysis

Predictability in an unpredictable artificial cultural market

In social, economic and cultural situations in which the decisions of individuals are influenced directly by the decisions of others, there appears to be an inherently high level of ex ante unpredictability. In cultural markets such as films, songs and books, well-informed experts routinely make predictions which turn out to be incorrect. We examine the extent to which the existence of social influence may, somewhat paradoxically, increase the extent to which winners can be identified at a very early stage in the process. Once the process of choice has begun, only a very small number of decisions may be necessary to give a reasonable prospect of being able to identify the eventual winner. We illustrate this by an analysis of the music download experiments of Salganik et.al. (2006). We derive a rule for early identification of the eventual winner. Although not perfect, it gives considerable practical success. We validate the rule by applying it to similar data not used in the process of constructing the rule

Early Warning Analysis for Social Diffusion Events

There is considerable interest in developing predictive capabilities for social diffusion processes, for instance to permit early identification of emerging contentious situations, rapid detection of disease outbreaks, or accurate forecasting of the ultimate reach of potentially viral ideas or behaviors. This paper proposes a new approach to this predictive analytics problem, in which analysis of meso-scale network dynamics is leveraged to generate useful predictions for complex social phenomena. We begin by deriving a stochastic hybrid dynamical systems (S-HDS) model for diffusion processes taking place over social networks with realistic topologies; this modeling approach is inspired by recent work in biology demonstrating that S-HDS offer a useful mathematical formalism with which to represent complex, multi-scale biological network dynamics. We then perform formal stochastic reachability analysis with this S-HDS model and conclude that the outcomes of social diffusion processes may depend crucially upon the way the early dynamics of the process interacts with the underlying network's community structure and core-periphery structure. This theoretical finding provides the foundations for developing a machine learning algorithm that enables accurate early warning analysis for social diffusion events. The utility of the warning algorithm, and the power of network-based predictive metrics, are demonstrated through an empirical investigation of the propagation of political memes over social media networks. Additionally, we illustrate the potential of the approach for security informatics applications through case studies involving early warning analysis of large-scale protests events and politically-motivated cyber attacks

Leveraging Sociological Models for Predictive Analytics

Abstract—There is considerable interest in developing techniques for predicting human behavior, for instance to enable emerging contentious situations to be forecast or the nature of ongoing but “hidden ” activities to be inferred. A promising approach to this problem is to identify and collect appropriate empirical data and then apply machine learning methods to these data to generate the predictions. This paper shows the performance of such learning algorithms often can be improved substantially by leveraging sociological models in their development and implementation. In particular, we demonstrate that sociologically-grounded learning algorithms outperform gold-standard methods in three important and challenging tasks: 1.) inferring the (unobserved) nature of relationships in adversarial social networks, 2.) predicting whether nascent social diffusion events will “go viral”, and 3.) anticipating and defending future actions of opponents in adversarial settings. Significantly, the new algorithms perform well even when there is limited data available for their training and execution. Keywords—predictive analysis, sociological models, social networks, empirical analysis, machine learning. I