From Signal to Social : Steps Towards Pervasive Social Context

The widespread adoption of smartphones with advanced sensing, computing and data transfer capabilities has made scientific studies of human social behavior possible at a previously unprecedented scale. It has also allowed context-awareness to become a natural feature in many applications using features such as activity recognition and location information. However, one of the most important aspects of context remains largely untapped at scale, i.e. social interactions and social context. Social interaction sensing has been explored using smartphones and specialized hardware for research purposes within computational social science and ubiquitous computing, but several obstacles remain to make it usable in practice by applications at industrial scale. In this thesis, I explore methods of physical proximity sensing and extraction of social context information from user-generated data for the purpose of context-aware applications. Furthermore, I explore the application space made possible through these methods, especially in the class of use cases that are characterized by embodied social agency, through field studies and a case study.A major concern when collecting context information is the impact on user privacy. I have performed a user study in which I have surveyed the user attitudes towards the privacy implications of proximity sensing. Finally, I present results from quantitatively estimating the sensitivity of a simple type of context information, i.e. application usage, in terms of risk of user re-identification

Nonverbal Social Sensing: What Social Sensing Can and Cannot Do for the Study of Nonverbal Behavior From Video

The study of nonverbal behavior (NVB), and in particular kinesics (i.e., face and body motions), is typically seen as cost-intensive. However, the development of new technologies (e.g., ubiquitous sensing, computer vision, and algorithms) and approaches to study social behavior [i.e., social signal processing (SSP)] makes it possible to train algorithms to automatically code NVB, from action/motion units to inferences. Nonverbal social sensing refers to the use of these technologies and approaches for the study of kinesics based on video recordings. Nonverbal social sensing appears as an inspiring and encouraging approach to study NVB at reduced costs, making it a more attractive research field. However, does this promise hold? After presenting what nonverbal social sensing is and can do, we discussed the key challenges that researchers face when using nonverbal social sensing on video data. Although nonverbal social sensing is a promising tool, researchers need to be aware of the fact that algorithms might be as biased as humans when extracting NVB or that the automated NVB coding might remain context-dependent. We provided study examples to discuss these challenges and point to potential solutions

Social Isolation and Serious Mental Illness: The Role of Context-Aware Mobile Interventions

Social isolation is a common problem faced by individuals with serious mental illness (SMI), and current intervention approaches have limited effectiveness. This paper presents a blended intervention approach, called mobile Social Interaction Therapy by Exposure (mSITE), to address social isolation in individuals with serious mental illness. The approach combines brief in-person cognitive-behavioral therapy (CBT) with context-triggered mobile CBT interventions that are personalized using mobile sensing data. Our approach targets social behavior and is the first context-aware intervention for improving social outcomes in serious mental illness

Exploring the Linkage of Spatial Indicators from Remote Sensing Data with Survey Data: The Case of the Socio-Economic Panel (SOEP) and 3D City Models

This paper demonstrates the spatial evaluation of survey data from the German Socio-Economic Panel (SOEP) study using geo-coordinates and spatially relevant indicators from remote sensing data. By geocoding the addresses of survey households with block-level geographic precision (while preventing their identification by name and guaranteeingtheir complete anonymity), data on SOEP respondents can now be analyzed in a specific spatial context. In the past, regional analyses of SOEP based on official regional indicators (e.g., the unemployment rate) always had only very imprecise spatial information to work with. This limitation has now been overcome with the geocoded respondents' information. Within a protected unit of the fieldwork organization responsible for SOEP (TNS Infratest, Munich), the addresses of survey households can now be used to generate a variable describing the location of the household with block-level precision. At DIW Berlin, this additional variable is fed into a special computer infrastructure with multiple security layers that makes the socio-economic analysis possible. This paper demonstrates the use of this geographicallocation and remote sensing data to check respondents' subjective assessments of the location of their residence, anddiscusses the analytical potential of linking remote sensing data and survey data.Remote sensing data, social sciences, behavioral sciences, multi-disciplinarity, SOEP

Understanding the Social Context of Eating with Multimodal Smartphone Sensing: The Role of Country Diversity

Understanding the social context of eating is crucial for promoting healthy eating behaviors by providing timely interventions. Multimodal smartphone sensing data has the potential to provide valuable insights into eating behavior, particularly in mobile food diaries and mobile health applications. However, research on the social context of eating with smartphone sensor data is limited, despite extensive study in nutrition and behavioral science. Moreover, the impact of country differences on the social context of eating, as measured by multimodal phone sensor data and self-reports, remains under-explored. To address this research gap, we present a study using a smartphone sensing dataset from eight countries (China, Denmark, India, Italy, Mexico, Mongolia, Paraguay, and the UK). Our study focuses on a set of approximately 24K self-reports on eating events provided by 678 college students to investigate the country diversity that emerges from smartphone sensors during eating events for different social contexts (alone or with others). Our analysis revealed that while some smartphone usage features during eating events were similar across countries, others exhibited unique behaviors in each country. We further studied how user and country-specific factors impact social context inference by developing machine learning models with population-level (non-personalized) and hybrid (partially personalized) experimental setups. We showed that models based on the hybrid approach achieve AUC scores up to 0.75 with XGBoost models. These findings have implications for future research on mobile food diaries and mobile health sensing systems, emphasizing the importance of considering country differences in building and deploying machine learning models to minimize biases and improve generalization across different populations

Dynamics of Information Diffusion and Social Sensing

Statistical inference using social sensors is an area that has witnessed remarkable progress and is relevant in applications including localizing events for targeted advertising, marketing, localization of natural disasters and predicting sentiment of investors in financial markets. This chapter presents a tutorial description of four important aspects of sensing-based information diffusion in social networks from a communications/signal processing perspective. First, diffusion models for information exchange in large scale social networks together with social sensing via social media networks such as Twitter is considered. Second, Bayesian social learning models and risk averse social learning is considered with applications in finance and online reputation systems. Third, the principle of revealed preferences arising in micro-economics theory is used to parse datasets to determine if social sensors are utility maximizers and then determine their utility functions. Finally, the interaction of social sensors with YouTube channel owners is studied using time series analysis methods. All four topics are explained in the context of actual experimental datasets from health networks, social media and psychological experiments. Also, algorithms are given that exploit the above models to infer underlying events based on social sensing. The overview, insights, models and algorithms presented in this chapter stem from recent developments in network science, economics and signal processing. At a deeper level, this chapter considers mean field dynamics of networks, risk averse Bayesian social learning filtering and quickest change detection, data incest in decision making over a directed acyclic graph of social sensors, inverse optimization problems for utility function estimation (revealed preferences) and statistical modeling of interacting social sensors in YouTube social networks.Comment: arXiv admin note: text overlap with arXiv:1405.112

Bacterial Quorum Sensing and Microbial Community Interactions

Many bacteria use a cell-cell communication system called quorum sensing to coordinate population density-dependent changes in behavior. Quorum sensing involves production of and response to diffusible or secreted signals, which can vary substantially across different types of bacteria. In many species, quorum sensing modulates virulence functions and is important for pathogenesis. Over the past half-century, there has been a significant accumulation of knowledge of the molecular mechanisms, signal structures, gene regulons, and behavioral responses associated with quorum-sensing systems in diverse bacteria. More recent studies have focused on understanding quorum sensing in the context of bacterial sociality. Studies of the role of quorum sensing in cooperative and competitive microbial interactions have revealed how quorum sensing coordinates interactions both within a species and between species. Such studies of quorum sensing as a social behavior have relied on the development of “synthetic ecological” models that use nonclonal bacterial populations. In this review, we discuss some of these models and recent advances in understanding how microbes might interact with one another using quorum sensing. The knowledge gained from these lines of investigation has the potential to guide studies of microbial sociality in natural settings and the design of new medicines and therapies to treat bacterial infections

Assessing texture pattern in slum across scales: an unsupervised approach

According to the Global Report on Human Settlements (United Nations, 2003), almost 1 billion people (32% of the world ’s population) live in squatter settlements or slums. Recently, the perception of these settlements has changed, from harmful tumours which would spread around sickly and unhealthy cities, to a new perspective that interpret them as social expressions of more complex urban dynamics. However, considering a report from UNCHS - United Nations Center for Human Settlements, in relation to illegal and disordered urbanisation issue, some of the main challenges faced by cities are related to mapping and registering geographic information and social data spatial analysis. In this context, we present, in this paper, preliminary results from a study that aims to interpret city from the perspective of urban texture, using for this purpose, high resolution remote sensing images. We have developed analytic experiments of "urban tissue" samples, trying to identify texture patterns which could (or could not) represent distinct levels of urban poverty associated to spatial patterns. Such analysis are based on some complex theory concepts and tools, such as fractal dimension and lacunarity. Preliminary results seems to suggest that the urban tissue is fractal by nature, and from the distinct texture patterns it is possible to relate social pattern to spatial configuration, making possible the development of methodologies and computational tools which could generate, via satellite, alternative and complementary mapping and classifications for urban poverty

Anticipatory Mobile Computing: A Survey of the State of the Art and Research Challenges

Today's mobile phones are far from mere communication devices they were ten years ago. Equipped with sophisticated sensors and advanced computing hardware, phones can be used to infer users' location, activity, social setting and more. As devices become increasingly intelligent, their capabilities evolve beyond inferring context to predicting it, and then reasoning and acting upon the predicted context. This article provides an overview of the current state of the art in mobile sensing and context prediction paving the way for full-fledged anticipatory mobile computing. We present a survey of phenomena that mobile phones can infer and predict, and offer a description of machine learning techniques used for such predictions. We then discuss proactive decision making and decision delivery via the user-device feedback loop. Finally, we discuss the challenges and opportunities of anticipatory mobile computing.Comment: 29 pages, 5 figure

Emotions in context: examining pervasive affective sensing systems, applications, and analyses

Pervasive sensing has opened up new opportunities for measuring our feelings and understanding our behavior by monitoring our affective states while mobile. This review paper surveys pervasive affect sensing by examining and considering three major elements of affective pervasive systems, namely; “sensing”, “analysis”, and “application”. Sensing investigates the different sensing modalities that are used in existing real-time affective applications, Analysis explores different approaches to emotion recognition and visualization based on different types of collected data, and Application investigates different leading areas of affective applications. For each of the three aspects, the paper includes an extensive survey of the literature and finally outlines some of challenges and future research opportunities of affective sensing in the context of pervasive computing