Human Behavior Analysis from Video Data Using Bag-of-Gestures

Human Behavior Analysis in Uncontrolled Environments can be categorized in two main challenges: 1) Feature extraction and 2) Behavior analysis from a set of corporal language vocabulary. In this work, we present our achievements characterizing some simple behaviors from visual data on different real applications and discuss our plan for future work: low level vocabulary definition from bag-of-gesture units and high level modelling and inference of human behaviors

Machine Understanding of Human Behavior

A widely accepted prediction is that computing will move to the background, weaving itself into the fabric of our everyday living spaces and projecting the human user into the foreground. If this prediction is to come true, then next generation computing, which we will call human computing, should be about anticipatory user interfaces that should be human-centered, built for humans based on human models. They should transcend the traditional keyboard and mouse to include natural, human-like interactive functions including understanding and emulating certain human behaviors such as affective and social signaling. This article discusses a number of components of human behavior, how they might be integrated into computers, and how far we are from realizing the front end of human computing, that is, how far are we from enabling computers to understand human behavior

Probability-Based Dynamic Time Warping for Gesture Recognition on RGB-D Data

Dynamic Time Warping (DTW) is commonly used in gesture recognition tasks in order to tackle the temporal length variability of gestures. In the DTW framework, a set of gesture patterns are compared one by one to a maybe infinite test sequence, and a query gesture category is recognized if a warping cost below a certain threshold is found within the test sequence. Nevertheless, either taking one single sample per gesture category or a set of isolated samples may not encode the variability of such gesture category. In this paper, a probability-based DTW for gesture recognition is proposed. Different samples of the same gesture pattern obtained from RGB-Depth data are used to build a Gaussian-based probabilistic model of the gesture. Finally, the cost of DTW has been adapted accordingly to the new model. The proposed approach is tested in a challenging scenario, showing better performance of the probability-based DTW in comparison to state-of-the-art approaches for gesture recognition on RGB-D data

Social sensing: Obesity, unhealthy eating and exercise in face-to-face networks

What is the role of face-to-face interactions in the diffusion of health-related behaviors- diet choices, exercise habits, and long-term weight changes? We use co-location and communication sensors in mass-market mobile phones to model the diffusion of health-related behaviors via face-to-face interactions amongst the residents of an undergraduate residence hall during the academic year of 2008--09. The dataset used in this analysis includes bluetooth proximity scans, 802.11 WLAN AP scans, calling and SMS networks and self-reported diet, exercise and weight-related information collected periodically over a nine month period. We find that the health behaviors of participants are correlated with the behaviors of peers that they are exposed to over long durations. Such exposure can be estimated using automatically captured social interactions between individuals. To better understand this adoption mechanism, we contrast the role of exposure to different sub-behaviors, i.e., exposure to peers that are obese, are inactive, have unhealthy dietary habits and those that display similar weight changes in the observation period. These results suggest that it is possible to design self-feedback tools and real-time interventions in the future. In stark contrast to previous work, we find that self-reported friends and social acquaintances do not show similar predictive ability for these social health behaviors.United States. Army Research Office (Award Number FA9550-08-1- 0132)United States. Army Research Laboratory (Cooperative Agreement Number W911NF-09-2-0053)United States. Air Force Office of Scientific Research (Award Number FA9550-10-1-0122

When the Social Meets the Semantic: Social Semantic Web or Web 2.5

The social trend is progressively becoming the key feature of current Web understanding (Web 2.0). This trend appears irrepressible as millions of users, directly or indirectly connected through social networks, are able to share and exchange any kind of content, information, feeling or experience. Social interactions radically changed the user approach. Furthermore, the socialization of content around social objects provides new unexplored commercial marketplaces and business opportunities. On the other hand, the progressive evolution of the web towards the Semantic Web (or Web 3.0) provides a formal representation of knowledge based on the meaning of data. When the social meets semantics, the social intelligence can be formed in the context of a semantic environment in which user and community profiles as well as any kind of interaction is semantically represented (Semantic Social Web). This paper first provides a conceptual analysis of the second and third version of the Web model. That discussion is aimed at the definition of a middle concept (Web 2.5) resulting in the convergence and integration of key features from the current and next generation Web. The Semantic Social Web (Web 2.5) has a clear theoretical meaning, understood as the bridge between the overused Web 2.0 and the not yet mature Semantic Web (Web 3.0).Pileggi, SF.; Fernández Llatas, C.; Traver Salcedo, V. (2012). When the Social Meets the Semantic: Social Semantic Web or Web 2.5. Future Internet. 4(3):852-854. doi:10.3390/fi4030852S85285443Chi, E. H. (2008). The Social Web: Research and Opportunities. Computer, 41(9), 88-91. doi:10.1109/mc.2008.401Bulterman, D. C. A. (2001). SMIL 2.0 part 1: overview, concepts, and structure. IEEE Multimedia, 8(4), 82-88. doi:10.1109/93.959106Boll, S. (2007). MultiTube--Where Web 2.0 and Multimedia Could Meet. IEEE Multimedia, 14(1), 9-13. doi:10.1109/mmul.2007.17Fraternali, P., Rossi, G., & Sánchez-Figueroa, F. (2010). Rich Internet Applications. IEEE Internet Computing, 14(3), 9-12. doi:10.1109/mic.2010.76Lassila, O., & Hendler, J. (2007). Embracing «Web 3.0». IEEE Internet Computing, 11(3), 90-93. doi:10.1109/mic.2007.52Dikaiakos, M. D., Katsaros, D., Mehra, P., Pallis, G., & Vakali, A. (2009). Cloud Computing: Distributed Internet Computing for IT and Scientific Research. IEEE Internet Computing, 13(5), 10-13. doi:10.1109/mic.2009.103Mangione-Smith, W. H. (1998). Mobile computing and smart spaces. IEEE Concurrency, 6(4), 5-7. doi:10.1109/4434.736391Greaves, M. (2007). Semantic Web 2.0. IEEE Intelligent Systems, 22(2), 94-96. doi:10.1109/mis.2007.40Bojars, U., Breslin, J. G., Peristeras, V., Tummarello, G., & Decker, S. (2008). Interlinking the Social Web with Semantics. IEEE Intelligent Systems, 23(3), 29-40. doi:10.1109/mis.2008.50Definition of Web 2.0http://www.oreillynet.com/pub/a/oreilly/tim/news/2005/09/30/what-is-web-20.htmlZhang, D., Guo, B., & Yu, Z. (2011). The Emergence of Social and Community Intelligence. Computer, 44(7), 21-28. doi:10.1109/mc.2011.65Pentlan, A. (2005). Socially aware, computation and communication. Computer, 38(3), 33-40. doi:10.1109/mc.2005.104Staab, S., Domingos, P., Mika, P., Golbeck, J., Li Ding, Finin, T., … Vallacher, R. R. (2005). Social Networks Applied. IEEE Intelligent Systems, 20(1), 80-93. doi:10.1109/mis.2005.16The Semantic Webhttp://www.scientificamerican.com/article.cfm?id=the-semantic-webDecker, S., Melnik, S., van Harmelen, F., Fensel, D., Klein, M., Broekstra, J., … Horrocks, I. (2000). The Semantic Web: the roles of XML and RDF. IEEE Internet Computing, 4(5), 63-73. doi:10.1109/4236.877487OWL Web Ontology Language Overviewhttp://www.w3.org/TR/owl-features/Vetere, G., & Lenzerini, M. (2005). Models for semantic interoperability in service-oriented architectures. IBM Systems Journal, 44(4), 887-903. doi:10.1147/sj.444.0887Fensel, D., & Musen, M. A. (2001). The semantic web: a brain for humankind. IEEE Intelligent Systems, 16(2), 24-25. doi:10.1109/mis.2001.920595Shadbolt, N., Berners-Lee, T., & Hall, W. (2006). The Semantic Web Revisited. IEEE Intelligent Systems, 21(3), 96-101. doi:10.1109/mis.2006.62Dodds, P. S., & Danforth, C. M. (2009). Measuring the Happiness of Large-Scale Written Expression: Songs, Blogs, and Presidents. Journal of Happiness Studies, 11(4), 441-456. doi:10.1007/s10902-009-9150-9Pang, B., & Lee, L. (2008). Opinion Mining and Sentiment Analysis. Foundations and Trends® in Information Retrieval, 2(1–2), 1-135. doi:10.1561/1500000011Thelwall, M., Buckley, K., & Paltoglou, G. (2011). Sentiment strength detection for the social web. Journal of the American Society for Information Science and Technology, 63(1), 163-173. doi:10.1002/asi.21662Blogmeterhttp://www.blogmeter.it/Christakis, N. A., & Fowler, J. H. (2010). Social Network Sensors for Early Detection of Contagious Outbreaks. PLoS ONE, 5(9), e12948. doi:10.1371/journal.pone.0012948Jansen, B. J., Zhang, M., Sobel, K., & Chowdury, A. (2009). Twitter power: Tweets as electronic word of mouth. Journal of the American Society for Information Science and Technology, 60(11), 2169-2188. doi:10.1002/asi.21149Bernal, P. A. (2010). Web 2.5: The Symbiotic Web. International Review of Law, Computers & Technology, 24(1), 25-37. doi:10.1080/13600860903570145Mikroyannidis, A. (2007). Toward a Social Semantic Web. Computer, 40(11), 113-115. doi:10.1109/mc.2007.405Jung, J. J. (2012). Computational reputation model based on selecting consensus choices: An empirical study on semantic wiki platform. Expert Systems with Applications, 39(10), 9002-9007. doi:10.1016/j.eswa.2012.02.03

The Centrality of Processes in IT-Enabled Decisions

We use detailed data from individuals in multiple organizations to examine the role of information systems (IS) use for decision support, formal processes and social interactions on individual decision making. We found, contrary to our expectations, that the extent of IS use for decision making alone did not significantly influence individual decision making efficacy. However, the extent to which a decision process is defined and operationalized has a significant positive effect on decision making efficacy. In addition, social interactions also influence decision making positively. The joint effect of decision process and social interactions offers a clue to how decision support systems (DSS) can be designed and operationalized. These results not only extend our theoretical understanding of the role of DSSs, they offer insights on how structural improvements can be made to leverage decision making capabilities of individuals in organizations

Attention-Block Deep Learning Based Features Fusion in Wearable Social Sensor for Mental Wellbeing Evaluations

With the progressive increase of stress, anxiety and depression in working and living environment, mental health assessment becomes an important social interaction research topic. Generally, clinicians evaluate the psychology of participants through an effective psychological evaluation and questionnaires. However, these methods suffer from subjectivity and memory effects. In this paper, a new multi- sensing wearable device has been developed and applied in self-designed psychological tests. Speech under different emotions as well as behavior signals are captured and analyzed. The mental state of the participants is objectively assessed through a group of psychological questionnaires. In particular, we propose an attention-based block deep learning architecture within the device for multi-feature classification and fusion analysis. This enables the deep learning architecture to autonomously train to obtain the optimum fusion weights of different domain features. The proposed attention-based architecture has led to improving performance compared with direct connecting fusion method. Experimental studies have been carried out in order to verify the effectiveness and robustness of the proposed architecture. The obtained results have shown that the wearable multi-sensing devices equipped with the attention-based block deep learning architecture can effectively classify mental state with better performance