Emergent Leadership Detection Across Datasets

Automatic detection of emergent leaders in small groups from nonverbal behaviour is a growing research topic in social signal processing but existing methods were evaluated on single datasets -- an unrealistic assumption for real-world applications in which systems are required to also work in settings unseen at training time. It therefore remains unclear whether current methods for emergent leadership detection generalise to similar but new settings and to which extent. To overcome this limitation, we are the first to study a cross-dataset evaluation setting for the emergent leadership detection task. We provide evaluations for within- and cross-dataset prediction using two current datasets (PAVIS and MPIIGroupInteraction), as well as an investigation on the robustness of commonly used feature channels (visual focus of attention, body pose, facial action units, speaking activity) and online prediction in the cross-dataset setting. Our evaluations show that using pose and eye contact based features, cross-dataset prediction is possible with an accuracy of 0.68, as such providing another important piece of the puzzle towards emergent leadership detection in the real world.Comment: 5 pages, 3 figure

Boosted Multiple Kernel Learning for First-Person Activity Recognition

Activity recognition from first-person (ego-centric) videos has recently gained attention due to the increasing ubiquity of the wearable cameras. There has been a surge of efforts adapting existing feature descriptors and designing new descriptors for the first-person videos. An effective activity recognition system requires selection and use of complementary features and appropriate kernels for each feature. In this study, we propose a data-driven framework for first-person activity recognition which effectively selects and combines features and their respective kernels during the training. Our experimental results show that use of Multiple Kernel Learning (MKL) and Boosted MKL in first-person activity recognition problem exhibits improved results in comparison to the state-of-the-art. In addition, these techniques enable the expansion of the framework with new features in an efficient and convenient way.Comment: First published in the Proceedings of the 25th European Signal Processing Conference (EUSIPCO-2017) in 2017, published by EURASI

Investigating Social Interactions Using Multi-Modal Nonverbal Features

Every day, humans are involved in social situations and interplays, with the goal of sharing emotions and thoughts, establishing relationships with or acting on other human beings. These interactions are possible thanks to what is called social intelligence, which is the ability to express and recognize social signals produced during the interactions. These signals aid the information exchange and are expressed through verbal and non-verbal behavioral cues, such as facial expressions, gestures, body pose or prosody. Recently, many works have demonstrated that social signals can be captured and analyzed by automatic systems, giving birth to a relatively new research area called social signal processing, which aims at replicating human social intelligence with machines. In this thesis, we explore the use of behavioral cues and computational methods for modeling and understanding social interactions. Concretely, we focus on several behavioral cues in three specic contexts: rst, we analyze the relationship between gaze and leadership in small group interactions. Second, we expand our analysis to face and head gestures in the context of deception detection in dyadic interactions. Finally, we analyze the whole body for group detection in mingling scenarios

Sensing, interpreting, and anticipating human social behaviour in the real world

Low-level nonverbal social signals like glances, utterances, facial expressions and body language are central to human communicative situations and have been shown to be connected to important high-level constructs, such as emotions, turn-taking, rapport, or leadership. A prerequisite for the creation of social machines that are able to support humans in e.g. education, psychotherapy, or human resources is the ability to automatically sense, interpret, and anticipate human nonverbal behaviour. While promising results have been shown in controlled settings, automatically analysing unconstrained situations, e.g. in daily-life settings, remains challenging. Furthermore, anticipation of nonverbal behaviour in social situations is still largely unexplored. The goal of this thesis is to move closer to the vision of social machines in the real world. It makes fundamental contributions along the three dimensions of sensing, interpreting and anticipating nonverbal behaviour in social interactions. First, robust recognition of low-level nonverbal behaviour lays the groundwork for all further analysis steps. Advancing human visual behaviour sensing is especially relevant as the current state of the art is still not satisfactory in many daily-life situations. While many social interactions take place in groups, current methods for unsupervised eye contact detection can only handle dyadic interactions. We propose a novel unsupervised method for multi-person eye contact detection by exploiting the connection between gaze and speaking turns. Furthermore, we make use of mobile device engagement to address the problem of calibration drift that occurs in daily-life usage of mobile eye trackers. Second, we improve the interpretation of social signals in terms of higher level social behaviours. In particular, we propose the first dataset and method for emotion recognition from bodily expressions of freely moving, unaugmented dyads. Furthermore, we are the first to study low rapport detection in group interactions, as well as investigating a cross-dataset evaluation setting for the emergent leadership detection task. Third, human visual behaviour is special because it functions as a social signal and also determines what a person is seeing at a given moment in time. Being able to anticipate human gaze opens up the possibility for machines to more seamlessly share attention with humans, or to intervene in a timely manner if humans are about to overlook important aspects of the environment. We are the first to propose methods for the anticipation of eye contact in dyadic conversations, as well as in the context of mobile device interactions during daily life, thereby paving the way for interfaces that are able to proactively intervene and support interacting humans.Blick, Gesichtsausdrücke, Körpersprache, oder Prosodie spielen als nonverbale Signale eine zentrale Rolle in menschlicher Kommunikation. Sie wurden durch vielzählige Studien mit wichtigen Konzepten wie Emotionen, Sprecherwechsel, Führung, oder der Qualität des Verhältnisses zwischen zwei Personen in Verbindung gebracht. Damit Menschen effektiv während ihres täglichen sozialen Lebens von Maschinen unterstützt werden können, sind automatische Methoden zur Erkennung, Interpretation, und Antizipation von nonverbalem Verhalten notwendig. Obwohl die bisherige Forschung in kontrollierten Studien zu ermutigenden Ergebnissen gekommen ist, bleibt die automatische Analyse nonverbalen Verhaltens in weniger kontrollierten Situationen eine Herausforderung. Darüber hinaus existieren kaum Untersuchungen zur Antizipation von nonverbalem Verhalten in sozialen Situationen. Das Ziel dieser Arbeit ist, die Vision vom automatischen Verstehen sozialer Situationen ein Stück weit mehr Realität werden zu lassen. Diese Arbeit liefert wichtige Beiträge zur autmatischen Erkennung menschlichen Blickverhaltens in alltäglichen Situationen. Obwohl viele soziale Interaktionen in Gruppen stattfinden, existieren unüberwachte Methoden zur Augenkontakterkennung bisher lediglich für dyadische Interaktionen. Wir stellen einen neuen Ansatz zur Augenkontakterkennung in Gruppen vor, welcher ohne manuelle Annotationen auskommt, indem er sich den statistischen Zusammenhang zwischen Blick- und Sprechverhalten zu Nutze macht. Tägliche Aktivitäten sind eine Herausforderung für Geräte zur mobile Augenbewegungsmessung, da Verschiebungen dieser Geräte zur Verschlechterung ihrer Kalibrierung führen können. In dieser Arbeit verwenden wir Nutzerverhalten an mobilen Endgeräten, um den Effekt solcher Verschiebungen zu korrigieren. Neben der Erkennung verbessert diese Arbeit auch die Interpretation sozialer Signale. Wir veröffentlichen den ersten Datensatz sowie die erste Methode zur Emotionserkennung in dyadischen Interaktionen ohne den Einsatz spezialisierter Ausrüstung. Außerdem stellen wir die erste Studie zur automatischen Erkennung mangelnder Verbundenheit in Gruppeninteraktionen vor, und führen die erste datensatzübergreifende Evaluierung zur Detektion von sich entwickelndem Führungsverhalten durch. Zum Abschluss der Arbeit präsentieren wir die ersten Ansätze zur Antizipation von Blickverhalten in sozialen Interaktionen. Blickverhalten hat die besondere Eigenschaft, dass es sowohl als soziales Signal als auch der Ausrichtung der visuellen Wahrnehmung dient. Somit eröffnet die Fähigkeit zur Antizipation von Blickverhalten Maschinen die Möglichkeit, sich sowohl nahtloser in soziale Interaktionen einzufügen, als auch Menschen zu warnen, wenn diese Gefahr laufen wichtige Aspekte der Umgebung zu übersehen. Wir präsentieren Methoden zur Antizipation von Blickverhalten im Kontext der Interaktion mit mobilen Endgeräten während täglicher Aktivitäten, als auch während dyadischer Interaktionen mittels Videotelefonie

Machines as Teammates: A Collaboration Research Agenda

Humans will soon need to adapt to a collaborative setting in which technology becomes a smart collaboration partner that works with a group to achieve its goals. It is therefore time for collaboration researchers to explore the vast opportunities afforded by smart technology and to test its utility for enhancing team processes and outcomes. In this paper, we take a long view on the implications of smart technology for collaboration process design, and propose a research agenda for the next decade of collaboration research. We create a reference model to frame the research agenda

IT quality and organization development: using action research to promote employee engagement, leadership development, learning, and organizational improvement

Within IT areas, Quality is often reduced to visions strongly influenced by operational and tactical instruments, relegating to minor dimensions crucial Organization Development (OD) aspects which sustain Learning, and Innovation. The current Program, grounded on the relevance of these aspects, has targeted, within a Bank’s IT Division, an approach to induce organizational change, and to produce strategic actions and behavioural changes which have led to an effective improvement on Customers, and Employees’ Satisfaction. It has followed an Action Research paradigm – addressing a complex, transformational, planed change, and using a multidimensional, integrative approach, based on a holistic, open systemic view – not targeting for the development of new theories, but, mainly, the fulfilment of existing empirical, and methodological gaps. It has integrated a two-cycle OD approach, where a first cycle focused on Service Culture, Leadership, and Employee Engagement has developed the conditions for a second cycle based on the acquired knowledge (double loop) and devoted to strategy implementation. Although the intervention’s achievements cannot be generalized outside the context, they can be transposed to other settings. They’ve revealed important Management Implications which form the relevance basis for this doctoral dissertation, namely a holistic, values-based, and participative framework to address organizational transformation, and the associated critical success factors. An opportunity exists to further research in the field, linking together an OD approach with a TQM approach to organizational excellence. Also, a metamodel of the Action Research process which has been followed – evidencing, at a conceptual level, the main sub-processes, data groups, and linking points between the action and the research dimensions – has been produced. An opportunity exists for further research on the development of this metamodel, including a conceptual data model and a system behavioural perspective (responding to events).Nas áreas de TI, a Qualidade é frequentemente reduzida a visões fortemente influenciadas por instrumentos táticos e operacionais, menorizando aspetos de Desenvolvimento Organizacional (DO) que são essenciais para sustentar a Aprendizagem e a Inovação. O presente programa, alicerçado na relevância destes aspetos, visou, no contexto da Divisão de TI de um Banco, desenvolver uma aproximação indutora de mudança organizacional; produzindo ações de índole estratégica e mudanças comportamentais; tendo conduzindo a um incremento significativo na Satisfação de Clientes Internos e de Colaboradores. Um primeiro ciclo – focado na Cultura de Serviço, na Liderança e no Envolvimento dos Colaboradores – criou as condições para num segundo ciclo, baseado no conhecimento organizacional adquirido, e nas decisões estratégicas emergentes (“double loop” learning), proceder à respetiva implementação. Foi seguido um paradigma de Investigação-Ação – endereçando uma mudança complexa, transformacional, planeada; usando uma abordagem multidimensional e integrativa; baseada numa perspetiva holística e de sistemas abertos – não visando diretamente o desenvolvimento de novas teorias, mas, fundamentalmente o colmatar de lacunas de índole empírica e metodológica. Embora os resultados obtidos não possam ser generalizados fora do contexto, eles podem, contudo, ser transpostos para outras intervenções; evidenciando-se como importantes Implicações para a Gestão que integram a base de relevância desta tese: um Quadro de Referência para a Transformação Organizacional Holística, Participativa e Baseada em Valores e respetivos Fatores Críticos de Sucesso. Numa perspetiva de Desenvolvimento Organizacional abrem-se ainda oportunidades de investigação-ação futura, no mesmo contexto, progredindo para uma abordagem à Qualidade Total e à Excelência Organizacional. Para além disso, outro dos resultados relevantes da investigação corresponde à produção do Metamodelo do Processo de Investigação-Ação que foi seguido – evidenciando, ao nível conceptual, os seus principais subprocessos, grupos de dados e pontos de articulação entre a vertente de ação e a vertente de investigação. Nesta vertente, abrem-se ainda oportunidades de investigação futura em termos de desenvolvimento do metamodelo, por forma a incluir uma visão conceptual de dados e uma perspetiva comportamental de sistema (resposta a eventos)

Contextualizing Secure Information System Design: A Socio-Technical Approach

Secure Information Systems (SIS) design paradigms have evolved in generations to adapt to IS security needs. However, modern IS are still vulnerable and are far from secure. The development of an underlying IS cannot be reduced to “technological fixes” neither is the design of SIS. Technical security cannot ensure IS security. Generations of SIS design paradigms have evolved, all with their own sets of shortcomings. A SIS design paradigm must meet well-defined requirements, yet contemporary paradigms do not meet all these requirements. Current SIS design paradigms are not easily applicable to IS. They lack a comprehensive modeling support and ignore the socio-technical organizational role of IS security. This research introduced the use of action research in design science research. Design science paradigm was leveraged to introduce a meta-design artifact explaining how IS requirements including security requirements can be incorporated in the design of SIS. The introduced artifact CSIS provided design comprehensiveness to emergent and changing requirements to IS from a socio-technical perspective. The CSIS artifact meets secure system meta-design requirements. This study presented a secure IS design principle that ensures IS security

A Nonverbal Behavior Approach to Identify Emergent Leaders in Small Groups

dentifying emergent leaders in organizations is a key issue in organizational behavioral research, and a new problem in social computing. This paper presents an analysis on how an emergent leader is perceived in newly formed, small groups, and then tackles the task of automatically inferring emergent leaders, using a variety of communicative nonverbal cues extracted from audio and video channels. The inference task uses rule-based and collective classification approaches with the combination of acoustic and visual features extracted from a new small group corpus specifically collected to analyze the emergent leadership phenomenon. Our results show that the emergent leader is perceived by his/her peers as an active and dominant person; that visual information augments acoustic information; and that adding relational information to the nonverbal cues improves the inference of each participant's leadership rankings in the group

An Emergent Architecture for Scaling Decentralized Communication Systems (DCS)

With recent technological advancements now accelerating the mobile and wireless Internet solution space, a ubiquitous computing Internet is well within the research and industrial community's design reach - a decentralized system design, which is not solely driven by static physical models and sound engineering principals, but more dynamically, perhaps sub-optimally at initial deployment and socially-influenced in its evolution. To complement today's Internet system, this thesis proposes a Decentralized Communication System (DCS) architecture with the following characteristics: flat physical topologies with numerous compute oriented and communication intensive nodes in the network with many of these nodes operating in multiple functional roles; self-organizing virtual structures formed through alternative mobility scenarios and capable of serving ad hoc networking formations; emergent operations and control with limited dependency on centralized control and management administration. Today, decentralized systems are not commercially scalable or viable for broad adoption in the same way we have to come to rely on the Internet or telephony systems. The premise in this thesis is that DCS can reach high levels of resilience, usefulness, scale that the industry has come to experience with traditional centralized systems by exploiting the following properties: (i.) network density and topological diversity; (ii.) self-organization and emergent attributes; (iii.) cooperative and dynamic infrastructure; and (iv.) node role diversity. This thesis delivers key contributions towards advancing the current state of the art in decentralized systems. First, we present the vision and a conceptual framework for DCS. Second, the thesis demonstrates that such a framework and concept architecture is feasible by prototyping a DCS platform that exhibits the above properties or minimally, demonstrates that these properties are feasible through prototyped network services. Third, this work expands on an alternative approach to network clustering using hierarchical virtual clusters (HVC) to facilitate self-organizing network structures. With increasing network complexity, decentralized systems can generally lead to unreliable and irregular service quality, especially given unpredictable node mobility and traffic dynamics. The HVC framework is an architectural strategy to address organizational disorder associated with traditional decentralized systems. The proposed HVC architecture along with the associated promotional methodology organizes distributed control and management services by leveraging alternative organizational models (e.g., peer-to-peer (P2P), centralized or tiered) in hierarchical and virtual fashion. Through simulation and analytical modeling, we demonstrate HVC efficiencies in DCS structural scalability and resilience by comparing static and dynamic HVC node configurations against traditional physical configurations based on P2P, centralized or tiered structures. Next, an emergent management architecture for DCS exploiting HVC for self-organization, introduces emergence as an operational approach to scaling DCS services for state management and policy control. In this thesis, emergence scales in hierarchical fashion using virtual clustering to create multiple tiers of local and global separation for aggregation, distribution and network control. Emergence is an architectural objective, which HVC introduces into the proposed self-management design for scaling and stability purposes. Since HVC expands the clustering model hierarchically and virtually, a clusterhead (CH) node, positioned as a proxy for a specific cluster or grouped DCS nodes, can also operate in a micro-capacity as a peer member of an organized cluster in a higher tier. As the HVC promotional process continues through the hierarchy, each tier of the hierarchy exhibits emergent behavior. With HVC as the self-organizing structural framework, a multi-tiered, emergent architecture enables the decentralized management strategy to improve scaling objectives that traditionally challenge decentralized systems. The HVC organizational concept and the emergence properties align with and the view of the human brain's neocortex layering structure of sensory storage, prediction and intelligence. It is the position in this thesis, that for DCS to scale and maintain broad stability, network control and management must strive towards an emergent or natural approach. While today's models for network control and management have proven to lack scalability and responsiveness based on pure centralized models, it is unlikely that singular organizational models can withstand the operational complexities associated with DCS. In this work, we integrate emergence and learning-based methods in a cooperative computing manner towards realizing DCS self-management. However, unlike many existing work in these areas which break down with increased network complexity and dynamics, the proposed HVC framework is utilized to offset these issues through effective separation, aggregation and asynchronous processing of both distributed state and policy. Using modeling techniques, we demonstrate that such architecture is feasible and can improve the operational robustness of DCS. The modeling emphasis focuses on demonstrating the operational advantages of an HVC-based organizational strategy for emergent management services (i.e., reachability, availability or performance). By integrating the two approaches, the DCS architecture forms a scalable system to address the challenges associated with traditional decentralized systems. The hypothesis is that the emergent management system architecture will improve the operational scaling properties of DCS-based applications and services. Additionally, we demonstrate structural flexibility of HVC as an underlying service infrastructure to build and deploy DCS applications and layered services. The modeling results demonstrate that an HVC-based emergent management and control system operationally outperforms traditional structural organizational models. In summary, this thesis brings together the above contributions towards delivering a scalable, decentralized system for Internet mobile computing and communications