SALSA: A Novel Dataset for Multimodal Group Behavior Analysis

Studying free-standing conversational groups (FCGs) in unstructured social settings (e.g., cocktail party ) is gratifying due to the wealth of information available at the group (mining social networks) and individual (recognizing native behavioral and personality traits) levels. However, analyzing social scenes involving FCGs is also highly challenging due to the difficulty in extracting behavioral cues such as target locations, their speaking activity and head/body pose due to crowdedness and presence of extreme occlusions. To this end, we propose SALSA, a novel dataset facilitating multimodal and Synergetic sociAL Scene Analysis, and make two main contributions to research on automated social interaction analysis: (1) SALSA records social interactions among 18 participants in a natural, indoor environment for over 60 minutes, under the poster presentation and cocktail party contexts presenting difficulties in the form of low-resolution images, lighting variations, numerous occlusions, reverberations and interfering sound sources; (2) To alleviate these problems we facilitate multimodal analysis by recording the social interplay using four static surveillance cameras and sociometric badges worn by each participant, comprising the microphone, accelerometer, bluetooth and infrared sensors. In addition to raw data, we also provide annotations concerning individuals' personality as well as their position, head, body orientation and F-formation information over the entire event duration. Through extensive experiments with state-of-the-art approaches, we show (a) the limitations of current methods and (b) how the recorded multiple cues synergetically aid automatic analysis of social interactions. SALSA is available at http://tev.fbk.eu/salsa.Comment: 14 pages, 11 figure

A Tutorial on Environment-Aware Communications via Channel Knowledge Map for 6G

Sixth-generation (6G) mobile communication networks are expected to have dense infrastructures, large-dimensional channels, cost-effective hardware, diversified positioning methods, and enhanced intelligence. Such trends bring both new challenges and opportunities for the practical design of 6G. On one hand, acquiring channel state information (CSI) in real time for all wireless links becomes quite challenging in 6G. On the other hand, there would be numerous data sources in 6G containing high-quality location-tagged channel data, making it possible to better learn the local wireless environment. By exploiting such new opportunities and for tackling the CSI acquisition challenge, there is a promising paradigm shift from the conventional environment-unaware communications to the new environment-aware communications based on the novel approach of channel knowledge map (CKM). This article aims to provide a comprehensive tutorial overview on environment-aware communications enabled by CKM to fully harness its benefits for 6G. First, the basic concept of CKM is presented, and a comparison of CKM with various existing channel inference techniques is discussed. Next, the main techniques for CKM construction are discussed, including both the model-free and model-assisted approaches. Furthermore, a general framework is presented for the utilization of CKM to achieve environment-aware communications, followed by some typical CKM-aided communication scenarios. Finally, important open problems in CKM research are highlighted and potential solutions are discussed to inspire future work

Doctor of Philosophy

dissertationDevice-free localization (DFL) and tracking services are important components in security, emergency response, home and building automation, and assisted living applications where an action is taken based on a person's location. In this dissertation, we develop new methods and models to enable and improve DFL in a variety of radio frequency sensor network configurations. In the first contribution of this work, we develop a linear regression and line stabbing method which use a history of line crossing measurements to estimate the track of a person walking through a wireless network. Our methods provide an alternative approach to DFL in wireless networks where the number of nodes that can communicate with each other in a wireless network is limited and traditional DFL methods are ill-suited. We then present new methods that enable through-wall DFL when nodes in the network are in motion. We demonstrate that we can detect when a person crosses between ultra-wideband radios in motion based on changes in the energy contained in the first few nanoseconds of a measured channel impulse response. Through experimental testing, we show how our methods can localize a person through walls with transceivers in motion. Next, we develop new algorithms to localize boundary crossings when a person crosses between multiple nodes simultaneously. We experimentally evaluate our algorithms with received signal strength (RSS) measurements collected from a row of radio frequency (RF) nodes placed along a boundary and show that our algorithms achieve orders of magnitude better localization classification than baseline DFL methods. We then present a way to improve the models used in through-wall radio tomographic imaging with E-shaped patch antennas we develop and fabricate which remain tuned even when placed against a dielectric. Through experimentation, we demonstrate the E-shaped patch antennas lower localization error by 44% compared with omnidirectional and microstrip patch antennas. In our final contribution, we develop a new mixture model that relates a link's RSS as a function of a person's location in a wireless network. We develop new localization methods that compute the probabilities of a person occupying a location based on our mixture model. Our methods continuously recalibrate the model to achieve a low localization error even in changing environments

Pattern-theoretic foundations of automatic target recognition in clutter

Issued as final reportAir Force Office of Scientific Research (U.S.

Monte Carlo Method with Heuristic Adjustment for Irregularly Shaped Food Product Volume Measurement

Volume measurement plays an important role in the production and processing of food products. Various methods have been proposed to measure the volume of food products with irregular shapes based on 3D reconstruction. However, 3D reconstruction comes with a high-priced computational cost. Furthermore, some of the volume measurement methods based on 3D reconstruction have a low accuracy. Another method for measuring volume of objects uses Monte Carlo method. Monte Carlo method performs volume measurements using random points. Monte Carlo method only requires information regarding whether random points fall inside or outside an object and does not require a 3D reconstruction. This paper proposes volume measurement using a computer vision system for irregularly shaped food products without 3D reconstruction based on Monte Carlo method with heuristic adjustment. Five images of food product were captured using five cameras and processed to produce binary images. Monte Carlo integration with heuristic adjustment was performed to measure the volume based on the information extracted from binary images. The experimental results show that the proposed method provided high accuracy and precision compared to the water displacement method. In addition, the proposed method is more accurate and faster than the space carving method