Analysis and enhancement of interpersonal coordination using inertial measurement unit solutions

Die heutigen mobilen Kommunikationstechnologien haben den Umfang der verbalen und textbasierten Kommunikation mit anderen Menschen, sozialen Robotern und künstlicher Intelligenz erhöht. Auf der anderen Seite reduzieren diese Technologien die nonverbale und die direkte persönliche Kommunikation, was zu einer gesellschaftlichen Thematik geworden ist, weil die Verringerung der direkten persönlichen Interaktionen eine angemessene Wahrnehmung sozialer und umgebungsbedingter Reizmuster erschweren und die Entwicklung allgemeiner sozialer Fähigkeiten bremsen könnte. Wissenschaftler haben aktuell die Bedeutung nonverbaler zwischenmenschlicher Aktivitäten als soziale Fähigkeiten untersucht, indem sie menschliche Verhaltensmuster in Zusammenhang mit den jeweilgen neurophysiologischen Aktivierungsmustern analzsiert haben. Solche Querschnittsansätze werden auch im Forschungsprojekt der Europäischen Union "Socializing sensori-motor contingencies" (socSMCs) verfolgt, das darauf abzielt, die Leistungsfähigkeit sozialer Roboter zu verbessern und Autismus-Spektrumsstörungen (ASD) adäquat zu behandeln. In diesem Zusammenhang ist die Modellierung und das Benchmarking des Sozialverhaltens gesunder Menschen eine Grundlage für theorieorientierte und experimentelle Studien zum weiterführenden Verständnis und zur Unterstützung interpersoneller Koordination. In diesem Zusammenhang wurden zwei verschiedene empirische Kategorien in Abhängigkeit von der Entfernung der Interagierenden zueinander vorgeschlagen: distale vs. proximale Interaktionssettings, da sich die Struktur der beteiligten kognitiven Systeme zwischen den Kategorien ändert und sich die Ebene der erwachsenden socSMCs verschiebt. Da diese Dissertation im Rahmen des socSMCs-Projekts entstanden ist, wurden Interaktionssettings für beide Kategorien (distal und proximal) entwickelt. Zudem wurden Ein-Sensor-Lösungen zur Reduzierung des Messaufwands (und auch der Kosten) entwickelt, um eine Messung ausgesuchter Verhaltensparameter bei einer Vielzahl von Menschen und sozialen Interaktionen zu ermöglichen. Zunächst wurden Algorithmen für eine kopfgetragene Trägheitsmesseinheit (H-IMU) zur Messung der menschlichen Kinematik als eine Ein-Sensor-Lösung entwickelt. Die Ergebnisse bestätigten, dass die H-IMU die eigenen Gangparameter unabhängig voneinander allein auf Basis der Kopfkinematik messen kann. Zweitens wurden—als ein distales socSMC-Setting—die interpersonellen Kopplungen mit einem Bezug auf drei interagierende Merkmale von „Übereinstimmung“ (engl.: rapport) behandelt: Positivität, gegenseitige Aufmerksamkeit und Koordination. Die H-IMUs überwachten bestimmte soziale Verhaltensereignisse, die sich auf die Kinematik der Kopforientierung und Oszillation während des Gehens und Sprechens stützen, so dass der Grad der Übereinstimmung geschätzt werden konnte. Schließlich belegten die Ergebnisse einer experimentellen Studie, die zu einer kollaborativen Aufgabe mit der entwickelten IMU-basierten Tablet-Anwendung durchgeführt wurde, unterschiedliche Wirkungen verschiedener audio-motorischer Feedbackformen für eine Unterstützung der interpersonellen Koordination in der Kategorie proximaler sensomotorischer Kontingenzen. Diese Dissertation hat einen intensiven interdisziplinären Charakter: Technologische Anforderungen in den Bereichen der Sensortechnologie und der Softwareentwicklung mussten in direktem Bezug auf vordefinierte verhaltenswissenschaftliche Fragestellungen entwickelt und angewendet bzw. gelöst werden—und dies in zwei unterschiedlichen Domänen (distal, proximal). Der gegebene Bezugsrahmen wurde als eine große Herausforderung bei der Entwicklung der beschriebenen Methoden und Settings wahrgenommen. Die vorgeschlagenen IMU-basierten Lösungen könnten dank der weit verbreiteten IMU-basierten mobilen Geräte zukünftig in verschiedene Anwendungen perspektiv reich integriert werden.Today’s mobile communication technologies have increased verbal and text-based communication with other humans, social robots and intelligent virtual assistants. On the other hand, the technologies reduce face-to-face communication. This social issue is critical because decreasing direct interactions may cause difficulty in reading social and environmental cues, thereby impeding the development of overall social skills. Recently, scientists have studied the importance of nonverbal interpersonal activities to social skills, by measuring human behavioral and neurophysiological patterns. These interdisciplinary approaches are in line with the European Union research project, “Socializing sensorimotor contingencies” (socSMCs), which aims to improve the capability of social robots and properly deal with autism spectrum disorder (ASD). Therefore, modelling and benchmarking healthy humans’ social behavior are fundamental to establish a foundation for research on emergence and enhancement of interpersonal coordination. In this research project, two different experimental settings were categorized depending on interactants’ distance: distal and proximal settings, where the structure of engaged cognitive systems changes, and the level of socSMCs differs. As a part of the project, this dissertation work referred to this spatial framework. Additionally, single-sensor solutions were developed to reduce costs and efforts in measuring human behaviors, recognizing the social behaviors, and enhancing interpersonal coordination. First of all, algorithms using a head worn inertial measurement unit (H-IMU) were developed to measure human kinematics, as a baseline for social behaviors. The results confirmed that the H-IMU can measure individual gait parameters by analyzing only head kinematics. Secondly, as a distal sensorimotor contingency, interpersonal relationship was considered with respect to a dynamic structure of three interacting components: positivity, mutual attentiveness, and coordination. The H-IMUs monitored the social behavioral events relying on kinematics of the head orientation and oscillation during walk and talk, which can contribute to estimate the level of rapport. Finally, in a new collaborative task with the proposed IMU-based tablet application, results verified effects of different auditory-motor feedbacks on the enhancement of interpersonal coordination in a proximal setting. This dissertation has an intensive interdisciplinary character: Technological development, in the areas of sensor and software engineering, was required to apply to or solve issues in direct relation to predefined behavioral scientific questions in two different settings (distal and proximal). The given frame served as a reference in the development of the methods and settings in this dissertation. The proposed IMU-based solutions are also promising for various future applications due to widespread wearable devices with IMUs.European Commission/HORIZON2020-FETPROACT-2014/641321/E

Development of Unmanned Aerial Vehicle (Quadcopter)With Real-Time Object Tracking

In the previous decade, Unmanned Aerial Vehicles (UAVs) have turned into a subject of enthusiasm for some exploration associations. UAVs are discovering applications in different regions going from military applications to activity reconnaissance. This thesis is an overview of a particular sort of UAV called quadrotor or quadcopter. Scientists are often picking quadrotors for their exploration because a quadrotor can precisely and productively perform assignments that future of high hazard for a human pilot to perform. This thesis includes the dynamic models of a quadrotor and model-autonomous control systems. It also explains the complete description of developed quadcopter used for surveillance purpose with real-time object detection. In the present time, the focus has moved to outlining autonomous quadrotors. Ultimately, it examines the potential applications of quadrotors and their part in multi-operators frameworks. The Unmanned aerial vehicle (Quadcopter) has been developed that could be used for search and surveillance purpose. This project comprised of both hardware and software part. The hardware part comprised of the development of unmanned aerial vehicle (Quadcopter). The main components that were used in this project are KK2 flight controller board, outrunner brushless DC motor, Electronic Speed Controllers (ESC), GPS (Global Positioning System) receiver, video transmitter and receiver, HD (High Definition) camera, RC (Radio Controlled) transmitter and receiver. Software part comprised of real-time object detection and tracking algorithm for detecting and tracking of human beings that were done with the help of Matlab software. After achieving the stable flight, the camera installed on the quadcopter would transmit a video signal to the receiver placed on the ground station. Video signal from the receiver would then be transferred to Matlab software for further processing or for tracking human beings using real-time object detection and tracking algorith

A vision system for mobile maritime surveillance platforms

Mobile surveillance systems play an important role to minimise security and safety threats in high-risk or hazardous environments. Providing a mobile marine surveillance platform with situational awareness of its environment is important for mission success. An essential part of situational awareness is the ability to detect and subsequently track potential target objects.Typically, the exact type of target objects is unknown, hence detection is addressed as a problem of finding parts of an image that stand out in relation to their surrounding regions or are atypical to the domain. Contrary to existing saliency methods, this thesis proposes the use of a domain specific visual attention approach for detecting potential regions of interest in maritime imagery. For this, low-level features that are indicative of maritime targets are identified. These features are then evaluated with respect to their local, regional, and global significance. Together with a domain specific background segmentation technique, the features are combined in a Bayesian classifier to direct visual attention to potential target objects.The maritime environment introduces challenges to the camera system: gusts, wind, swell, or waves can cause the platform to move drastically and unpredictably. Pan-tilt-zoom cameras that are often utilised for surveillance tasks can adjusting their orientation to provide a stable view onto the target. However, in rough maritime environments this requires high-speed and precise inputs. In contrast, omnidirectional cameras provide a full spherical view, which allows the acquisition and tracking of multiple targets at the same time. However, the target itself only occupies a small fraction of the overall view. This thesis proposes a novel, target-centric approach for image stabilisation. A virtual camera is extracted from the omnidirectional view for each target and is adjusted based on the measurements of an inertial measurement unit and an image feature tracker. The combination of these two techniques in a probabilistic framework allows for stabilisation of rotational and translational ego-motion. Furthermore, it has the specific advantage of being robust to loosely calibrated and synchronised hardware since the fusion of tracking and stabilisation means that tracking uncertainty can be used to compensate for errors in calibration and synchronisation. This then completely eliminates the need for tedious calibration phases and the adverse effects of assembly slippage over time.Finally, this thesis combines the visual attention and omnidirectional stabilisation frameworks and proposes a multi view tracking system that is capable of detecting potential target objects in the maritime domain. Although the visual attention framework performed well on the benchmark datasets, the evaluation on real-world maritime imagery produced a high number of false positives. An investigation reveals that the problem is that benchmark data sets are unconsciously being influenced by human shot selection, which greatly simplifies the problem of visual attention. Despite the number of false positives, the tracking approach itself is robust even if a high number of false positives are tracked

Contribution to the development of pico-satellites for Earth observation and technology demonstrators

The submitted Ph.D. thesis is a contribution to the development of pico-satellites and nano-satellites for Earth Observation and Technology Demonstrators. Development of satellites with a mass from 1 kg up to 10 kg has really boosted during the past 15 years offering large costs reduction with respect to conventional (larger) satellites at compromised performances. However, these performances are increasing step by step, and their potential is about to explode. With 1.3 kg and a volume of 1 liter, 3Cat-1 is the first pico-satellite class satellite with up to seven different payloads on board: a harvesting energy powered beacon, a new topology solar cells, an optical sensor, a Geiger counter, a mono-atomic MEMS oxygen detector, a graphene transistor, and a Wireless Power Transfer experiment. Additionally, many main subsystems like the Electrical Power, the Attitude Determination and Control, and the Communication subsystems have been developed during this PhD. A description of each subsystem and how they have been integrated under these constrains are described in this PhD Thesis report. In the subsystems and in the experiments there are novel concepts and designs that will be explained. Some have been already published in the technical and scientific literature, others will be in the near future.Tesi doctoral és una contribució al desenvolupament de pico-satèl.lits per a observació de la Terra i demostradors tecnològics. El desenvolupament de satèl¿lits amb massa d'entre 1 kg i 10 kg ha crescut ràpidament durant els darrers 15 anys, oferint importants reduccions de cost respecte els satèl.lits més convencionals (i més grans) amb unes prestacions menors però al mateix temps, acceptables. No obstant, aquestes prestacions estan millorant pas a pas, i el seu potencial a prop d'explosionar. Amb una massa de 1.3 kg i un volum d'un litre, el 3Cat-1 és el primer pico-satèl.lit de la seva categoria equipat amb set diferents càrregues útils a bord: un senyal de radio-balisa alimentat amb energia provinent d'un sistema recol.lector d'energia ambient, una nova topologia de cel.les solars, un sensor òptic, un comptador Geiger, un detector MEMS d'oxigen atòmic, un transistor de grafè, i un experiment de transmissió d'energia per acoblament inductiu. A més a més, molts dels subsistemes com ara el d'energia elèctrica, control i determinació d'actitud, i el subsistema de comunicacions han estat desenvolupats durant aquesta Tesi doctoral. Una descripció de cada subsistema, i com han estat integrats en condicions d'espai, massa i energia reduïdes, s'explica en aquest document. En els subsistemes i en els experiments hi ha nous conceptes i dissenys explicats. Alguns han estat ja publicats en la literatura científica mentre que altres ho seran en un futur prope

Future payload technology requirements study

Technology advances needed for an overall mission model standpoint as well as those for individual shuttle payloads are defined. The technology advances relate to the mission scientific equipment, spacecraft subsystems that functionally support this equipment, and other payload-related equipment, software, and environment necessary to meet broad program objectives. In the interest of obtaining commonality of requirements, the study was structured according to technology categories rather than in terms of individual payloads

Wearable and BAN Sensors for Physical Rehabilitation and eHealth Architectures

The demographic shift of the population towards an increase in the number of elderly citizens, together with the sedentary lifestyle we are adopting, is reflected in the increasingly debilitated physical health of the population. The resulting physical impairments require rehabilitation therapies which may be assisted by the use of wearable sensors or body area network sensors (BANs). The use of novel technology for medical therapies can also contribute to reducing the costs in healthcare systems and decrease patient overflow in medical centers. Sensors are the primary enablers of any wearable medical device, with a central role in eHealth architectures. The accuracy of the acquired data depends on the sensors; hence, when considering wearable and BAN sensing integration, they must be proven to be accurate and reliable solutions. This book is a collection of works focusing on the current state-of-the-art of BANs and wearable sensing devices for physical rehabilitation of impaired or debilitated citizens. The manuscripts that compose this book report on the advances in the research related to different sensing technologies (optical or electronic) and body area network sensors (BANs), their design and implementation, advanced signal processing techniques, and the application of these technologies in areas such as physical rehabilitation, robotics, medical diagnostics, and therapy

Motion Artifact Processing Techniques for Physiological Signals

The combination of reducing birth rate and increasing life expectancy continues to drive the demographic shift toward an ageing population and this is placing an ever-increasing burden on our healthcare systems. The urgent need to address this so called healthcare \time bomb" has led to a rapid growth in research into ubiquitous, pervasive and distributed healthcare technologies where recent advances in signal acquisition, data storage and communication are helping such systems become a reality. However, similar to recordings performed in the hospital environment, artifacts continue to be a major issue for these systems. The magnitude and frequency of artifacts can vary signicantly depending on the recording environment with one of the major contributions due to the motion of the subject or the recording transducer. As such, this thesis addresses the challenges of the removal of this motion artifact removal from various physiological signals. The preliminary investigations focus on artifact identication and the tagging of physiological signals streams with measures of signal quality. A new method for quantifying signal quality is developed based on the use of inexpensive accelerometers which facilitates the appropriate use of artifact processing methods as needed. These artifact processing methods are thoroughly examined as part of a comprehensive review of the most commonly applicable methods. This review forms the basis for the comparative studies subsequently presented. Then, a simple but novel experimental methodology for the comparison of artifact processing techniques is proposed, designed and tested for algorithm evaluation. The method is demonstrated to be highly eective for the type of artifact challenges common in a connected health setting, particularly those concerned with brain activity monitoring. This research primarily focuses on applying the techniques to functional near infrared spectroscopy (fNIRS) and electroencephalography (EEG) data due to their high susceptibility to contamination by subject motion related artifact. Using the novel experimental methodology, complemented with simulated data, a comprehensive comparison of a range of artifact processing methods is conducted, allowing the identication of the set of the best performing methods. A novel artifact removal technique is also developed, namely ensemble empirical mode decomposition with canonical correlation analysis (EEMD-CCA), which provides the best results when applied on fNIRS data under particular conditions. Four of the best performing techniques were then tested on real ambulatory EEG data contaminated with movement artifacts comparable to those observed during in-home monitoring. It was determined that when analysing EEG data, the Wiener lter is consistently the best performing artifact removal technique. However, when employing the fNIRS data, the best technique depends on a number of factors including: 1) the availability of a reference signal and 2) whether or not the form of the artifact is known. It is envisaged that the use of physiological signal monitoring for patient healthcare will grow signicantly over the next number of decades and it is hoped that this thesis will aid in the progression and development of artifact removal techniques capable of supporting this growth