Biosignal controlled recommendation in entertainment systems

With the explosive growth of the entertainment contents and the ubiquitous access of them via fixed or mobile computing devices, recommendation systems become essential tools to help the user to find the right entertainment at the right time and location. I envision that by integrating the bio signal input into the recommendation process, it will help the users not only to find interesting contents, but also to increase one’s comfort level by taking into account the biosginal feedback from the users. The goal of this project was to develop a biosignal controlled entertainment recommendation system that increases the user’s comfort level by reducing the level of stress. As the starting point, this project aims to contribute to the field of recommendation systems with two points. The first is the mechanism of embedding the biosignal non-intrusively into the recommendation process. The second is the strategy of the biosignal controlled recommendation to reduce stress. Heart rate controlled in-flight music recommendation is chosen as its application domain. The hypothesis of this application is that, the passenger's heart rate deviates from the normal due to unusual long haul flight cabin environment. By properly designing a music recommendation system to recommend heart rate controlled personalized music playlists to the passenger, the passengers' heart rate can be uplifted, down-lifted back to normal or kept within normal, thus their stress can be reduced. Four research questions have been formulated based on this hypothesis. After the literature study, the project went mainly through three phases: framework design, system implementation and user evaluation to answer these research questions. During the framework design phase, the heart rate was firstly modeled as the states of bradycardia, normal and tachycardia. The objective of the framework is that, if the user's heart rate is higher or lower than the normal heart rate, the system recommends a personalized music playlist accordingly to transfer the user’s heart rate back to normal, otherwise to keep it at normal. The adaptive framework integrates the concepts of context adaptive systems, user profiling, and the methods of using music to adjust the heart rate in a feedback control system. In the feedback loop, the playlists were composed using a Markov decision process. Yet, the framework allows the user to reject the recommendations and to manually select the favorite music items. During this process, the system logs the interactions between the user and the system for later learning the user’s latest music preferences. The designed framework was then implemented with platform independent software architecture. The architecture has five abstraction levels. The lowest resource level contains the music source, the heart rate sensors and the user profile information. The second layer is for resource management. In this layer are the manager components to manage the resources from the first layer and to modulate the access from upper layers to these resources. The third layer is the database, acting as a data repository. The fourth layer is for the adaptive control, which includes the user feedback log, the inference engine and the preference learning component. The top layer is the user interface. In this architecture, the layers and the components in the layers are loosely coupled, which ensures the flexibility. The implemented system was used in the user experiments to validate the hypothesis. The experiments simulated the long haul flights from Amsterdam to Shanghai with the same time schedule as the KLM flights. Twelve subjects were invited to participate in the experiments. Six were allocated to the controlled group and others were allocated to the treatment group. In addition to a normal entertainment system for the control group, the treatment group was also provided with the heart rate controlled music recommendation system. The experiments results validated the hypothesis and answered the research questions. The passenger's heart rate deviates from normal. In our user experiments, the passenger's heart rate was in the bradycardia state 24.6% of time and was in the tachycardia state 7.3% of time. The recommended uplifting music reduces the average bradycardia state duration from 14.78 seconds in the control group to 6.86 seconds in the treatment group. The recommended keeping music increases the average normal state duration from 24.66 seconds in the control group to 29.79 seconds in the treatment group. The recommended down-lifting music reduces the average tachycardia state duration from 13.89 seconds in the control group to 6.53 seconds in the treatment group. Compared to the control group, the stress of the treatment group has been reduced significantly

Adaptive Cognitive Interaction Systems

Adaptive kognitive Interaktionssysteme beobachten und modellieren den Zustand ihres Benutzers und passen das Systemverhalten entsprechend an. Ein solches System besteht aus drei Komponenten: Dem empirischen kognitiven Modell, dem komputationalen kognitiven Modell und dem adaptiven Interaktionsmanager. Die vorliegende Arbeit enthält zahlreiche Beiträge zur Entwicklung dieser Komponenten sowie zu deren Kombination. Die Ergebnisse werden in zahlreichen Benutzerstudien validiert

Optimized Biosignals Processing Algorithms for New Designs of Human Machine Interfaces on Parallel Ultra-Low Power Architectures

The aim of this dissertation is to explore Human Machine Interfaces (HMIs) in a variety of biomedical scenarios. The research addresses typical challenges in wearable and implantable devices for diagnostic, monitoring, and prosthetic purposes, suggesting a methodology for tailoring such applications to cutting edge embedded architectures. The main challenge is the enhancement of high-level applications, also introducing Machine Learning (ML) algorithms, using parallel programming and specialized hardware to improve the performance. The majority of these algorithms are computationally intensive, posing significant challenges for the deployment on embedded devices, which have several limitations in term of memory size, maximum operative frequency, and battery duration. The proposed solutions take advantage of a Parallel Ultra-Low Power (PULP) architecture, enhancing the elaboration on specific target architectures, heavily optimizing the execution, exploiting software and hardware resources. The thesis starts by describing a methodology that can be considered a guideline to efficiently implement algorithms on embedded architectures. This is followed by several case studies in the biomedical field, starting with the analysis of a Hand Gesture Recognition, based on the Hyperdimensional Computing algorithm, which allows performing a fast on-chip re-training, and a comparison with the state-of-the-art Support Vector Machine (SVM); then a Brain Machine Interface (BCI) to detect the respond of the brain to a visual stimulus follows in the manuscript. Furthermore, a seizure detection application is also presented, exploring different solutions for the dimensionality reduction of the input signals. The last part is dedicated to an exploration of typical modules for the development of optimized ECG-based applications

Procedures and Methodologies for the Control and Improvement of Energy-Environmental Quality in Construction

This Special Issue aims at providing the state-of-the-art on procedures and methodologies developed to improve energy and environmental performance through building renovation. We are greatly thankful to our colleagues building physics experts, building technology researchers, and urban environment scholars who contributed to this Special Issue, for sharing their original works in the field

Challenges and Limitation Analysis of an IoT-Dependent System for Deployment in Smart Healthcare Using Communication Standards Features

The use of IoT technology is rapidly increasing in healthcare development and smart healthcare system for fitness programs, monitoring, data analysis, etc. To improve the efficiency of monitoring, various studies have been conducted in this field to achieve improved precision. The architecture proposed herein is based on IoT integrated with a cloud system in which power absorption and accuracy are major concerns. We discuss and analyze development in this domain to improve the performance of IoT systems related to health care. Standards of communication for IoT data transmission and reception can help to understand the exact power absorption in different devices to achieve improved performance for healthcare development. We also systematically analyze the use of IoT in healthcare systems using cloud features, as well as the performance and limitations of IoT in this field. Furthermore, we discuss the design of an IoT system for efficient monitoring of various healthcare issues in elderly people and limitations of an existing system in terms of resources, power absorption and security when implemented in different devices as per requirements. Blood pressure and heartbeat monitoring in pregnant women are examples of high-intensity applications of NB-IoT (narrowband IoT), technology that supports widespread communication with a very low data cost and minimum processing complexity and battery lifespan. This article also focuses on analysis of the performance of narrowband IoT in terms of delay and throughput using singleand multinode approaches. We performed analysis using the message queuing telemetry transport protocol (MQTTP), which was found to be efficient compared to the limited application protocol (LAP) in sending information from sensors.Ministerio Español de Ciencia e Innovación under project number PID2020-115570GB-C22 (DemocratAI::UGR)Cátedra de Empresa Tecnología para las Personas (UGR-Fujitsu

Sensor Technologies for Intelligent Transportation Systems

Modern society faces serious problems with transportation systems, including but not limited to traffic congestion, safety, and pollution. Information communication technologies have gained increasing attention and importance in modern transportation systems. Automotive manufacturers are developing in-vehicle sensors and their applications in different areas including safety, traffic management, and infotainment. Government institutions are implementing roadside infrastructures such as cameras and sensors to collect data about environmental and traffic conditions. By seamlessly integrating vehicles and sensing devices, their sensing and communication capabilities can be leveraged to achieve smart and intelligent transportation systems. We discuss how sensor technology can be integrated with the transportation infrastructure to achieve a sustainable Intelligent Transportation System (ITS) and how safety, traffic control and infotainment applications can benefit from multiple sensors deployed in different elements of an ITS. Finally, we discuss some of the challenges that need to be addressed to enable a fully operational and cooperative ITS environment

Resource Management for Edge Computing in Internet of Things (IoT)

Die große Anzahl an Geräten im Internet der Dinge (IoT) und deren kontinuierliche Datensammlungen führen zu einem rapiden Wachstum der gesammelten Datenmenge. Die Daten komplett mittels zentraler Cloud Server zu verarbeiten ist ineffizient und zum Teil sogar unmöglich oder unnötig. Darum wird die Datenverarbeitung an den Rand des Netzwerks verschoben, was zu den Konzepten des Edge Computings geführt hat. Informationsverarbeitung nahe an der Datenquelle (z.B. auf Gateways und Edge Geräten) reduziert nicht nur die hohe Arbeitslast zentraler Server und Netzwerke, sondern verringer auch die Latenz für Echtzeitanwendungen, da die potentiell unzuverlässige Kommunikation zu Cloud Servern mit ihrer unvorhersehbaren Netzwerklatenz vermieden wird. Aktuelle IoT Architekturen verwenden Gateways, um anwendungsspezifische Verbindungen zu IoT Geräten herzustellen. In typischen Konfigurationen teilen sich mehrere IoT Edge Geräte ein IoT Gateway. Wegen der begrenzten verfügbaren Bandbreite und Rechenkapazität eines IoT Gateways muss die Servicequalität (SQ) der verbundenen IoT Edge Geräte über die Zeit angepasst werden. Nicht nur um die Anforderungen der einzelnen Nutzer der IoT Geräte zu erfüllen, sondern auch um die SQBedürfnisse der anderen IoT Edge Geräte desselben Gateways zu tolerieren. Diese Arbeit untersucht zuerst essentielle Technologien für IoT und existierende Trends. Dabei werden charakteristische Eigenschaften von IoT für die Embedded Domäne, sowie eine umfassende IoT Perspektive für Eingebettete Systeme vorgestellt. Mehrere Anwendungen aus dem Gesundheitsbereich werden untersucht und implementiert, um ein Model für deren Datenverarbeitungssoftware abzuleiten. Dieses Anwendungsmodell hilft bei der Identifikation verschiedener Betriebsmodi. IoT Systeme erwarten von den Edge Geräten, dass sie mehrere Betriebsmodi unterstützen, um sich während des Betriebs an wechselnde Szenarien anpassen zu können. Z.B. Energiesparmodi bei geringen Batteriereserven trotz gleichzeitiger Aufrechterhaltung der kritischen Funktionalität oder einen Modus, um die Servicequalität auf Wunsch des Nutzers zu erhöhen etc. Diese Modi verwenden entweder verschiedene Auslagerungsschemata (z.B. die übertragung von Rohdaten, von partiell bearbeiteten Daten, oder nur des finalen Ergebnisses) oder verschiedene Servicequalitäten. Betriebsmodi unterscheiden sich in ihren Ressourcenanforderungen sowohl auf dem Gerät (z.B. Energieverbrauch), wie auch auf dem Gateway (z.B. Kommunikationsbandbreite, Rechenleistung, Speicher etc.). Die Auswahl des besten Betriebsmodus für Edge Geräte ist eine Herausforderung in Anbetracht der begrenzten Ressourcen am Rand des Netzwerks (z.B. Bandbreite und Rechenleistung des gemeinsamen Gateways), diverser Randbedingungen der IoT Edge Geräte (z.B. Batterielaufzeit, Servicequalität etc.) und der Laufzeitvariabilität am Rand der IoT Infrastruktur. In dieser Arbeit werden schnelle und effiziente Auswahltechniken für Betriebsmodi entwickelt und präsentiert. Wenn sich IoT Geräte in der Reichweite mehrerer Gateways befinden, ist die Verwaltung der gemeinsamen Ressourcen und die Auswahl der Betriebsmodi für die IoT Geräte sogar noch komplexer. In dieser Arbeit wird ein verteilter handelsorientierter Geräteverwaltungsmechanismus für IoT Systeme mit mehreren Gateways präsentiert. Dieser Mechanismus zielt auf das kombinierte Problem des Bindens (d.h. ein Gateway für jedes IoT Gerät bestimmen) und der Allokation (d.h. die zugewiesenen Ressourcen für jedes Gerät bestimmen) ab. Beginnend mit einer initialen Konfiguration verhandeln und kommunizieren die Gateways miteinander und migrieren IoT Geräte zwischen den Gateways, wenn es den Nutzen für das Gesamtsystem erhöht. In dieser Arbeit werden auch anwendungsspezifische Optimierungen für IoT Geräte vorgestellt. Drei Anwendungen für den Gesundheitsbereich wurden realisiert und für tragbare IoT Geräte untersucht. Es wird auch eine neuartige Kompressionsmethode vorgestellt, die speziell für IoT Anwendungen geeignet ist, die Bio-Signale für Gesundheitsüberwachungen verarbeiten. Diese Technik reduziert die zu übertragende Datenmenge des IoT Gerätes, wodurch die Ressourcenauslastung auf dem Gerät und dem gemeinsamen Gateway reduziert wird. Um die vorgeschlagenen Techniken und Mechanismen zu evaluieren, wurden einige Anwendungen auf IoT Plattformen untersucht, um ihre Parameter, wie die Ausführungszeit und Ressourcennutzung, zu bestimmen. Diese Parameter wurden dann in einem Rahmenwerk verwendet, welches das IoT Netzwerk modelliert, die Interaktion zwischen Geräten und Gateway erfasst und den Kommunikationsoverhead sowie die erreichte Batterielebenszeit und Servicequalität der Geräte misst. Die Algorithmen zur Auswahl der Betriebsmodi wurden zusätzlich auf IoT Plattformen implementiert, um ihre Overheads bzgl. Ausführungszeit und Speicherverbrauch zu messen

Sophisticated Batteryless Sensing

Wireless embedded sensing systems have revolutionized scientific, industrial, and consumer applications. Sensors have become a fixture in our daily lives, as well as the scientific and industrial communities by allowing continuous monitoring of people, wildlife, plants, buildings, roads and highways, pipelines, and countless other objects. Recently a new vision for sensing has emerged---known as the Internet-of-Things (IoT)---where trillions of devices invisibly sense, coordinate, and communicate to support our life and well being. However, the sheer scale of the IoT has presented serious problems for current sensing technologies---mainly, the unsustainable maintenance, ecological, and economic costs of recycling or disposing of trillions of batteries. This energy storage bottleneck has prevented massive deployments of tiny sensing devices at the edge of the IoT. This dissertation explores an alternative---leave the batteries behind, and harvest the energy required for sensing tasks from the environment the device is embedded in. These sensors can be made cheaper, smaller, and will last decades longer than their battery powered counterparts, making them a perfect fit for the requirements of the IoT. These sensors can be deployed where battery powered sensors cannot---embedded in concrete, shot into space, or even implanted in animals and people. However, these batteryless sensors may lose power at any point, with no warning, for unpredictable lengths of time. Programming, profiling, debugging, and building applications with these devices pose significant challenges. First, batteryless devices operate in unpredictable environments, where voltages vary and power failures can occur at any time---often devices are in failure for hours. Second, a device\u27s behavior effects the amount of energy they can harvest---meaning small changes in tasks can drastically change harvester efficiency. Third, the programming interfaces of batteryless devices are ill-defined and non- intuitive; most developers have trouble anticipating the problems inherent with an intermittent power supply. Finally, the lack of community, and a standard usable hardware platform have reduced the resources and prototyping ability of the developer. In this dissertation we present solutions to these challenges in the form of a tool for repeatable and realistic experimentation called Ekho, a reconfigurable hardware platform named Flicker, and a language and runtime for timely execution of intermittent programs called Mayfly