CHORUS Deliverable 2.1: State of the Art on Multimedia Search Engines

Based on the information provided by European projects and national initiatives related to multimedia search as well as domains experts that participated in the CHORUS Think-thanks and workshops, this document reports on the state of the art related to multimedia content search from, a technical, and socio-economic perspective. The technical perspective includes an up to date view on content based indexing and retrieval technologies, multimedia search in the context of mobile devices and peer-to-peer networks, and an overview of current evaluation and benchmark inititiatives to measure the performance of multimedia search engines. From a socio-economic perspective we inventorize the impact and legal consequences of these technical advances and point out future directions of research

Multi-modal on-body sensing of human activities

Increased usage and integration of state-of-the-art information technology in our everyday work life aims at increasing the working efficiency. Due to unhandy human-computer-interaction methods this progress does not always result in increased efficiency, for mobile workers in particular. Activity recognition based contextual computing attempts to balance this interaction deficiency. This work investigates wearable, on-body sensing techniques on their applicability in the field of human activity recognition. More precisely we are interested in the spotting and recognition of so-called manipulative hand gestures. In particular the thesis focuses on the question whether the widely used motion sensing based approach can be enhanced through additional information sources. The set of gestures a person usually performs on a specific place is limited -- in the contemplated production and maintenance scenarios in particular. As a consequence this thesis investigates whether the knowledge about the user's hand location provides essential hints for the activity recognition process. In addition, manipulative hand gestures -- due to their object manipulating character -- typically start in the moment the user's hand reaches a specific place, e.g. a specific part of a machinery. And the gestures most likely stop in the moment the hand leaves the position again. Hence this thesis investigates whether hand location can help solving the spotting problem. Moreover, as user-independence is still a major challenge in activity recognition, this thesis investigates location context as a possible key component in a user-independent recognition system. We test a Kalman filter based method to blend absolute position readings with orientation readings based on inertial measurements. A filter structure is suggested which allows up-sampling of slow absolute position readings, and thus introduces higher dynamics to the position estimations. In such a way the position measurement series is made aware of wrist motions in addition to the wrist position. We suggest location based gesture spotting and recognition approaches. Various methods to model the location classes used in the spotting and recognition stages as well as different location distance measures are suggested and evaluated. In addition a rather novel sensing approach in the field of human activity recognition is studied. This aims at compensating drawbacks of the mere motion sensing based approach. To this end we develop a wearable hardware architecture for lower arm muscular activity measurements. The sensing hardware based on force sensing resistors is designed to have a high dynamic range. In contrast to preliminary attempts the proposed new design makes hardware calibration unnecessary. Finally we suggest a modular and multi-modal recognition system; modular with respect to sensors, algorithms, and gesture classes. This means that adding or removing a sensor modality or an additional algorithm has little impact on the rest of the recognition system. Sensors and algorithms used for spotting and recognition can be selected and fine-tuned separately for each single activity. New activities can be added without impact on the recognition rates of the other activities

Mobile Health Technologies

Mobile Health Technologies, also known as mHealth technologies, have emerged, amongst healthcare providers, as the ultimate Technologies-of-Choice for the 21st century in delivering not only transformative change in healthcare delivery, but also critical health information to different communities of practice in integrated healthcare information systems. mHealth technologies nurture seamless platforms and pragmatic tools for managing pertinent health information across the continuum of different healthcare providers. mHealth technologies commonly utilize mobile medical devices, monitoring and wireless devices, and/or telemedicine in healthcare delivery and health research. Today, mHealth technologies provide opportunities to record and monitor conditions of patients with chronic diseases such as asthma, Chronic Obstructive Pulmonary Diseases (COPD) and diabetes mellitus. The intent of this book is to enlighten readers about the theories and applications of mHealth technologies in the healthcare domain

Automated Productivity Models for Earthmoving Operations

Earthmoving operations have significant importance, particularly for civil infrastructure projects. The performance of these operations should be monitored regularly to support timely recognition of undesirable productivity variances. Although productivity assessment occupies high importance in earthmoving operations, it does not provide sufficient information to assist project managers in taking the necessary actions in a timely manner. Assessment only is not capable of identifying problems encountered in these operations and their causes. Many studies recognized conditions and related factors that influence productivity of earthmoving operations. These conditions are mainly project-specific and vary from one project to another. Most of reported work in the literature focused on assessment rather than analysis of productivity. This study presents three integrated models that automate productivity measurement and analysis processes with capabilities to detect different adverse conditions that influence the productivity of earthmoving operations. The models exploit innovations in wireless and remote sensing technologies to provide project managers, contractors, and decision makers with a near-real-time automated productivity measurement and analysis. The developed models account for various uncertainties associated with earthmoving projects. The first model introduces a fuzzy-based standardization for customizing the configuration of onsite data acquisition systems for earthmoving operations. While the second model consists of two interrelated modules. The first is a customized automated data acquisition module, where a variety of sensors, smart boards, and microcontrollers are used to automate the data acquisition process. This module encompasses onsite fixed unit and a set of portable units attached to each truck used in the earthmoving fleet. The fixed unit is a communication gateway (Meshlium®), which has integrated MySQL database with data processing capabilities. Each mobile unit consists of a microcontroller equipped with a smart board that hosts a GPS module as well as a number of sensors such as accelerometer, temperature and humidity sensors, load cell and automated weather station. The second is a productivity measurement and analysis module, which processes and analyzes the data collected automatically in the first module. It automates the analysis process using data mining and machine learning techniques; providing a near-real-time web-based visualized representation of measurement and analysis outcomes. Artificial Neural Network (ANN) was used to model productivity losses due to the existence of different influencing conditions. Laboratory and field work was conducted in the development and validation processes of the developed models. The work encompassed field and scaled laboratory experiments. The laboratory experiments were conducted in an open to sky terrace to allow for a reliable access to GPS satellites. Also, to make a direct connection between the data communication gateway (Meshlium®), initially installed on a PC computer to observe the received data latency. The laboratory experiments unitized 1:24 scaled loader and dumping truck to simulate loading, hauling and dumping operations. The truck was instrumented with the microcontroller equipped with an accelerometer, GPS module, load cell, and soil water content sensor. Thirty simulated earthmoving cycles were conducted using the scaled equipment. The collected data was recorded in a micro secure digital (SD) card in a comma separated value (CSV) format. The field work was carried out in the city of Saint-Laurent, Montreal, Quebec, Canada using a passenger vehicle to mimic the hauling truck operational modes. Fifteen Field simulated earthmoving cycles were performed. In this work two roads with different surface conditions, but of equal length (1150 m) represented the haul and return roads. These two roads were selected to validate the developed road condition analysis algorithm and to study the model’s capability in determining the consequences of adverse road conditions on the haul and return durations and thus on the tuck and fleet productivity. The data collected from the lab experiments and field work was used as input for the developed model. The developed model has shown perfect recognition of the state of truck throughout the fifteen field simulated earthmoving cycles. The developed road condition analysis algorithm has demonstrated an accuracy of 83.3% and 82.6% in recognizing road bumps and potholes, respectively. Also, the results indicated tiny variances in measuring the durations compared with actual durations using time laps displayed on a smart cell telephone; with an average invalidity percentage AIP% of 1.89 % and 1.33% for the joint hauling and return duration and total cycle duration, respectively

Innovative intelligent sensors to objectively understand exercise interventions for older adults

The population of most western countries is ageing and, therefore, the ageing issue now matters more than ever. According to the reports of the United Nations in 2017, there were a total of 15.8 million (26.9%) people over 60 years of age in the United Kindom, and the numbers are projected to reach 23.5 million (31.5%) by 2050. Spending on medical treatment and healthcare for older adults accounts for two-fifths of the UK National Health Service (NHS) budget. Keeping older people healthy is a challenge. In general, exercise is believed to benefit both mental and physical health. Specifically, resistance band exercises are proven by many studies that they have potentially positive effects on both mental and physical health. However, treatment using resistance band exercise is usually done in unmonitored environments, such as at home or in a rehabilitation centre; therefore, the exercise cannot be measured and/or quantified accurately. Despite many years of research, the true effectiveness of resistance band exercises remains unclear. [Continues.]</div

Transparent Authentication Utilising Gait Recognition

Securing smartphones has increasingly become inevitable due to their massive popularity and significant storage and access to sensitive information. The gatekeeper of securing the device is authenticating the user. Amongst the many solutions proposed, gait recognition has been suggested to provide a reliable yet non-intrusive authentication approach – enabling both security and usability. While several studies exploring mobile-based gait recognition have taken place, studies have been mainly preliminary, with various methodological restrictions that have limited the number of participants, samples, and type of features; in addition, prior studies have depended on limited datasets, actual controlled experimental environments, and many activities. They suffered from the absence of real-world datasets, which lead to verify individuals incorrectly. This thesis has sought to overcome these weaknesses and provide, a comprehensive evaluation, including an analysis of smartphone-based motion sensors (accelerometer and gyroscope), understanding the variability of feature vectors during differing activities across a multi-day collection involving 60 participants. This framed into two experiments involving five types of activities: standard, fast, with a bag, downstairs, and upstairs walking. The first experiment explores the classification performance in order to understand whether a single classifier or multi-algorithmic approach would provide a better level of performance. The second experiment investigated the feature vector (comprising of a possible 304 unique features) to understand how its composition affects performance and for a comparison a more particular set of the minimal features are involved. The controlled dataset achieved performance exceeded the prior work using same and cross day methodologies (e.g., for the regular walk activity, the best results EER of 0.70% and EER of 6.30% for the same and cross day scenarios respectively). Moreover, multi-algorithmic approach achieved significant improvement over the single classifier approach and thus a more practical approach to managing the problem of feature vector variability. An Activity recognition model was applied to the real-life gait dataset containing a more significant number of gait samples employed from 44 users (7-10 days for each user). A human physical motion activity identification modelling was built to classify a given individual's activity signal into a predefined class belongs to. As such, the thesis implemented a novel real-world gait recognition system that recognises the subject utilising smartphone-based real-world dataset. It also investigates whether these authentication technologies can recognise the genuine user and rejecting an imposter. Real dataset experiment results are offered a promising level of security particularly when the majority voting techniques were applied. As well as, the proposed multi-algorithmic approach seems to be more reliable and tends to perform relatively well in practice on real live user data, an improved model employing multi-activity regarding the security and transparency of the system within a smartphone. Overall, results from the experimentation have shown an EER of 7.45% for a single classifier (All activities dataset). The multi-algorithmic approach achieved EERs of 5.31%, 6.43% and 5.87% for normal, fast and normal and fast walk respectively using both accelerometer and gyroscope-based features – showing a significant improvement over the single classifier approach. Ultimately, the evaluation of the smartphone-based, gait authentication system over a long period of time under realistic scenarios has revealed that it could provide a secured and appropriate activities identification and user authentication system

Wireless sensor networks for medical care.

Chen, Xijun.Thesis (M.Phil.)--Chinese University of Hong Kong, 2008.Includes bibliographical references (leaves 72-77).Abstracts in English and Chinese.Chapter Chapter 1 --- Introduction --- p.1Chapter 1.1 --- Design Challenges --- p.2Chapter 1.2 --- Wireless Sensor Network Applications --- p.6Chapter 1.2.1 --- Military Applications --- p.7Chapter 1.2.2 --- Environmental Applications --- p.9Chapter 1.2.3 --- Health Applications --- p.11Chapter 1.3 --- Wireless Biomedical Sensor Networks (WBSN) --- p.12Chapter 1.4 --- Text Organization --- p.13Chapter Chapter 2 --- Design a Wearable Platform for Wireless Biomedical Sensor Networks --- p.15Chapter 2.1 --- Objective --- p.17Chapter 2.2 --- Requirements for Wireless Medical Sensors --- p.19Chapter 2.3 --- Hardware design --- p.21Chapter 2.3.1 --- Materials and Methods --- p.21Chapter 2.3.2 --- Results --- p.24Chapter 2.3.3 --- Conclusion --- p.27Chapter 2.4 --- Software design --- p.28Chapter 2.4.1 --- TinyOS --- p.28Chapter 2.4.2 --- Software Organization --- p.28Chapter Chapter 3 --- Wireless Medical Sensors --- p.32Chapter 3.1 --- Sensing Physiological Information --- p.32Chapter 3.1.1 --- Pulse Oximetry --- p.32Chapter 3.1.2 --- Electrocardiograph --- p.36Chapter 3.1.3 --- Galvanic Skin Response --- p.41Chapter 3.2 --- Location Tracking --- p.43Chapter 3.2.1 --- Outdoor Location Tracking --- p.43Chapter 3.2.2 --- Indoor Location Tracking --- p.44Chapter 3.3 --- Motion Tracking --- p.49Chapter 3.3.1 --- Technology --- p.50Chapter 3.3.2 --- Motion Analysis Sensor Board --- p.51Chapter 3.4 --- Discussions --- p.52Chapter Chapter 4 --- Applications in Medical Care --- p.54Chapter 4.1 --- Introduction --- p.54Chapter 4.2 --- Wearable Wireless Body Area Network --- p.56Chapter 4.2.1 --- Architecture --- p.58Chapter 4.2.2 --- Deployment Scenarios --- p.62Chapter 4.3 --- Application in Ambulatory Setting --- p.63Chapter 4.3.1 --- Method --- p.64Chapter 4.3.2 --- The Software Architecture --- p.66Chapter Chapter 5 --- Conclusions and Future Work --- p.69References --- p.72Appendix --- p.7

Exploring Hybrid Indoor Positioning Systems

Ubiquitous applications collect contextual information, process it, and then use this derived data to deliver valuable services. Location is one these contexts, and has been significant in providing navigation and guidance services for GPS devices. However, GPS is designed for outdoor use and is not precise enough, in terms of location accuracy for indoor applications. There are many indoor location systems that rely on a single technology, but these systems are either inaccurate in uncontrolled environments or require the installation of a dedicated infrastructure. This has led to the investigation of hybrid systems. This thesis examines the creation of a hybrid indoor positioning system combining different tech­ nologies and techniques; Wi-Fi access points and their associated signal strength, image analysis using machine learning to create location specific scene classifiers, and an altimeter sensor to determine the user\u27s current floor. This system is meant to provide indoor positioning data to location-aware applications