Multi-touch Detection and Semantic Response on Non-parametric Rear-projection Surfaces

The ability of human beings to physically touch our surroundings has had a profound impact on our daily lives. Young children learn to explore their world by touch; likewise, many simulation and training applications benefit from natural touch interactivity. As a result, modern interfaces supporting touch input are ubiquitous. Typically, such interfaces are implemented on integrated touch-display surfaces with simple geometry that can be mathematically parameterized, such as planar surfaces and spheres; for more complicated non-parametric surfaces, such parameterizations are not available. In this dissertation, we introduce a method for generalizable optical multi-touch detection and semantic response on uninstrumented non-parametric rear-projection surfaces using an infrared-light-based multi-camera multi-projector platform. In this paradigm, touch input allows users to manipulate complex virtual 3D content that is registered to and displayed on a physical 3D object. Detected touches trigger responses with specific semantic meaning in the context of the virtual content, such as animations or audio responses. The broad problem of touch detection and response can be decomposed into three major components: determining if a touch has occurred, determining where a detected touch has occurred, and determining how to respond to a detected touch. Our fundamental contribution is the design and implementation of a relational lookup table architecture that addresses these challenges through the encoding of coordinate relationships among the cameras, the projectors, the physical surface, and the virtual content. Detecting the presence of touch input primarily involves distinguishing between touches (actual contact events) and hovers (near-contact proximity events). We present and evaluate two algorithms for touch detection and localization utilizing the lookup table architecture. One of the algorithms, a bounded plane sweep, is additionally able to estimate hover-surface distances, which we explore for interactions above surfaces. The proposed method is designed to operate with low latency and to be generalizable. We demonstrate touch-based interactions on several physical parametric and non-parametric surfaces, and we evaluate both system accuracy and the accuracy of typical users in touching desired targets on these surfaces. In a formative human-subject study, we examine how touch interactions are used in the context of healthcare and present an exploratory application of this method in patient simulation. A second study highlights the advantages of touch input on content-matched physical surfaces achieved by the proposed approach, such as decreases in induced cognitive load, increases in system usability, and increases in user touch performance. In this experiment, novice users were nearly as accurate when touching targets on a 3D head-shaped surface as when touching targets on a flat surface, and their self-perception of their accuracy was higher

High-precision grasping and placing for mobile robots

This work presents a manipulation system for multiple labware in life science laboratories using the H20 mobile robots. The H20 robot is equipped with the Kinect V2 sensor to identify and estimate the position of the required labware on the workbench. The local features recognition based on SURF algorithm is used. The recognition process is performed for the labware to be grasped and for the workbench holder. Different grippers and labware containers are designed to manipulate different weights of labware and to realize a safe transportation

Facial feature point fitting with combined color and depth information for interactive displays

Interactive displays are driven by natural interaction with the user, necessitating a computer system that recognizes body gestures and facial expressions. User inputs are not easily or reliably recognized for a satisfying user experience, as the complexities of human communication are difficult to interpret in real-time. Recognizing facial expressions in particular is a problem that requires high-accuracy and efficiency for stable interaction environments. The recent availability of the Kinect, a low cost, low resolution sensor that supplies simultaneous color and depth images, provides a breakthrough opportunity to enhance the interactive capabilities of displays and overall user experience. This new RGBD (RGB + depth) sensor generates an additional channel of depth information that can be used to improve the performance of existing state of the art technology and develop new techniques. The Active Shape Model (ASM) is a well-known deformable model that has been extensively studied for facial feature point placement. Previous shape model techniques have applied 3D reconstruction techniques using multiple cameras or other statistical methods for producing 3D information from 2D color images. These methods showed improved results compared to using only color data, but required an additional deformable model or expensive imaging equipment. In this thesis, an ASM model is trained using the RGBD image produced by the Kinect. The real-time information from the depth sensor is registered to the color image to create a pixel-for-pixel match. To improve the quality of the depth image, a temporal median filter is applied to reduce random noise produced by the sensor. The resulting combined model is designed to produce more robust fitting of facial feature points compared to a purely color based active shape model

Development platform for elderly-oriented tabletop games

Tese de mestrado integrado. Engenharia Informática e Computação. Universidade do Porto. Faculdade de Engenharia. 201

A Contextual Inquiry of People with Vision Impairments in Cooking

Individuals with vision impairments employ a variety of strategies for object identification, such as pans or soy sauce, in the culinary process. In addition, they often rely on contextual details about objects, such as location, orientation, and current status, to autonomously execute cooking activities. To understand how people with vision impairments collect and use the contextual information of objects while cooking, we conducted a contextual inquiry study with 12 participants in their own kitchens. This research aims to analyze object interaction dynamics in culinary practices to enhance assistive vision technologies for visually impaired cooks. We outline eight different types of contextual information and the strategies that blind cooks currently use to access the information while preparing meals. Further, we discuss preferences for communicating contextual information about kitchen objects as well as considerations for the deployment of AI-powered assistive technologies.Comment: CHI 202

Development of new intelligent autonomous robotic assistant for hospitals

Continuous technological development in modern societies has increased the quality of life and average life-span of people. This imposes an extra burden on the current healthcare infrastructure, which also creates the opportunity for developing new, autonomous, assistive robots to help alleviate this extra workload. The research question explored the extent to which a prototypical robotic platform can be created and how it may be implemented in a hospital environment with the aim to assist the hospital staff with daily tasks, such as guiding patients and visitors, following patients to ensure safety, and making deliveries to and from rooms and workstations. In terms of major contributions, this thesis outlines five domains of the development of an actual robotic assistant prototype. Firstly, a comprehensive schematic design is presented in which mechanical, electrical, motor control and kinematics solutions have been examined in detail. Next, a new method has been proposed for assessing the intrinsic properties of different flooring-types using machine learning to classify mechanical vibrations. Thirdly, the technical challenge of enabling the robot to simultaneously map and localise itself in a dynamic environment has been addressed, whereby leg detection is introduced to ensure that, whilst mapping, the robot is able to distinguish between people and the background. The fourth contribution is geometric collision prediction into stabilised dynamic navigation methods, thus optimising the navigation ability to update real-time path planning in a dynamic environment. Lastly, the problem of detecting gaze at long distances has been addressed by means of a new eye-tracking hardware solution which combines infra-red eye tracking and depth sensing. The research serves both to provide a template for the development of comprehensive mobile assistive-robot solutions, and to address some of the inherent challenges currently present in introducing autonomous assistive robots in hospital environments.Open Acces

Study and Characterization of a Camera-based Distributed System for Large-Volume Dimensional Metrology Applications

Large-Volume Dimensional Metrology (LVDM) deals with dimensional inspection of large objects with dimensions in the order of tens up to hundreds of meters. Typical large volume dimensional metrology applications concern the assembly/disassembly phase of large objects, referring to industrial engineering. Based on different technologies and measurement principles, a wealth of LVDM systems have been proposed and developed in the literature, just to name a few, e.g., optical based systems such as laser tracker, laser radar, and mechanical based systems such as gantry CMM and multi-joints artificial arm CMM, and so on. Basically, the main existing LVDM systems can be divided into two categories, i.e. centralized systems and distributed systems, according to the scheme of hardware configuration. By definition, a centralized system is a stand-alone unit which works independently to provide measurements of a spatial point, while a distributed system, is defined as a system that consists of a series of sensors which work cooperatively to provide measurements of a spatial point, and usually individual sensor cannot measure the coordinates separately. Some representative distributed systems in the literature are iGPS, MScMS-II, and etc. The current trend of LVDM systems seem to orient towards distributed systems, and actually, distributed systems demonstrate many advantages that distinguish themselves from conventional centralized systems

Advancing the objective measurement of physical activity and sedentary behaviour context

Objective data from national surveillance programmes show that, on average, individuals accumulate high amounts of sedentary time per day and only a small minority of adults achieve physical activity guidelines. One potential explanation for the failure of interventions to increase population levels of physical activity or decrease sedentary time is that research to date has been unable to identify the specific behavioural levers in specific contexts needed to change behaviour. Novel technology is emerging with the potential to elucidate these specific behavioural contexts and thus identify these specific behavioural levers. Therefore the aims of this four study thesis were to identify novel technologies capable of measuring the behavioural context, to evaluate and validate the most promising technology and to then pilot this technology to assess the behavioural context of older adults, shown by surveillance programmes to be the least physically active and most sedentary age group. Study one Purpose: To identify, via a systematic review, technologies which have been used or could be used to measure the location of physical activity or sedentary behaviour. Methods: Four electronic databases were searched using key terms built around behaviour, technology and location. To be eligible for inclusion papers were required to be published in English and describe a wearable or portable technology or device capable of measuring location. Searches were performed from the inception of the database up to 04/02/2015. Searches were also performed using three internet search engines. Specialised software was used to download search results and thus mitigate the potential pitfalls of changing search algorithms. Results: 188 research papers met the inclusion criteria. Global positioning systems were the most widely used location technology in the published research, followed by wearable cameras and Radio-frequency identification. Internet search engines identified 81 global positioning systems, 35 real-time locating systems and 21 wearable cameras. Conclusion: The addition of location information to existing measures of physical activity and sedentary behaviour will provide important behavioural information. Study Two Purpose: This study investigated the Actigraph proximity feature across three experiments. The aim of Experiment One was to assess the basic characteristics of the Actigraph RSSI signal across a range of straight line distances. Experiment Two aimed to assess the level of receiver device signal detection in a single room under unobstructed conditions, when various obstructions are introduced and the impacts these obstructions have on the intra and inter unit variability of the RSSI signal. Finally, Experiment Three aimed to assess signal contamination across multiple rooms (i.e. one beacon being detected in multiple rooms). Methods: Across all experiments, the receiver(s) collected data at 10 second epochs, the highest resolution possible. In Experiment One two devices, one receiver and one beacon, were placed opposite each other at 10cm increments for one minute at each distance. The RSSI-distance relationship was then visually assessed for linearity. In Experiment Two, a test room was demarcated into 0.5 x 0.5 m grids with receivers simultaneously placed in each demarcated grid. This process was then repeated under wood, metal and human obstruction conditions. Descriptive tallies were used to assess the signal detection achieved for each receiver from each beacon in each grid. Mean RSSI signal was calculated for each condition alongside intra and inter-unit standard deviation, coefficient of variation and standard error of the measurement. In Experiment Three, a test apartment was used with three beacons placed across two rooms. The researcher then completed simulated conditions for 10 minutes each across the two rooms. The percentage of epochs where a signal was detected from each of the three beacons across each test condition was then calculated. Results: In Experiment One, the relationship between RSSI and distance was found to be non-linear. In Experiment Two, high signal detection was achieved in all conditions; however, there was a large degree of intra and inter-unit variability in RSSI. In Experiment Three, there was a large degree of multi-room signal contamination. Conclusion: The Actigraph proximity feature can provide a binary indicator of room level location. Study Three Purpose: To use novel technology in three small feasibility trials to ascertain where the greatest utility can be demonstrated. Methods: Feasibility Trial One assessed the concurrent validity of electrical energy monitoring and wearable cameras as measures of television viewing. Feasibility Trial Two utilised indoor location monitoring to assess where older adult care home residents accumulate their sedentary time. Lastly, Feasibility Trial Three investigated the use of proximity sensors to quantify exposure to a height adjustable desk Results: Feasibility Trial One found that on average the television is switched on for 202 minutes per day but is visible in just 90 minutes of wearable camera images with a further 52 minutes where the participant is in their living room but the television is not visible in the image. Feasibility Trial Two found that residents were highly sedentary (sitting for an average of 720 minutes per day) and spent the majority of their time in their own rooms with more time spent in communal areas in the morning than in the afternoon. Feasibility Trial Three found a discrepancy between self-reported work hours and objectively measured office dwell time. Conclusion: The feasibility trials outlined in this study show the utility of objectively measuring context to provide more detailed and refined data. Study Four Purpose: To objectively measure the context of sedentary behaviour in the most sedentary age group, older adults. Methods: 26 residents and 13 staff were recruited from two care homes. Each participant wore an Actigraph GT9X on their non-dominant wrist and a LumoBack posture sensor on their lower back for one week. The Actigraph recorded proximity every 10 seconds and acceleration at 100 Hz. LumoBack data were provided as summaries per 5 minutes. Beacon Actigraphs were placed around each care home in the resident s rooms, communal areas and corridors. Proximity and posture data were combined in 5 minute epochs with descriptive analysis of average time spent sitting in each area produced. Acceleration data were summarised into 10 second epochs and combined with proximity data to show the average count per epoch in each area of the care home. Mann-Whitney tests were performed to test for differences between care homes. Results: No significant differences were found between Care Home One and Care Home Two in the amount of time spent sitting in communal areas of the care home (301 minutes per day and 39 minutes per day respectively, U=23, p=0.057) or in the amount of time residents spent sitting in their own room (215 minutes per day and 337 minutes per day in Care Home One and Two respectively, U=32, p=0.238). In both care homes, accelerometer measured average movement increases with the number of residents in the communal area. Conclusion: The Actigraph proximity system was able to quantify the context of sedentary behaviour in older adults. This enabled the identification of levers for behaviour change which can be used to reduce sedentary time in this group. Overall conclusion: There are a large number of technologies available with the potential to measure the context of physical activity or sedentary time. The Actigraph proximity feature is one such technology. This technology is able to provide a binary measure of proximity via the detection or non-detection of Bluetooth signal: however, the variability of the signal prohibits distance estimation. The Actigraph proximity feature, in combination with a posture sensor, is able to elucidate the context of physical activity and sedentary time

THE EFFECTS OF OBESITY ON RESULTANT KNEE JOINT LOADS FOR GAIT AND CYCLING

Osteoarthritis (OA) is a degenerative disease of cartilage and bone tissue and the most common form of arthritis, accounting for US$ 10.5 billion in hospital charges in 2006. Obesity (OB) has been linked to increased risk of developing knee OA due to increased knee joint loads and varus-valgus misalignment. Walking is recommended as a weight-loss activity but it may increase risk of knee OA as OB gait increases knee loads. Cycling has been proposed as an alternative weight-loss measure, however, lack of studies comparing normal weight (NW) and OB subjects in cycling and gait hinder identification of exercises that may best prevent knee OA incidence. The objective of this work is to determine if cycling is a better weight-loss exercise than gait in OB subjects as it relates to knee OA risk reduction due to decreased knee loads. A stationary bicycle was modified to measure forces and moments at the pedals in three dimensions. A pilot experiment was performed to calculate resultant knee loads during gait and cycling for NW (n = 4) and OB (n = 4) subjects. Statistical analyses were performed to compare knee loads and knee angles, and to determine statistical significance of results (p \u3c 0.05). Cycling knee loads were lower than gait knee loads for all subjects (p \u3c 0.033). OB axial knee loads were higher than NW axial knee loads in gait (p = 0.004) due to the weight-bearing nature of gait. No differences were observed in cycling knee loads between NW and OB subjects, suggesting cycling returns OB knee loads and biomechanics to normal levels. The lack of significant results in cycling could be due to the small sample size used or because rider weight is supported by the seat. Limitations to this study include small sample size, soft tissue artifact, and experimental errors in marker placement. Future studies should correct these limitations and find knee joint contact force rather than knee resultant loads using v EMG-driven experiments. In conclusion, cycling loads were lower than gait loads for NW and OB subjects suggesting cycling is a better weight-loss exercise than gait in the context of reducing knee OA risk

Päämallien tuottaminen lasten aivojen diffuusiin optiseen tomografiaan

Diffuse optical tomography (DOT) provides a practical method for studying cortical brain activity in toddlers. In DOT, tissue is illuminated with near-infrared light to estimate the three-dimensional distribution of optical parameter changes that result from the hemodynamic response to neuronal activity. This is an inverse problem where the correctness of the result is very sensitive to the accuracy of the forward solution. The forward solution requires an optical model of the head, including a segmented anatomical model, locations of the optodes (sources and detectors) and the baseline optical parameters of the tissue types. This thesis presents improved child-friendly methods for generating the optical model. The first part of the methods is for registering an age-appropriate atlas template to points measured with photogrammetry from the subject's head. The second part enables interpolating the locations of the optodes on the deformed template using the known layout of the measurement probe. The final algorithm performs a Gaussian random search in the parameter space to find the optical parameters. In this thesis, the first two methods were applied for 17 two-year-olds participating in an affective touch study. The average accuracy of the surface registration was 1.7 mm, and the interpolation reconstructed the optode layout with an average error of 0.6 mm. The probe positions also corresponded to the placements according to the photographs. Further study will combine these models with measurements to estimate the optical parameters of two-year-olds, which current literature does not provide.Diffuusi optinen tomografia (DOT) on käytännöllinen menetelmä lasten aivokuoren aktivaatioiden tutkimiseen. Aivohermosolujen sähköinen aktivaatio muuttaa paikallista verenkiertoa ja sitä kautta kudoksen optisia ominaisuuksia. DOT-menetelmässä kudosta läpivalaistaan lähi-infrapunavalolla, jotta saadaan kuvattua optisten parametrien muutosten kolmiulotteinen jakauma. Kyseessä olevan käänteisongelman ratkaisu on erittäin herkkä vastaavan suoran ongelman ratkaisun tarkkuudelle. Suoran ongelman ratkaisemiseksi tarvitaan optinen päämalli, johon kuuluu segmentoitu anatominen malli, optisten kuitujen (valon lähteet ja detektorit) paikat ja kudostyyppien optiset taustaparametrit. Tässä työssä esitetään lapsille suotuisa menetelmäkokonaisuus optisen päämallin tuottamiseksi. Ensiksi kohdehenkilön ikäluokkaa vastaava atlas-malli rekisteröidään henkilön päänpinnan muotoa vastaavaksi. Pään muoto on esitetty fotogrammetrialla mitattujen pisteiden avulla. Tämän jälkeen kuitujen paikat interpoloidaan mallin pinnalla hyödyntämällä niiden tunnettua sijoittelua mittauspäätteessä. Lopuksi valitaan satunnaisesti joukko optisia parametreja kullekin kudostyypille, ja etsitään simuloitua mittausdataa vastaavia suoran ongelman ratkaisuja tuottavat vaihtoehdot. Tässä työssä tuotettiin kuitujen paikoilla varustetut anatomiset mallit seitsemälletoista kaksivuotiaalle, jotka osallistuivat hellää kosketusta käsittelevään tutkimukseen. Rekisteröinnin keskimääräinen tarkkuus oli 1.7 mm, ja kuitujen sijoittelu vastasi tunnettua rakennetta keskimäärin 0.6 millimetrin tarkkuudella. Lisäksi mittauspäätteen sijainti ja asento näyttivät samoilta kuin valokuvissa. Muodostettuja malleja käytetään jatkossa todellisten optisten parametrien määrittämiseen, koska näitä arvoja ei löydy vielä kirjallisuudesta