65 research outputs found
Intelligent ultrasound hand gesture recognition system
With the booming development of technology, hand gesture recognition has become a hotspot in Human-Computer Interaction (HCI) systems. Ultrasound hand gesture recognition is an innovative method that has attracted ample interest due to its strong real-time performance, low cost, large field of view, and illumination independence. Well-investigated HCI applications include external digital pens, game controllers on smart mobile devices, and web browser control on laptops. This thesis probes gesture recognition systems on multiple platforms to study the behavior of system performance with various gesture features. Focused on this topic, the contributions of this thesis can be summarized from the perspectives of smartphone acoustic field and hand model simulation, real-time gesture recognition on smart devices with speed categorization algorithm, fast reaction gesture recognition based on temporal neural networks, and angle of arrival-based gesture recognition system.
Firstly, a novel pressure-acoustic simulation model is developed to examine its potential for use in acoustic gesture recognition. The simulation model is creating a new system for acoustic verification, which uses simulations mimicking real-world sound elements to replicate a sound pressure environment as authentically as possible. This system is fine-tuned through sensitivity tests within the simulation and validate with real-world measurements. Following this, the study constructs novel simulations for acoustic applications, informed by the verified acoustic field distribution, to assess their effectiveness in specific devices. Furthermore, a simulation focused on understanding the effects of the placement of sound devices and hand-reflected sound waves is properly designed. Moreover, a feasibility test on phase control modification is conducted, revealing the practical applications and boundaries of this model.
Mobility and system accuracy are two significant factors that determine gesture recognition performance. As smartphones have high-quality acoustic devices for developing gesture recognition, to achieve a portable gesture recognition system with high accuracy, novel algorithms were developed to distinguish gestures using smartphone built-in speakers and microphones. The proposed system adopts Short-Time-Fourier-Transform (STFT) and machine learning to capture hand movement and determine gestures by the pretrained neural network. To differentiate gesture speeds, a specific neural network was designed and set as part of the classification algorithm. The final accuracy rate achieves 96% among nine gestures and three speed levels. The proposed algorithms were evaluated comparatively through algorithm comparison, and the accuracy outperformed state-of-the-art systems.
Furthermore, a fast reaction gesture recognition based on temporal neural networks was designed. Traditional ultrasound gesture recognition adopts convolutional neural networks that have flaws in terms of response time and discontinuous operation. Besides, overlap intervals in network processing cause cross-frame failures that greatly reduce system performance. To mitigate these problems, a novel fast reaction gesture recognition system that slices signals in short time intervals was designed. The proposed system adopted a novel convolutional recurrent neural network (CRNN) that calculates gesture features in a short time and combines features over time. The results showed the reaction time significantly reduced from 1s to 0.2s, and accuracy improved to 100% for six gestures.
Lastly, an acoustic sensor array was built to investigate the angle information of performed gestures. The direction of a gesture is a significant feature for gesture classification, which enables the same gesture in different directions to represent different actions. Previous studies mainly focused on types of gestures and analyzing approaches (e.g., Doppler Effect and channel impulse response, etc.), while the direction of gestures was not extensively studied. An acoustic gesture recognition system based on both speed information and gesture direction was developed. The system achieved 94.9% accuracy among ten different gestures from two directions. The proposed system was evaluated comparatively through numerical neural network structures, and the results confirmed that incorporating additional angle information improved the system's performance.
In summary, the work presented in this thesis validates the feasibility of recognizing hand gestures using remote ultrasonic sensing across multiple platforms. The acoustic simulation explores the smartphone acoustic field distribution and response results in the context of hand gesture recognition applications. The smartphone gesture recognition system demonstrates the accuracy of recognition through ultrasound signals and conducts an analysis of classification speed. The fast reaction system proposes a more optimized solution to address the cross-frame issue using temporal neural networks, reducing the response latency to 0.2s. The speed and angle-based system provides an additional feature for gesture recognition. The established work will accelerate the development of intelligent hand gesture recognition, enrich the available gesture features, and contribute to further research in various gestures and application scenarios
Underwater 3D positioning on smart devices
The emergence of water-proof mobile and wearable devices (e.g., Garmin
Descent and Apple Watch Ultra) designed for underwater activities like
professional scuba diving, opens up opportunities for underwater networking and
localization capabilities on these devices. Here, we present the first
underwater acoustic positioning system for smart devices. Unlike conventional
systems that use floating buoys as anchors at known locations, we design a
system where a dive leader can compute the relative positions of all other
divers, without any external infrastructure. Our intuition is that in a
well-connected network of devices, if we compute the pairwise distances, we can
determine the shape of the network topology. By incorporating orientation
information about a single diver who is in the visual range of the leader
device, we can then estimate the positions of all the remaining divers, even if
they are not within sight. We address various practical problems including
detecting erroneous distance estimates, addressing rotational and flipping
ambiguities as well as designing a distributed timestamp protocol that scales
linearly with the number of devices. Our evaluations show that our distributed
system running on underwater deployments of 4-5 commodity smart devices can
perform pairwise ranging and localization with median errors of 0.5-0.9 m and
0.9-1.6
Pattern detection platform using disruptive technologies to improve people’s daily tasks
Tesis por compendio de publicaciones[ES] En los últimos años la miniaturización de los dispositivos electrónicos y el abaratamiento
de los procesos de fabricación de los componentes ha permitido que las redes de
sensores inalámbricas sean cada vez mas importantes y se empleen en multitud de casos.
Adicionalmente, y debido en parte a la mejora en cuanto a las capacidades de almacenamiento
y procesamiento de datos se refiere, ha permitido construir sistemas sensibles al
contexto en áreas como la medicina, la monitorización o la robótica que permiten hacer
un análisis detallado y adaptable de los procesos y servicios que se pueden proporcionar
a los usuarios. Esta tesis doctoral ha sido conformada mediante un “Compendio de Artículos”
donde se analiza la aplicación de paradigmas de inteligencia artificial en 3 casos
de estudio claramente diferenciados. Se ha planteado un novedoso sistema de localización
en interiores que hace uso de técnicas bayesianas y fingerprinting, con objeto de
automatizar y facilitar los procesos de adquisición de datos de calibración. A mayores,
se presenta un exoesqueleto que es conectado a una arquitectura sensible al contexto con
objeto de que los pacientes de rehabilitación hagan ejercicios de forma interactiva y haciendo
uso de técnicas de realidad aumentada. En el último artículo, se hace hincapié en
el diseño de una plataforma que hace uso de las redes inalámbricas de sensores, con objeto
de monitorizar el estado de los aseos mediante la incorporación de agentes embebidos
en dispositivos limitados computacionalmente. Esta información descentralizada es analizada
con objeto de detectar posibles anomalías y facilitar la toma de decisiones. Uno de
los principales hitos que se pretende con el estudio, es mostrar a la comunidad científica
los diferentes resultados que se han obtenido en la investigación, solventando problemas
cotidianos que han sido resueltos mediante la modelización de los casos de estudio mediante
la utilización de arquitecturas multi-agente y sistemas expertos. El filtrado de
señales, la utilización de clasificadores, minería de datos y la utilización de otras técnicas
de Inteligencia Artificial han sido empleadas para la consecución exitosa de este trabajo
Audio beacon technologies, surveillance and social order
This thesis explores audio beacon technology with the aim of elucidating the implications of this technology for the individual in contemporary society. Audio beacons are hidden inside digital devices. They emit and receive high frequency audio signals which are inaudible to the human ear, thereby generating and transmitting data without our knowledge. The motivation for this research is to raise awareness of the prevalence of audio beacon technologies and to explore their implications for contemporary society. The research takes an interdisciplinary approach involving – 1) a survey of audio beacon technology, 2) a contextualization in terms of contemporary theories of surveillance and control and 3) an interpretation in terms of 20th century dystopian literature. The hidden surveillance and privacy of this technology is examined mainly through the humanistic perspective of George Orwell’s book Nineteen Eighty-Four. The general conclusion formed is that audio beacon technologies can serve as a surveillance method enhancing authoritarian and exploitative regimes. To mitigate the negative impacts of audio beacons, this research proposes two types of solutions – 1) individual actions that will have an immediate effect and 2) governmental legislation that can improve privacy in the longer term. Both of these solutions cannot happen without a raised public awareness, towards which this research hopes to make a contribution. Finally, this research introduces the notion of a \u27digital paradox\u27 in which the dystopian worlds of George Orwell and Aldous Huxley are brought together in order to characterize surveillance and control in contemporary society
LOCATE-US: Indoor Positioning for Mobile Devices Using Encoded Ultrasonic Signals, Inertial Sensors and Graph- Matching
Indoor positioning remains a challenge and, despite much research and development carried out in the last decade, there is still no standard as with the Global Navigation Satellite Systems (GNSS) outdoors. This paper presents an indoor positioning system called LOCATE-US with adjustable granularity for use with commercial mobile devices, such as smartphones or tablets. LOCATE-US is privacy-oriented and allows every device to compute its own position by fusing ultrasonic, inertial sensor measurements and map information. Ultrasonic Local Positioning Systems (ULPS) based on encoded signals are placed in critical zones that require an accuracy below a few decimeters to correct the accumulated drift errors of the inertial measurements. These systems are well suited to work at room level as walls confine acoustic waves inside. To avoid audible artifacts, the U-LPS emission is set at 41.67 kHz, and an ultrasonic acquisition module with reduced dimensions is attached to the mobile device through the USB port to capture signals. Processing in the mobile device involves an improved Time Differences of Arrival (TDOA) estimation that is fused with the measurements from an external inertial sensor to obtain real-time location and trajectory display at a 10 Hz rate. Graph-matching has also been included, considering available prior knowledge about the navigation scenario. This kind of device is an adequate platform for Location-Based Services (LBS), enabling applications such as augmented reality, guiding applications, or people monitoring and assistance. The system architecture can easily incorporate new sensors in the future, such as UWB, RFiD or others.Universidad de AlcaláJunta de Comunidades de Castilla-La ManchaAgencia Estatal de Investigació
Applications across Co-located Devices
We live surrounded by many computing devices. However, their presence has yet to
be fully explored to create a richer ubiquitous computing environment. There is an
opportunity to take better advantage of those devices by combining them into a unified
user experience. To realize this vision, we studied and explored the use of a framework,
which provides the tools and abstractions needed to develop applications that distribute
UI components across co-located devices.
The framework comprises the following components: authentication and authorization
services; a broker to sync information across multiple application instances; background
services that gather the capabilities of the devices; and a library to integrate
web applications with the broker, determine which components to show based on UI
requirements and device capabilities, and that provides custom elements to manage the
distribution of the UI components and the multiple application states. Collaboration
between users is supported by sharing application states. An indoor positioning solution
had to be developed in order to determine when devices are close to each other to trigger
the automatic redistribution of UI components.
The research questions that we set out to respond are presented along with the contributions
that have been produced. Those contributions include a framework for crossdevice
applications, an indoor positioning solution for pervasive indoor environments,
prototypes, end-user studies and developer focused evaluation. To contextualize our
research, we studied previous research work about cross-device applications, proxemic
interactions and indoor positioning systems.
We presented four application prototypes. The first three were used to perform studies
to evaluate the user experience. The last one was used to study the developer experience
provided by the framework. The results were largely positive with users showing preference
towards using multiple devices under some circumstances. Developers were also
able to grasp the concepts provided by the framework relatively well.Vivemos rodeados de dispositivos computacionais. No entanto, ainda não tiramos partido
da sua presença para criar ambientes de computação ubíqua mais ricos. Existe uma
oportunidade de combiná-los para criar uma experiência de utilizador unificada. Para
realizar esta visão, estudámos e explorámos a utilização de uma framework que forneça
ferramentas e abstrações que permitam o desenvolvimento de aplicações que distribuem
os componentes da interface do utilizador por dispositivos co-localizados.
A framework é composta por: serviços de autenticação e autorização; broker que sincroniza
informação entre várias instâncias da aplicação; serviços que reúnem as capacidades
dos dispositivos; e uma biblioteca para integrar aplicações web com o broker, determinar
as componentes a mostrar com base nos requisitos da interface e nas capacidades dos
dispositivos, e que disponibiliza elementos para gerir a distribuição dos componentes da
interface e dos estados de aplicação. A colaboração entre utilizadores é suportada através
da partilha dos estados de aplicação. Foi necessário desenvolver um sistema de posicionamento
em interiores para determinar quando é que os dispositivos estão perto uns dos
outros para despoletar a redistribuição automática dos componentes da interface.
As questões de investigação inicialmente colocadas são apresentadas juntamente com
as contribuições que foram produzidas. Essas contribuições incluem uma framework para
aplicações multi-dispositivo, uma solução de posicionamento em interiores para computação
ubíqua, protótipos, estudos com utilizadores finais e avaliação com programadores.
Para contextualizar a nossa investigação, estudámos trabalhos anteriores sobre aplicações
multi-dispositivo, interação proxémica e sistemas de posicionamento em interiores.
Apresentámos quatro aplicações protótipo. As primeiras três foram utilizadas para
avaliar a experiência de utilização. A última foi utilizada para estudar a experiência
de desenvolvimento com a framework. Os resultados foram geralmente positivos, com
os utilizadores a preferirem utilizar múltiplos dispositivos em certas circunstâncias. Os
programadores também foram capazes de compreender a framework relativamente bem
Epidemic contact tracing with smartphone sensors
Contact tracing is widely considered as an effective procedure in the fight
against epidemic diseases. However, one of the challenges for technology based
contact tracing is the high number of false positives, questioning its
trust-worthiness and efficiency amongst the wider population for mass adoption.
To this end, this paper proposes a novel, yet practical smartphone-based
contact tracing approach, employing WiFi and acoustic sound for relative
distance estimate, in addition to the air pressure and the magnetic field for
ambient environment matching. We present a model combining 6 smartphone
sensors, prioritising some of them when certain conditions are met. We
empirically verified our approach in various realistic environments to
demonstrate an achievement of up to 95% fewer false positives, and 62% more
accurate than Bluetooth-only system. To the best of our knowledge, this paper
was one of the first work to propose a combination of smartphone sensors for
contact tracing
Recommended from our members
Multi-Mobile Computing
With mobile systems evermore ubiquitous, individual users often own multiple mobile systems and groups of users often have many mobile systems at their disposal. As a result, there is a growing demand for multi-mobile computing, the ability to combine the functionality of multiple mobile systems into a more capable one. However, there are several key challenges. First, mobile systems are highly heterogeneous with different software and hardware, each with their own interfaces and data formats. Second, there are no effective ways to allow users to easily and dynamically compose together multiple mobile systems for the quick interactions that typically take place with mobile systems. Finally, there is a lack of system infrastructure to allow existing apps to make use of multiple mobile systems, or to enable developers to write new multi-mobile aware apps. My thesis is that higher-level abstractions of mobile operating systems can be reused to combine heterogeneous mobile systems into a more capable one and enable existing and new apps to provide new functionality across multiple mobile systems.
First, we present M2, a system for multi-mobile computing that enables existing unmodified mobile apps to share and combine multiple devices, including cameras, displays, speakers, microphones, sensors, GPS, and input. To support heterogeneous devices, M2 introduces a new data-centric approach that leverages higher-level device abstractions and hardware acceleration to efficiently share device data, not API calls. M2 introduces device transformation, a new technique to mix and match heterogeneous devices, enabling, for example, existing apps to leverage a single larger display fused from multiple displays for better viewing, or use a Nintendo Wii-like gaming experience by translating accelerometer to touchscreen input. We have implemented M2 and show that it operates across heterogeneous systems, including multiple versions of Android and iOS, and can run existing apps across mobile systems with modest overhead and qualitative performance indistinguishable from using local device hardware.
Second, we present Tap, a framework that leverages M2’s data-centric architecture to make it easy for users to dynamically compose collections of mobile systems and developers to write new multi-mobile apps that make use of those impromptu collections. Tap allows users to simply tap systems together to compose them into a collection without the need for users to register or connect to any cloud infrastructure. Tap makes it possible for apps to use existing mobile platform APIs across multiple mobile systems by virtualizing data sources so that local and remote data sources can be combined together upon tapping. Virtualized data sources can be hardware or software features, including media, clipboard, calendar events, and devices such as cameras and microphones. Leveraging existing mobile platform APIs make it easy for developers to write apps that use hard- ware and software features across dynamically composed collections of mobile systems. We have implemented Tap and show that it provides good usability for dynamically composing multiple mobile systems and good performance for sharing hardware devices and software features across multiple mobile systems.
Finally, using M2 and Tap, we present various apps that show how existing apps can provide useful functionality across multiple mobile systems and how new apps can be easily developed to provide new multi-mobile functionality. Examples include panoramic video recording using cameras from multiple mobile systems, surround sound music player app that configures itself based on automatically detecting the location of multiple mobile systems, and an added feature to the Snapchat app that allows multiple users to share a live Snap, using their own cameras and filters. Our user studies with these apps show that multi-mobile computing offers a richer and more enhanced experience for users and a much simpler development effort for developers
- …