16,432 research outputs found
Proprioceptive Learning with Soft Polyhedral Networks
Proprioception is the "sixth sense" that detects limb postures with motor
neurons. It requires a natural integration between the musculoskeletal systems
and sensory receptors, which is challenging among modern robots that aim for
lightweight, adaptive, and sensitive designs at a low cost. Here, we present
the Soft Polyhedral Network with an embedded vision for physical interactions,
capable of adaptive kinesthesia and viscoelastic proprioception by learning
kinetic features. This design enables passive adaptations to omni-directional
interactions, visually captured by a miniature high-speed motion tracking
system embedded inside for proprioceptive learning. The results show that the
soft network can infer real-time 6D forces and torques with accuracies of
0.25/0.24/0.35 N and 0.025/0.034/0.006 Nm in dynamic interactions. We also
incorporate viscoelasticity in proprioception during static adaptation by
adding a creep and relaxation modifier to refine the predicted results. The
proposed soft network combines simplicity in design, omni-adaptation, and
proprioceptive sensing with high accuracy, making it a versatile solution for
robotics at a low cost with more than 1 million use cycles for tasks such as
sensitive and competitive grasping, and touch-based geometry reconstruction.
This study offers new insights into vision-based proprioception for soft robots
in adaptive grasping, soft manipulation, and human-robot interaction.Comment: 20 pages, 10 figures, 2 tables, submitted to the International
Journal of Robotics Research for revie
Understanding face and eye visibility in front-facing cameras of smartphones used in the wild
Commodity mobile devices are now equipped with high-resolution front-facing cameras, allowing applications in biometrics (e.g., FaceID in the iPhone X), facial expression analysis, or gaze interaction. However, it is unknown how often users hold devices in a way that allows capturing their face or eyes, and how this impacts detection accuracy. We collected 25,726 in-the-wild photos, taken from the front-facing camera of smartphones as well as associated application usage logs. We found that the full face is visible about 29% of the time, and that in most cases the face is only partially visible. Furthermore, we identified an influence of users' current activity; for example, when watching videos, the eyes but not the entire face are visible 75% of the time in our dataset. We found that a state-of-the-art face detection algorithm performs poorly against photos taken from front-facing cameras. We discuss how these findings impact mobile applications that leverage face and eye detection, and derive practical implications to address state-of-the art's limitations
Exploring the Front Touch Interface for Virtual Reality Headsets
In this paper, we propose a new interface for virtual reality headset: a
touchpad in front of the headset. To demonstrate the feasibility of the front
touch interface, we built a prototype device, explored VR UI design space
expansion, and performed various user studies. We started with preliminary
tests to see how intuitively and accurately people can interact with the front
touchpad. Then, we further experimented various user interfaces such as a
binary selection, a typical menu layout, and a keyboard. Two-Finger and
Drag-n-Tap were also explored to find the appropriate selection technique. As a
low-cost, light-weight, and in low power budget technology, a touch sensor can
make an ideal interface for mobile headset. Also, front touch area can be large
enough to allow wide range of interaction types such as multi-finger
interactions. With this novel front touch interface, we paved a way to new
virtual reality interaction methods
An original framework for understanding human actions and body language by using deep neural networks
The evolution of both fields of Computer Vision (CV) and Artificial Neural Networks (ANNs) has allowed the development of efficient automatic systems for the analysis of people's behaviour.
By studying hand movements it is possible to recognize gestures, often used by people to communicate information in a non-verbal way.
These gestures can also be used to control or interact with devices without physically touching them. In particular, sign language and semaphoric hand gestures are the two foremost areas of interest due to their importance in Human-Human Communication (HHC) and Human-Computer Interaction (HCI), respectively.
While the processing of body movements play a key role in the action recognition and affective computing fields. The former is essential to understand how people act in an environment, while the latter tries to interpret people's emotions based on their poses and movements;
both are essential tasks in many computer vision applications, including event recognition, and video surveillance.
In this Ph.D. thesis, an original framework for understanding Actions and body language is presented. The framework is composed of three main modules: in the first one, a Long Short Term Memory Recurrent Neural Networks (LSTM-RNNs) based method for the Recognition of Sign Language and Semaphoric Hand Gestures is proposed; the second module presents a solution based on 2D skeleton and two-branch stacked LSTM-RNNs for action recognition in video sequences; finally, in the last module, a solution for basic non-acted emotion recognition by using 3D skeleton and Deep Neural Networks (DNNs) is provided.
The performances of RNN-LSTMs are explored in depth, due to their ability to model the long term contextual information of temporal sequences, making them suitable for analysing body movements.
All the modules were tested by using challenging datasets, well known in the state of the art, showing remarkable results compared to the current literature methods
Interfaces baseadas em gestos e movimento
Dissertação de Mestrado em Engenharia InformáticaEsta tese estuda novas formas de interacção pessoa-máquina, baseadas em sensores de infravermelhos. O objectivo foi criar uma interface que tornasse a interacção com o computador mais natural e divertida, utilizando gestos e movimentos que são usados intuitivamente no dia-a-dia.
Foi necessário o desenho e implementação de um sistema flexível e modular, que permite
detectar as posições e movimentos das mãos e cabeça do utilizador. Adicionalmente, esta
interface tambem permite a utilização de botões e fornece feedback háptico ao utilizador. Foram encontrados vários problemas durante a realização do hardware, que levaram à utilização de novas abordagens e à construcção e teste de vários protótipos
Paralelamente à construção dos protótipos do hardware, foi implementada uma biblioteca que
permite detectar a posição das mãos e cabeça cabeça do utilizador, num espaço tridimensional.
Esta biblioteca trata de toda a comunicação com o hardware, fornecendo funções e callbacks simples ao programador das aplicações.
Foram desenvolvidas quatro aplicações que permitiram testar e demonstrar as várias
funcionalidades desta interface em diferentes cenários. Uma destas aplicações foi um jogo, que foi demonstrado publicamente durante o dia aberto da FCT/UNL, tendo sido experimentado e avaliado por um grande número de utilizadores
- …