1,432 research outputs found
Evaluation of Drop Shadows for Virtual Object Grasping in Augmented Reality
This paper presents the use of rendered visual cues as drop shadows and their impact on overall usability and accuracy of grasping interactions for monitor-based exocentric Augmented Reality (AR). We report on two conditions; grasping with drop shadows and without drop shadows and analyse a total of 1620 grasps of two virtual object types (cubes and spheres). We report on the accuracy of one grasp type, the Medium Wrap grasp, against Grasp Aperture (GAp), Grasp Displacement (GDisp), completion time and usability metrics from 30 participants. A comprehensive statistical analysis of the results is presented giving comparisons of the inclusion of drop shadows in AR grasping. Findings showed that the use of drop shadows increases usability of AR grasping while significantly decreasing task completion times. Furthermore, drop shadows also significantly improve user’s depth estimation of AR object position. However, this study also shows that using drop shadows does not improve user’s object size estimation, which remains as
a problematic element in grasping AR interaction literature
Natural freehand grasping of virtual objects for augmented reality
Grasping is a primary form of interaction with the surrounding world, and is an intuitive interaction technique by nature due to the highly complex structure of the human hand. Translating this versatile interaction technique to Augmented Reality (AR) can provide interaction designers with more opportunities to implement more intuitive and realistic AR applications. The work presented in this thesis uses quantifiable measures to evaluate the accuracy and usability of natural grasping of virtual objects in AR environments, and presents methods for improving this natural form of interaction.
Following a review of physical grasping parameters and current methods of mediating grasping interactions in AR, a comprehensive analysis of natural freehand grasping of virtual objects in AR is presented to assess the accuracy, usability and transferability of this natural form of grasping to AR environments. The analysis is presented in four independent user studies (120 participants, 30 participants for each study and 5760 grasping tasks in total), where natural freehand grasping performance is assessed for a range of virtual object sizes, positions and types in terms of accuracy of grasping, task completion time and overall system usability.
Findings from the first user study in this work highlighted two key problems for natural grasping in AR; namely inaccurate depth estimation and inaccurate size estimation of virtual objects.
Following the quantification of these errors, three different methods for mitigating user errors and assisting users during natural grasping were presented and analysed; namely dual view visual feedback, drop shadows and additional visual feedback when adding user based tolerances during interaction tasks. Dual view visual feedback was found to significantly improve user depth estimation, however this method also significantly increased task completion time.
Drop shadows provided an alternative, and a more usable solution, to dual view visual feedback through significantly improving depth estimation, task completion time and the overall usability of natural grasping. User based tolerances negated the fundamental problem of inaccurate size estimation of virtual objects, through enabling users to perform natural grasping without the need of being highly accurate in their grasping performance, thus providing evidence that natural grasping can be usable in task based AR environments. Finally recommendations for allowing and further improving natural grasping interaction in AR environments are provided, along with guidelines for translating this form of natural grasping to other AR environments and user interfaces
Expanding tangible tabletop interfaces beyond the display
L’augment
de
popularitat
de
les
taules
i
superfícies
interactives
està
impulsant
la
recerca
i
la
innovació
en
una
gran
varietat
d’àrees,
incloent-‐hi
maquinari,
programari,
disseny
de
la
interacció
i
noves
tècniques
d’interacció.
Totes,
amb
l’objectiu
de
promoure
noves
interfícies
dotades
d’un
llenguatge
més
ric,
potent
i
natural.
Entre
totes
aquestes
modalitats,
la
interacció
combinada
a
sobre
i
per
damunt
de
la
superfície
de
la
taula
mitjançant
tangibles
i
gestos
és
actualment
una
àrea
molt
prometedora.
Aquest
document
tracta
d’expandir
les
taules
interactives
més
enllà
de
la
superfície
per
mitjà
de
l’exploració
i
el
desenvolupament
d’un
sistema
o
dispositiu
enfocat
des
de
tres
vessants
diferents:
maquinari,
programari
i
disseny
de
la
interacció.
Durant
l’inici
d’aquest
document
s’estudien
i
es
resumeixen
els
diferents
trets
característics
de
les
superfícies
interactives
tangibles
convencionals
o
2D
i
es
presenten
els
treballs
previs
desenvolupats
per
l’autor
en
solucions
de
programari
que
acaben
resultant
en
aplicacions
que
suggereixen
l’ús
de
la
tercera
dimensió
a
les
superfícies
tangibles.
Seguidament,
es
presenta
un
repàs
del
maquinari
existent
en
aquest
tipus
d’interfícies
per
tal
de
concebre
un
dispositiu
capaç
de
detectar
gestos
i
generar
visuals
per
sobre
de
la
superfície,
per
introduir
els
canvis
realitzats
a
un
dispositiu
existent,
desenvolupat
i
cedit
per
Microsoft
Reseach
Cambridge.
Per
tal
d’explotar
tot
el
potencial
d’aquest
nou
dispositiu,
es
desenvolupa
un
nou
sistema
de
visió
per
ordinador
que
estén
el
seguiment
d’objectes
i
mans
en
una
superfície
2D
a
la
detecció
de
mans,
dits
i
etiquetes
amb
sis
graus
de
llibertat
per
sobre
la
superfície
incloent-‐hi
la
interacció
tangible
i
tàctil
convencional
a
la
superfície.
Finalment,
es
presenta
una
eina
de
programari
per
a
generar
aplicacions
per
al
nou
sistema
i
es
presenten
un
seguit
d’aplicacions
per
tal
de
provar
tot
el
desenvolupament
generat
al
llarg
de
la
tesi
que
es
conclou
presentant
un
seguit
de
gestos
tant
a
la
superfície
com
per
sobre
d’aquesta
i
situant-‐los
en
una
nova
classificació
que
alhora
recull
la
interacció
convencional
2D
i
la
interacció
estesa
per
damunt
de
la
superfície
desenvolupada.The
rising
popularity
of
interactive
tabletops
and
surfaces
is
spawning
research
and
innovation
in
a
wide
variety
of
areas,
including
hardware
and
software
technologies,
interaction
design
and
novel
interaction
techniques,
all
of
which
seek
to
promote
richer,
more
powerful
and
more
natural
interaction
modalities.
Among
these
modalities,
combined
interaction
on
and
above
the
surface,
both
with
gestures
and
with
tangible
objects,
is
a
very
promising
area.
This
dissertation
is
about
expanding
tangible
and
tabletops
surfaces
beyond
the
display
by
exploring
and
developing
a
system
from
the
three
different
perspectives:
hardware,
software,
and
interaction
design.
This
dissertation,
studies
and
summarizes
the
distinctive
affordances
of
conventional
2D
tabletop
devices,
with
a
vast
literature
review
and
some
additional
use
cases
developed
by
the
author
for
supporting
these
findings,
and
subsequently
explores
the
novel
and
not
yet
unveiled
potential
affordances
of
3D-‐augmented
tabletops.
It
overviews
the
existing
hardware
solutions
for
conceiving
such
a
device,
and
applies
the
needed
hardware
modifications
to
an
existing
prototype
developed
and
rendered
to
us
by
Microsoft
Research
Cambridge.
For
accomplishing
the
interaction
purposes,
it
is
developed
a
vision
system
for
3D
interaction
that
extends
conventional
2D
tabletop
tracking
for
the
tracking
of
hand
gestures,
6DoF
markers
and
on-‐surface
finger
interaction.
It
finishes
by
conceiving
a
complete
software
framework
solution,
for
the
development
and
implementation
of
such
type
of
applications
that
can
benefit
from
these
novel
3D
interaction
techniques,
and
implements
and
test
several
software
prototypes
as
proof
of
concepts,
using
this
framework.
With
these
findings,
it
concludes
presenting
continuous
tangible
interaction
gestures
and
proposing
a
novel
classification
for
3D
tangible
and
tabletop
gestures
Visual Perception and Cognition in Image-Guided Intervention
Surgical image visualization and interaction systems can dramatically affect the efficacy and efficiency of surgical training, planning, and interventions. This is even more profound in the case of minimally-invasive surgery where restricted access to the operative field in conjunction with limited field of view necessitate a visualization medium to provide patient-specific information at any given moment. Unfortunately, little research has been devoted to studying human factors associated with medical image displays and the need for a robust, intuitive visualization and interaction interfaces has remained largely unfulfilled to this day. Failure to engineer efficient medical solutions and design intuitive visualization interfaces is argued to be one of the major barriers to the meaningful transfer of innovative technology to the operating room. This thesis was, therefore, motivated by the need to study various cognitive and perceptual aspects of human factors in surgical image visualization systems, to increase the efficiency and effectiveness of medical interfaces, and ultimately to improve patient outcomes. To this end, we chose four different minimally-invasive interventions in the realm of surgical training, planning, training for planning, and navigation: The first chapter involves the use of stereoendoscopes to reduce morbidity in endoscopic third ventriculostomy. The results of this study suggest that, compared with conventional endoscopes, the detection of the basilar artery on the surface of the third ventricle can be facilitated with the use of stereoendoscopes, increasing the safety of targeting in third ventriculostomy procedures. In the second chapter, a contour enhancement technique is described to improve preoperative planning of arteriovenous malformation interventions. The proposed method, particularly when combined with stereopsis, is shown to increase the speed and accuracy of understanding the spatial relationship between vascular structures. In the third chapter, an augmented-reality system is proposed to facilitate the training of planning brain tumour resection. The results of our user study indicate that the proposed system improves subjects\u27 performance, particularly novices\u27, in formulating the optimal point of entry and surgical path independent of the sensorimotor tasks performed. In the last chapter, the role of fully-immersive simulation environments on the surgeons\u27 non-technical skills to perform vertebroplasty procedure is investigated. Our results suggest that while training surgeons may increase their technical skills, the introduction of crisis scenarios significantly disturbs the performance, emphasizing the need of realistic simulation environments as part of training curriculum
Bringing the Physical to the Digital
This dissertation describes an exploration of digital tabletop interaction styles, with the ultimate goal of informing the design of a new model for tabletop interaction. In the context of this thesis the term digital tabletop refers to an emerging class of devices that afford many novel ways of interaction with the digital. Allowing users to directly touch information presented on large,
horizontal displays. Being a relatively young field, many developments are in flux; hardware and software change at a fast pace and many interesting alternative approaches are available at the same time. In our research we are especially interested in systems that are capable of sensing multiple contacts (e.g., fingers) and richer information such as the outline of whole hands or other physical objects. New sensor hardware enable new ways to interact with the digital. When embarking into the research for this thesis, the question which interaction styles could
be appropriate for this new class of devices was a open question, with many equally promising answers.
Many everyday activities rely on our hands ability to skillfully control and manipulate physical objects. We seek to open up different possibilities to exploit our manual dexterity and provide users with richer interaction possibilities. This could be achieved through the use of physical objects as input mediators or through virtual interfaces that behave in a more realistic fashion.
In order to gain a better understanding of the underlying design space we choose an approach organized into two phases. First, two different prototypes, each representing a specific interaction style – namely gesture-based interaction and tangible interaction – have been implemented. The flexibility of use afforded by the interface and the level of physicality afforded by the interface elements are introduced as criteria for evaluation. Each approaches’ suitability to support the
highly dynamic and often unstructured interactions typical for digital tabletops is analyzed based
on these criteria.
In a second stage the learnings from these initial explorations are applied to inform the design of a novel model for digital tabletop interaction. This model is based on the combination of rich multi-touch sensing and a three dimensional environment enriched by a gaming physics simulation. The proposed approach enables users to interact with the virtual through richer quantities such as collision and friction. Enabling a variety of fine-grained interactions using multiple fingers, whole hands and physical objects.
Our model makes digital tabletop interaction even more “natural”. However, because the interaction – the sensed input and the displayed output – is still bound to the surface, there is a fundamental limitation in manipulating objects using the third dimension. To address this issue,
we present a technique that allows users to – conceptually – pick objects off the surface and control their position in 3D. Our goal has been to define a technique that completes our model for on-surface interaction and allows for “as-direct-as possible” interactions. We also present
two hardware prototypes capable of sensing the users’ interactions beyond the table’s surface.
Finally, we present visual feedback mechanisms to give the users the sense that they are actually lifting the objects off the surface.
This thesis contributes on various levels. We present several novel prototypes that we built and evaluated. We use these prototypes to systematically explore the design space of digital tabletop interaction. The flexibility of use afforded by the interaction style is introduced as criterion alongside the user interface elements’ physicality. Each approaches’ suitability to support the
highly dynamic and often unstructured interactions typical for digital tabletops are analyzed. We present a new model for tabletop interaction that increases the fidelity of interaction possible in
such settings. Finally, we extend this model so to enable as direct as possible interactions with
3D data, interacting from above the table’s surface
Distance mis-estimations can be reduced with specific shadow locations
Shadows in physical space are copious, yet the impact of specific shadow placement and their abundance is yet to be determined in virtual environments. This experiment aimed to identify whether a target’s shadow was used as a distance indicator in the presence of binocular distance cues. Six lighting conditions were created and presented in virtual reality for participants to perform a perceptual matching task. The task was repeated in a cluttered and sparse environment, where the number of cast shadows (and their placement) varied. Performance in this task was measured by the directional bias of distance estimates and variability of responses. No significant difference was found between the sparse and cluttered environments, however due to the large amount of variance, one explanation is that some participants utilised the clutter objects as anchors to aid them, while others found them distracting. Under-setting of distances was found in all conditions and environments, as predicted. Having an ambient light source produced the most variable and inaccurate estimates of distance, whereas lighting positioned above the target reduced the mis-estimation of distances perceived
Deep learning for object detection in robotic grasping contexts
Dans la dernière décennie, les approches basées sur les réseaux de neurones convolutionnels sont devenus les standards pour la plupart des tâches en vision numérique. Alors qu'une grande partie des méthodes classiques de vision étaient basées sur des règles et algorithmes, les réseaux de neurones sont optimisés directement à partir de données d'entraînement qui sont étiquetées pour la tâche voulue. En pratique, il peut être difficile d'obtenir une quantité su sante de données d'entraînement ou d'interpréter les prédictions faites par les réseaux. Également, le processus d'entraînement doit être recommencé pour chaque nouvelle tâche ou ensemble d'objets. Au final, bien que très performantes, les solutions basées sur des réseaux de neurones peuvent être difficiles à mettre en place. Dans cette thèse, nous proposons des stratégies visant à contourner ou solutionner en partie ces limitations en contexte de détection d'instances d'objets. Premièrement, nous proposons d'utiliser une approche en cascade consistant à utiliser un réseau de neurone comme pré-filtrage d'une méthode standard de "template matching". Cette façon de faire nous permet d'améliorer les performances de la méthode de "template matching" tout en gardant son interprétabilité. Deuxièmement, nous proposons une autre approche en cascade. Dans ce cas, nous proposons d'utiliser un réseau faiblement supervisé pour générer des images de probabilité afin d'inférer la position de chaque objet. Cela permet de simplifier le processus d'entraînement et diminuer le nombre d'images d'entraînement nécessaires pour obtenir de bonnes performances. Finalement, nous proposons une architecture de réseau de neurones ainsi qu'une procédure d'entraînement permettant de généraliser un détecteur d'objets à des objets qui ne sont pas vus par le réseau lors de l'entraînement. Notre approche supprime donc la nécessité de réentraîner le réseau de neurones pour chaque nouvel objet.In the last decade, deep convolutional neural networks became a standard for computer vision applications. As opposed to classical methods which are based on rules and hand-designed features, neural networks are optimized and learned directly from a set of labeled training data specific for a given task. In practice, both obtaining sufficient labeled training data and interpreting network outputs can be problematic. Additionnally, a neural network has to be retrained for new tasks or new sets of objects. Overall, while they perform really well, deployment of deep neural network approaches can be challenging. In this thesis, we propose strategies aiming at solving or getting around these limitations for object detection. First, we propose a cascade approach in which a neural network is used as a prefilter to a template matching approach, allowing an increased performance while keeping the interpretability of the matching method. Secondly, we propose another cascade approach in which a weakly-supervised network generates object-specific heatmaps that can be used to infer their position in an image. This approach simplifies the training process and decreases the number of required training images to get state-of-the-art performances. Finally, we propose a neural network architecture and a training procedure allowing detection of objects that were not seen during training, thus removing the need to retrain networks for new objects
Interactive tabletops in education
Interactive tabletops are gaining increased attention from CSCL researchers. This paper analyses the relation between this technology and teaching and learning processes. At a global level, one could argue that tabletops convey a socio-constructivist flavor: they support small teams that solve problems by exploring multiple solutions. The development of tabletop applications also witnesses the growing importance of face-to-face collaboration in CSCL and acknowledges the physicality of learning. However, this global analysis is insufficient. To analyze the educational potential of tabletops in education, we present 33 points that should be taken into consideration. These points are structured on four levels: individual user-system interaction, teamwork, classroom orchestration, and socio-cultural contexts. God lies in the detail
- …