45 research outputs found
Concurrent Speech Synthesis to Improve Document First Glance for the Blind
International audienceSkimming and scanning are two well-known reading processes, which are combined to access the document content as quickly and efficiently as possible. While both are available in visual reading mode, it is rather difficult to use them in non visual environments because they mainly rely on typographical and layout properties. In this article, we introduce the concept of tag thunder as a way (1) to achieve the oral transposition of the web 2.0 concept of tag cloud and (2) to produce an innovative interactive stimulus to observe the emergence of self-adapted strategies for non-visual skimming of written texts. We first present our general and theoretical approach to the problem of both fast, global and non-visual access to web browsing; then we detail the progress of development and evaluation of the various components that make up our software architecture. We start from the hypothesis that the semantics of the visual architecture of web pages can be transposed into new sensory modalities thanks to three main steps (web page segmentation, keywords extraction and sound spatialization). We note the difficulty of simultaneously (1) evaluating a modular system as a whole at the end of the processing chain and (2) identifying at the level of each software brick the exact origin of its limits; despite this issue, the results of the first evaluation campaign seem promising
Sensory feedback for supernumerary limbs
Supernumerary robotic limbs (SL) are devices developed to increase human capability. For this to happen, SLs should assist users with minimal cognitive effort and be controlled both independently and in combination with the userâs natural limbs. Despite the development of many SL, their application is limited by a lack of easy-to-use and intuitive controllers. Integrating somatosensory feedback in the control loop could improve this issue. In particular, providing artificial proprioception, a somatosensory modality important for motor control, could help SL users to integrate the SL into their planning and body schema.
My objective is to find strategies to deliver intuitive and understandable proprioceptive feedback from a SL. To do so, I investigated possible ways of providing artificial proprioceptive feedback using 2degrees of freedom (DoFs) tactile cues delivered through electrical stimulation and vibration. I designed a set of mappings that provide position cues from a virtual arm for each feedback modality. Two studies, one for each modality, were conducted.
In the first study, I observed that the intensity of electrical stimulation affected its perception and comfort. I then found that it was easier to differentiate frequency variations than intensity variations. This study brings new insights into electrical stimulation perception and mapping design, considering comfort is rarely addressed in previous studies.
In the second study, I compared two mappings, task space and joint space, for the feedback and control of a virtual arm. Although I did not observe any effect on the performance, I found that the task space mapping was preferred and better understood than the joint space mapping.
Furthermore, a novel vibration feedback device was designed and tested to deliver 3DoFs position cues. The study showed that it is possible to transmit feedback at the torso and back, a location that few studies have considered. Moreover, this study proposes a novel strategy to provide 3DoFs feedback using vibration alone.Open Acces
Head-mounted Sensory Augmentation System for Navigation in Low Visibility Environments
Sensory augmentation can be used to assist in some tasks where sensory information is limited or sparse. This thesis focuses on the design and investigation of a head-mounted vibrotactile sensory augmentation interface to assist navigation in low visibility environments such as firefightersâ navigation or travel aids for visually impaired people.
A novel head-mounted vibrotactile interface comprising a 1-by-7 vibrotactile display worn on the forehead is developed. A series of psychophysical studies is carried out with this display to (1) determine the vibrotactile absolute threshold, (2) investigate the accuracy of vibrotactile localization, and (3) evaluate the funneling illusion and apparent motion as sensory phenomena that could be used to communicate navigation signals. The results of these studies provide guidelines for the design of head-mounted interfaces.
A 2nd generation head-mounted sensory augmentation interface called the Mark-II
Tactile Helmet is developed for the application of firefightersâ navigation. It consists of a ring of ultrasound sensors mounted to the outside of a helmet, a microcontroller, two batteries and a refined vibrotactile display composed of seven vibration motors based on the results of the aforementioned psychophysical studies.
A âtactile languageâ, that is, a set of distinguishable vibrotactile patterns, is developed for communicating navigation commands to the Mark-II Tactile Helmet. Four possible combinations of two command presentation modes (continuous, discrete) and two command types (recurring, single) are evaluated for their effectiveness in guiding users along a virtual wall in a structured environment. Continuous and discrete presentation modes use spatiotemporal patterns that induce the experience of apparent movement and discrete movement on the forehead, respectively. The recurring command type presents the tactile command repeatedly with an interval between patterns of 500 ms while the single command type presents the tactile command just once when there is a change in the command. The effectiveness of this tactile language is evaluated according to the objective measures of the usersâ walking speed and the smoothness of their trajectory parallel to the virtual wall and subjective measures of utility and
comfort employing Likert-type rating scales. The Recurring Continuous (RC) commands that exploit the phenomena of apparent motion are most effective in generating efficient routes and fast travel, and are most preferred.
Finally, the optimal tactile language (RC) is compared with audio guidance using
verbal instructions to investigate effectiveness in delivering navigation commands. The results show that haptic guidance leads to better performance as well as lower cognitive workload compared to auditory feedback. This research demonstrates that a head-mounted sensory augmentation interface can enhance spatial awareness in low visibility environments and could help firefightersâ navigation by providing them with supplementary sensory information
Designing usable mobile interfaces for spatial data
2010 - 2011This
dissertation
deals
mainly
with
the
discipline
of
Human-ÂâComputer
Interaction
(HCI),
with
particular
attention
on
the
role
that
it
plays
in
the
domain
of
modern
mobile
devices.
Mobile
devices
today
offer
a
crucial
support
to
a
plethora
of
daily
activities
for
nearly
everyone.
Ranging
from
checking
business
mails
while
traveling,
to
accessing
social
networks
while
in
a
mall,
to
carrying
out
business
transactions
while
out
of
office,
to
using
all
kinds
of
online
public
services,
mobile
devices
play
the
important
role
to
connect
people
while
physically
apart.
Modern
mobile
interfaces
are
therefore
expected
to
improve
the
user's
interaction
experience
with
the
surrounding
environment
and
offer
different
adaptive
views
of
the
real
world.
The
goal
of
this
thesis
is
to
enhance
the
usability
of
mobile
interfaces
for
spatial
data.
Spatial
data
are
particular
data
in
which
the
spatial
component
plays
an
important
role
in
clarifying
the
meaning
of
the
data
themselves.
Nowadays,
this
kind
of
data
is
totally
widespread
in
mobile
applications.
Spatial
data
are
present
in
games,
map
applications,
mobile
community
applications
and
office
automations.
In
order
to
enhance
the
usability
of
spatial
data
interfaces,
my
research
investigates
on
two
major
issues:
1. Enhancing
the
visualization
of
spatial
data
on
small
screens
2. Enhancing
the
text-Ââinput
methods
I
selected
the
Design Science Research approach
to
investigate
the
above
research
questions.
The
idea
underling
this
approach
is
âyou
build artifact to learn from itâ, in
other
words
researchers
clarify
what
is
new
in
their
design.
The
new
knowledge
carried
out
from
the
artifact
will
be
presented
in
form
of
interaction
design
patterns
in
order
to
support
developers
in
dealing
with
issues
of
mobile
interfaces.
The
thesis
is
organized
as
follows.
Initially
I
present
the
broader
context,
the
research
questions
and
the
approaches
I
used
to
investigate
them.
Then
the
results
are
split
into
two
main
parts.
In
the
first
part
I
present
the
visualization
technique
called
Framy.
The
technique
is
designed
to
support
users
in
visualizing
geographical
data
on
mobile
map
applications.
I
also
introduce
a
multimodal
extension
of
Framy
obtained
by
adding
sounds
and
vibrations.
After
that
I
present
the
process
that
turned
the
multimodal
interface
into
a
means
to
allow
visually
impaired
users
to
interact
with
Framy.
Some
projects
involving
the
design
principles
of
Framy
are
shown
in
order
to
demonstrate
the
adaptability
of
the
technique
in
different
contexts.
The
second
part
concerns
the
issue
related
to
text-Ââinput
methods.
In
particular
I
focus
on
the
work
done
in
the
area
of
virtual
keyboards
for
mobile
devices.
A
new
kind
of
virtual
keyboard
called
TaS
provides
users
with
an
input
system
more
efficient
and
effective
than
the
traditional
QWERTY
keyboard.
Finally,
in
the
last
chapter,
the
knowledge
acquired
is
formalized
in
form
of
interaction
design
patterns. [edited by author]X n.s
Investigating perceptual congruence between information and sensory parameters in auditory and vibrotactile displays
A fundamental interaction between a computer and its user(s) is the transmission of information between the two and there are many situations where it is necessary for this interaction to occur non-visually, such as using sound or vibration. To design successful interactions in these modalities, it is necessary to understand how users perceive mappings between information and acoustic or vibration parameters, so that these parameters can be designed such that they are perceived as congruent. This thesis investigates several data-sound and data-vibration mappings by using psychophysical scaling to understand how users perceive the mappings. It also investigates the impact that using these methods during design has when they are integrated into an auditory or vibrotactile display.
To investigate acoustic parameters that may provide more perceptually congruent data-sound mappings, Experiments 1 and 2 explored several psychoacoustic parameters for use in a mapping. These studies found that applying amplitude modulation â or roughness â to a signal, or applying broadband noise to it resulted in performance which were similar to conducting the task visually.
Experiments 3 and 4 used scaling methods to map how a user perceived a change in an information parameter, for a given change in an acoustic or vibrotactile parameter. Experiment 3 showed that increases in acoustic parameters that are generally considered undesirable in music were perceived as congruent with information parameters with negative valence such as stress or danger. Experiment 4 found that data-vibration mappings were more generalised â a given increase in a vibrotactile parameter was almost always perceived as an increase in an information parameter â regardless of the valence of the information parameter.
Experiments 5 and 6 investigated the impact that using results from the scaling methods used in Experiments 3 and 4 had on users' performance when using an auditory or vibrotactile display. These experiments also explored the impact that the complexity of the context which the display was placed had on user performance. These studies found that using mappings based on scaling results did not significantly impact user's performance with a simple auditory display, but it did reduce response times in a more complex use-case
Blending the Material and Digital World for Hybrid Interfaces
The development of digital technologies in the 21st century is progressing continuously and new device classes such as tablets, smartphones or smartwatches are finding their way into our everyday lives. However, this development also poses problems, as these prevailing touch and gestural interfaces often lack tangibility, take little account of haptic qualities and therefore require full attention from their users. Compared to traditional tools and analog interfaces, the human skills to experience and manipulate material in its natural environment and context remain unexploited. To combine the best of both, a key question is how it is possible to blend the material world and digital world to design and realize novel hybrid interfaces in a meaningful way. Research on Tangible User Interfaces (TUIs) investigates the coupling between physical objects and virtual data. In contrast, hybrid interfaces, which specifically aim to digitally enrich analog artifacts of everyday work, have not yet been sufficiently researched and systematically discussed.
Therefore, this doctoral thesis rethinks how user interfaces can provide useful digital functionality while maintaining their physical properties and familiar patterns of use in the real world. However, the development of such hybrid interfaces raises overarching research questions about the design: Which kind of physical interfaces are worth exploring? What type of digital enhancement will improve existing interfaces? How can hybrid interfaces retain their physical properties while enabling new digital functions? What are suitable methods to explore different design? And how to support technology-enthusiast users in prototyping?
For a systematic investigation, the thesis builds on a design-oriented, exploratory and iterative development process using digital fabrication methods and novel materials. As a main contribution, four specific research projects are presented that apply and discuss different visual and interactive augmentation principles along real-world applications. The applications range from digitally-enhanced paper, interactive cords over visual watch strap extensions to novel prototyping tools for smart garments. While almost all of them integrate visual feedback and haptic input, none of them are built on rigid, rectangular pixel screens or use standard input modalities, as they all aim to reveal new design approaches. The dissertation shows how valuable it can be to rethink familiar, analog applications while thoughtfully extending them digitally. Finally, this thesisâ extensive work of engineering versatile research platforms is accompanied by overarching conceptual work, user evaluations and technical experiments, as well as literature reviews.Die Durchdringung digitaler Technologien im 21. Jahrhundert schreitet stetig voran und neue GerĂ€teklassen wie Tablets, Smartphones oder Smartwatches erobern unseren Alltag. Diese Entwicklung birgt aber auch Probleme, denn die vorherrschenden berĂŒhrungsempfindlichen OberflĂ€chen berĂŒcksichtigen kaum haptische QualitĂ€ten und erfordern daher die volle Aufmerksamkeit ihrer Nutzer:innen. Im Vergleich zu traditionellen Werkzeugen und analogen Schnittstellen bleiben die menschlichen FĂ€higkeiten ungenutzt, die Umwelt mit allen Sinnen zu begreifen und wahrzunehmen. Um das Beste aus beiden Welten zu vereinen, stellt sich daher die Frage, wie neuartige hybride Schnittstellen sinnvoll gestaltet und realisiert werden können, um die materielle und die digitale Welt zu verschmelzen. In der Forschung zu Tangible User Interfaces (TUIs) wird die Verbindung zwischen physischen Objekten und virtuellen Daten untersucht. Noch nicht ausreichend erforscht wurden hingegen hybride Schnittstellen, die speziell darauf abzielen, physische GegenstĂ€nde des Alltags digital zu erweitern und anhand geeigneter Designparameter und EntwurfsrĂ€ume systematisch zu untersuchen.
In dieser Dissertation wird daher untersucht, wie MaterialitĂ€t und DigitalitĂ€t nahtlos ineinander ĂŒbergehen können. Es soll erforscht werden, wie kĂŒnftige Benutzungsschnittstellen nĂŒtzliche digitale Funktionen bereitstellen können, ohne ihre physischen Eigenschaften und vertrauten Nutzungsmuster in der realen Welt zu verlieren. Die Entwicklung solcher hybriden AnsĂ€tze wirft jedoch ĂŒbergreifende Forschungsfragen zum Design auf: Welche Arten von physischen Schnittstellen sind es wert, betrachtet zu werden? Welche Art von digitaler Erweiterung verbessert das Bestehende? Wie können hybride Konzepte ihre physischen Eigenschaften beibehalten und gleichzeitig neue digitale Funktionen ermöglichen? Was sind geeignete Methoden, um verschiedene Designs zu erforschen? Wie kann man Technologiebegeisterte bei der Erstellung von Prototypen unterstĂŒtzen?
FĂŒr eine systematische Untersuchung stĂŒtzt sich die Arbeit auf einen designorientierten, explorativen und iterativen Entwicklungsprozess unter Verwendung digitaler Fabrikationsmethoden und neuartiger Materialien. Im Hauptteil werden vier Forschungsprojekte vorgestellt, die verschiedene visuelle und interaktive Prinzipien entlang realer Anwendungen diskutieren. Die Szenarien reichen von digital angereichertem Papier, interaktiven Kordeln ĂŒber visuelle Erweiterungen von UhrarmbĂ€ndern bis hin zu neuartigen Prototyping-Tools fĂŒr intelligente KleidungsstĂŒcke. Um neue DesignansĂ€tze aufzuzeigen, integrieren nahezu alle visuelles Feedback und haptische Eingaben, um Alternativen zu Standard-EingabemodalitĂ€ten auf starren Pixelbildschirmen zu schaffen. Die Dissertation hat gezeigt, wie wertvoll es sein kann, bekannte, analoge Anwendungen zu ĂŒberdenken und sie dabei gleichzeitig mit Bedacht digital zu erweitern. Dabei umfasst die vorliegende Arbeit sowohl realisierte technische Forschungsplattformen als auch ĂŒbergreifende konzeptionelle Arbeiten, Nutzerstudien und technische Experimente sowie die Analyse existierender Forschungsarbeiten
Haptics: Science, Technology, Applications
This open access book constitutes the proceedings of the 13th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, EuroHaptics 2022, held in Hamburg, Germany, in May 2022. The 36 regular papers included in this book were carefully reviewed and selected from 129 submissions. They were organized in topical sections as follows: haptic science; haptic technology; and haptic applications
Somatic ABC's: A Theoretical Framework for Designing, Developing and Evaluating the Building Blocks of Touch-Based Information Delivery
abstract: Situations of sensory overload are steadily becoming more frequent as the ubiquity of technology approaches reality--particularly with the advent of socio-communicative smartphone applications, and pervasive, high speed wireless networks. Although the ease of accessing information has improved our communication effectiveness and efficiency, our visual and auditory modalities--those modalities that today's computerized devices and displays largely engage--have become overloaded, creating possibilities for distractions, delays and high cognitive load; which in turn can lead to a loss of situational awareness, increasing chances for life threatening situations such as texting while driving. Surprisingly, alternative modalities for information delivery have seen little exploration. Touch, in particular, is a promising candidate given that it is our largest sensory organ with impressive spatial and temporal acuity. Although some approaches have been proposed for touch-based information delivery, they are not without limitations including high learning curves, limited applicability and/or limited expression. This is largely due to the lack of a versatile, comprehensive design theory--specifically, a theory that addresses the design of touch-based building blocks for expandable, efficient, rich and robust touch languages that are easy to learn and use. Moreover, beyond design, there is a lack of implementation and evaluation theories for such languages. To overcome these limitations, a unified, theoretical framework, inspired by natural, spoken language, is proposed called Somatic ABC's for Articulating (designing), Building (developing) and Confirming (evaluating) touch-based languages. To evaluate the usefulness of Somatic ABC's, its design, implementation and evaluation theories were applied to create communication languages for two very unique application areas: audio described movies and motor learning. These applications were chosen as they presented opportunities for complementing communication by offloading information, typically conveyed visually and/or aurally, to the skin. For both studies, it was found that Somatic ABC's aided the design, development and evaluation of rich somatic languages with distinct and natural communication units.Dissertation/ThesisPh.D. Computer Science 201
Design and Effect of Continuous Wearable Tactile Displays
Our sense of touch is one of our core senses and while not as information rich as sight and hearing, it tethers us to reality.
Our skin is the largest sensory organ in our body and we rely on it so much that we don\u27t think about it most of the time.
Tactile displays - with the exception of actuators for notifications on smartphones and smartwatches - are currently understudied and underused.
Currently tactile cues are mostly used in smartphones and smartwatches to notify the user of an incoming call or text message.
Specifically continuous displays - displays that do not just send one notification but stay active for an extended period of time and continuously communicate information - are rarely studied.
This thesis aims at exploring the utilization of our vibration perception to create continuous tactile displays.
Transmitting a continuous stream of tactile information to a user in a wearable format can help elevate tactile displays from being mostly used for notifications to becoming more like additional senses enabling us to perceive our environment in new ways.
This work provides a serious step forward in design, effect and use of continuous tactile displays and their use in human-computer interaction.
The main contributions include:
Exploration of Continuous Wearable Tactile Interfaces
This thesis explores continuous tactile displays in different contexts and with different types of tactile information systems. The use-cases were explored in various domains for tactile displays - Sports, Gaming and Business applications. The different types of continuous tactile displays feature one- or multidimensional tactile patterns, temporal patterns and discrete tactile patterns.
Automatic Generation of Personalized Vibration Patterns
In this thesis a novel approach of designing vibrotactile patterns without expert knowledge by leveraging evolutionary algorithms to create personalized vibration patterns - is described. This thesis presents the design of an evolutionary algorithm with a human centered design generating abstract vibration patterns. The evolutionary algorithm was tested in a user study which offered evidence that interactive generation of abstract vibration patterns is possible and generates diverse sets of vibration patterns that can be recognized with high accuracy.
Passive Haptic Learning for Vibration Patterns
Previous studies in passive haptic learning have shown surprisingly strong results for learning Morse Code. If these findings could be confirmed and generalized, it would mean that learning a new tactile alphabet could be made easier and learned in passing. Therefore this claim was investigated in this thesis and needed to be corrected and contextualized. A user study was conducted to study the effects of the interaction design and distraction tasks on the capability to learn stimulus-stimulus-associations with Passive Haptic Learning. This thesis presents evidence that Passive Haptic Learning of vibration patterns induces only a marginal learning effect and is not a feasible and efficient way to learn vibration patterns that include more than two vibrations.
Influence of Reference Frames for Spatial Tactile Stimuli
Designing wearable tactile stimuli that contain spatial information can be a challenge due to the natural body movement of the wearer. An important consideration therefore is what reference frame to use for spatial cues. This thesis investigated allocentric versus egocentric reference frames on the wrist and compared them for induced cognitive load, reaction time and accuracy in a user study. This thesis presents evidence that using an allocentric reference frame drastically lowers cognitive load and slightly lowers reaction time while keeping the same accuracy as an egocentric reference frame, making a strong case for the utilization of allocentric reference frames in tactile bracelets with several tactile actuators
Computer-supported movement guidance: investigating visual/visuotactile guidance and informing the design of vibrotactile body-worn interfaces
This dissertation explores the use of interactive systems to support
movement guidance, with applications in various fields such as sports,
dance, physiotherapy, and immersive sketching. The research focuses
on visual, haptic, and visuohaptic approaches and aims to overcome
the limitations of traditional guidance methods, such as dependence
on an expert and high costs for the novice. The main contributions of
the thesis are (1) an evaluation of the suitability of various types of
displays and visualizations of the human body for posture guidance,
(2) an investigation into the influence of different viewpoints/perspectives,
the addition of haptic feedback, and various movement
properties on movement guidance in virtual environments, (3) an
investigation into the effectiveness of visuotactile guidance for hand
movements in a virtual environment, (4) two in-depth studies of haptic
perception on the body to inform the design of wearable and handheld
interfaces that leverage tactile output technologies, and (5) an
investigation into new interaction techniques for tactile guidance of
arm movements. The results of this research advance the state of the
art in the field, provide design and implementation insights, and pave
the way for new investigations in computer-supported movement
guidance