109 research outputs found
What is lost in translation from visual graphics to text for accessibility
Many blind and low-vision individuals are unable to access
digital media visually. Currently, the solution to this
accessibility problem is to produce text descriptions of visual
graphics, which are then translated via text-to-speech screen
reader technology. However, if a text description can
accurately convey the meaning intended by an author of a
visualization, then why did the author create the visualization
in the first place? This essay critically examines this problem
by comparing the so-called graphic–linguistic distinction to
similar distinctions between the properties of sound and
speech. It also presents a provisional model for identifying
visual properties of graphics that are not conveyed via text-tospeech
translations, with the goal of informing the design of
more effective sonic translations of visual graphics
Research statement
My research lies in the domain of visual media, and graphic representation in particular. I seek a scientifically-grounded understanding of established and emerging conventions in the fine & applied visual arts. This is a perceptual-cognitive approach to understanding visual media
A cognitive exploration of the “non-visual” nature of geometric proofs
Why are Geometric Proofs (Usually) “Non-Visual”? We asked this question as
a way to explore the similarities and differences between diagrams and text (visual
thinking versus language thinking). Traditional text-based proofs are considered
(by many to be) more rigorous than diagrams alone. In this paper we focus on
human perceptual-cognitive characteristics that may encourage textual modes for
proofs because of the ergonomic affordances of text relative to diagrams. We suggest
that visual-spatial perception of physical objects, where an object is perceived
with greater acuity through foveal vision rather than peripheral vision, is similar
to attention navigating a conceptual visual-spatial structure. We suggest that attention
has foveal-like and peripheral-like characteristics and that textual modes
appeal to what we refer to here as foveal-focal attention, an extension of prior
work in focused attention
Iconic Properties are Lost when Translating Visual Graphics to Text for Accessibility
For many blind and low-vision individuals, accessing charts and graphs often means
accessing a text description of the graphics, usually aurally. However, in doing so, parts
of the charts that are not originally conveyed textually are lost in the translation into text.
By synthesizing ideas from the science and philosophy of perception and cognition,
diagrammatic reasoning, and semiotics, this essay makes the case that translating charts
into text descriptions results in the loss of iconic properties of the graphics, and proposes
that non-linguistic sonification can be recruited to preserve such properties. The essay
concludes by proposing how predictions based on this synthesis can inform design
The audio game laboratory: Building maps from games
Audio games demonstrate an emergence of interactive
parameter mapping sonifications that potentially optimally
display geographical information and a large number of
simultaneous data variables. Our preliminary investigation of
audio games is in response to a call for more research on
parameter mapping sonifications, such as the best way of
presenting auditory legends for representations, effectiveness
of spatial audio, map comprehension techniques, and finding
optimal sonic variable mappings. We also present a proposed
set of auditory map interfaces observed in audio games.
Commercially available interactive interfaces and audio
games – that have been shaped and informally “tested” by
the selection pressures of a demanding consumer market –
can serve as examples of potentially effective conventions
informing future work in the auditory display research
community
Mobility in Europe: Analysis of the 2005 Eurobarometer Survey on Geographical and Labour Market Mobility
The European Commission has designated the year 2006 as 'European Year of Workers' Mobility'. The purpose of the initiative is to inform EU citizens of the benefits and the costs of both geographical mobility and job or labour market mobility; the realities of working in another country or changing job or career; and the rights they are entitled to as migrant workers. The initiative also aims to promote the exchange of good practice between public authorities and institutions, the social partners and the private sector, and to promote greater study of the scale and nature of geographical and job mobility within the Union
Progress toward sonifying Napoleon’s march and fluid flow simulations through binaural horizons
Cross-modal data analytics—that can be rendered for experience
through vision, hearing, and touch—poses a fundamental
challenge to designers. Non-linguistic sonification is a
well-researched means for non-visual pattern recognition but
higher density datasets pose a challenge. Because human hearing
is optimized for detecting locations on a horizontal plane, our
approach recruits this optimization by employing an immersive
binaural horizontal plane using auditory icons. Two case studies
demonstrate our approach: A sonic translation of a map and a
sonic translation of a computational fluid dynamics simulation
- …