Sonic City is a project exploring mobile interaction and wearable technology for everyday music creation. A wearable system has been developed that creates electronic music in real-time based on sensing bodily and environmental factors - thus, a personal soundscape is co-produced by physical movement, local activity, and urban ambiance simply by walking through the city. Applying multi-disciplinary methods, we have developed the wearable from a scenario-driven, aesthetic and lifestyle perspective. A garment has been crafted for 'trying on' interaction and wearabilty options with users on-site in the city. With this prototype, we have been able to expore and rapidly iterate context and content, social and human factors of the wearable application

Jacobs, M.

Mazé, Ramia

English

Sonic City is a project exploring mobile interaction and wearable technology for everyday music creation. A wearable system has been developed that creates electronic music in real-time based on sensing bodily and environmental factors - thus, a personal soundscape is co-produced by physical movement, local activity, and urban ambiance simply by walking through the city. Applying multi-disciplinary methods, we have developed the wearable from a scenario-driven, aesthetic and lifestyle perspective. A garment has been crafted for &apos;trying on&apos; interaction and wearabilty options with users on-site in the city. With this prototype, we have been able to expore and rapidly iterate context and content, social and human factors of the wearable application.IT + TextilesPublic Play SpacesSonic Cit

Digitala Vetenskapliga Arkivet - Academic Archive On-line

Sonic City: Prototyping a wearable experience

Software institutes' Online Digital Archive

Figure 1. Wearing the Prototype
Sonic City: Prototyping a Wearable Experience
Ramia Mazé and Margot Jacobs
PLAY studio, Interactive Institute
Hugo Grauers gata 3, 412 96 Göteborg, Sweden
{ramia.maze, margot.jacobs}@tii.se
Abstract
Sonic City is a project exploring mobile interaction
and wearable technology for everyday music creation. A
wearable system has been developed that creates
electronic music in real-time based on sensing bodily and
environmental factors – thus, a personal soundscape is
co-produced by physical movement, local activity, and
urban ambiance simply by walking through the city.
Applying multi-disciplinary methods, we have
developed the wearable from a scenario-driven, aesthetic
and lifestyle perspective. A garment has been crafted for
‘trying on’ interaction and wearabilty options with users
on-site in the city. With this prototype, we have been able
to explore and rapidly iterate context and content, social
and human factors of the wearable application.
Keywords: design methods, interaction design, wearable
computing, wearability, rapid prototyping, gesture
1. Introduction
With advances in ubiquitous computing and mobile
applications, technology woven seamlessly into everyday
experience and clothing becomes increasingly feasible.
With woven circuits, conductive textiles, and miniature
sensors, we can start to use technology as a computational
material in the same way that we use other, traditional
materials in product and fashion design [9]. Application
domains for wearables continue to expand, from task-
oriented safety-critical applications to leisure-oriented
consumer products. As the material and conceptual
possibilities expand, it becomes necessary for designers to
rethink design processes and prototyping methods to
support the development of innovative new user
experiences with wearable technology.
In Sonic City, we are a project team developing a
wearable system for music creation in our everyday lives
and environments. Drawing on gestural interfaces and
context-aware computing, we propose an interaction
model based on sensor perception of bodily and context
factors. If gesture can generally be defined as a conscious
or unconscious movement from part of the body, then
Sonic City treats mobility through the city as gestural
interaction on a larger scale. Thus, the simple act of
taking a walk becomes a creative composition, a personal
soundscape characterized by our actions, emotions, and
the dynamic urban landscape. As a wearable application,
music creation is integrated seamlessly into natural
behaviors and everyday life.
We have developed a wearable prototype for testing
experiential and technical variables in Sonic City.
Intended as a personal, expressive and everyday
experience, we employ experimental and user-centered
methods to design and investigate perceptual, lifestyle,
physical, and dynamic variables. Design materials and a
wearable garment have been crafted to test and refine
options together with users on-site in the city. Here, we
present our approach to the design process and interaction,
briefly introduce the technical application, and describe
initial tests of the wearable prototype that have led to a
greater understanding of design for wearable experiences.
2. Related Issues and Work
Intended to support personal expression and creativity,
our project is inspired by others that deal with cultural
and expressive factors. Informed by performance genres
and innovative musical interfaces (e.g. [7], [11]), Sonic
City intends to take music creation outside the recording
studio and performance event and into everyday street life.
To this end, Sonic City draws on work in multi-sensor
perception and context-awareness (e.g. [3], [5]).
Taking into account issues of social perception and
aesthetics dealt with in other wearable projects (e.g. [4]),
      
Figures 2 & 3. Sensor input and sound output
our design process brings such issues to bear upon
interaction and technical choices. With mobility as the
essence of interaction in Sonic City, design factors are
heavily concerned with ‘wearability’ (e.g. [6]). From
scenario methods and experience prototyping (e.g. [1],
[2]), we have used design materials to approach wearable
prototyping experientially.
3. Our Approach
In order to better understand, prototype, and design for
experience, our process has been highly collaborative and
multi-disciplinary. The core team has competence in
interaction and product design, engineering, and
architecture, and has involved outside experts in sound
art, sociology, and cognitive science. To include both
experiential and technical perspectives at each stage in the
process, design and system development have occurred in
parallel. From the technical side, questions and
requirements concerned sensing and retrieval, input and
control parameters, and sound processing and mapping
[5]. From the design side, user-centered design methods
have developed these in relation to models for user
interaction, wearability, and lifestyle factors.
We believe that in designing for new use contexts and
future technology applications, innovative methods are
needed to involve users and development teams in
considering emerging human factors.  Specifically, in
Sonic City we had to consider a wide complexity of
factors in the user experience, including heterogeneous
and multiple use contexts, extended use everyday and
over time, and a wide spectrum of user types. Experience
prototyping and scenarios have provided vehicles for
sharing and decision-making internally and with users.
Our approach to prototype development can be
described as ‘design and reflection in action’. Staging
participatory experiments in the city using design
materials, experiential prototypes, and re-configurable
features have enabled concept evolution ‘in-the-field’. For
instance, we have designed and performed initial tests
with the prototype encased in a garment designed for
modular testing of the technical and experiential variables
in the city streets. As a design and research project, Sonic
City is a platform for applying innovative design
methods for developing an everyday wearable experience.
3.1 Defining the interaction and system
At the start of the project, we carried out mini-
ethnographic observations of everyday behavior in the city
in order to determine relevant input parameters and
implications for context perception. Observations
included actions, events, ambiances, and pedestrian’s
paths as a essentially a composition for structuring these
– for example behavioral sequences at crosswalks in terms
of glancing, changing course, and speed. Resulting
descriptions were broken down into two levels – the body
and the environment – and cues that could be used for
action and context recognition. From this, we refined the
system as a series of wearable sensors for generating
music from bodily and environmental activity.
Alongside technical development, we involved users
and scenario methods to explore interaction alternatives,
perceptions, and design qualities of Sonic City. Written
and visual scenarios were developed based on lifestyles,
values, and character of people we interviewed. Together
with the technical prototype, these were the basis for an
in-depth user workshop. This helped us to further define
design criteria, system implications, and sound
possibilities in terms of a user experience perspective. We
decided on a sound mapping balancing foreground/back-
ground qualities and explicit/implicit control to support
pleasing, dynamic, and comprehensible use everyday.
4. Wearable Prototype Development
The Sonic City prototype has been designed to work
both experientially and technically. Components of the
electronic system are housed in a wearable garment in
order ‘try on’ and rapidly iterate interaction, system, and
aesthetic variables. Specifically:
• choice of sensors or combination of sensors
• location of the sensors on the body
• perceived effect of context and action on the music
• options in music processing parameters
• latency and sensitivity in system response
• garment size and shape for movement and mobility
• adjustability and modularity in the wearable design
While this prototype has a pivotal role in informing
future directions of the project, it is by no means intended
as a final form, aesthetic or technology solution. To this
end, certain aspects have been deliberately simplified and
underplayed, while other variables and options elaborated
and highly crafted. Consistent with our approach
described earlier, it should be seen as a design material for
communicating and testing development questions.
4.1 Developing the Electronic System
The technology system in Sonic City [5] has been
developed in order to test and iterate significant variables
in the experience. Fixed-location hardware components
include headphones, a Basic X-24 micro-controller, a
Figure 5. Front view of the jacket showing the
pattern for a Velcro grid, a potential sensor
configuration, & re-sizeable areas
Figure 4. Back view of the jacket showing fixed &
variable placement of electronic components:
(a) laptop, (b) USB-MIDI converter, (c) micro-
controller, (1) microphone and (2)-(4) sensors
USB-MIDI converter, and a small laptop running the
interactive music programming environment PD.
Variable hardware components include a range of
sensors worn on the body, including a light-to-frequency
converter, sound gate (microphone), metal detector,
accelerometer, thermometer, and pollution sensor. Each
extends on a cable from the micro-controller – while not
optimal in terms of aesthetics and tidiness, this supports
‘plug-and-play’-style connectivity. Thus, individual
sensors are easily isolated, combinations quickly tested,
and damaged components simply replaced. Additionally,
cables provide helpful visual cues as to current sensor
configuration when testing multiple options.
As a platform for developing an interaction model and
testing the experience outdoors with users, we have
prioritized robustness, portability, and modularity in
technical decision-making. Ideally, processing would
happen in a small, PDA-type device and a future version
might be implemented with multiple, miniature sensors
wirelessly networked using Smart-Its [10]. For current
purposes of discovering and designing for manifold
experiential variables, this prototype effectively supports
rapid calibration, flexible reconfiguration, and means of
trying out multiple options with users in the city.
4.2 Building the Wearable Form
A wearable housing for the electronics has been built
as a jacket for testing – a sort of Sonic City ‘lab coat’ for
testing in an urban ‘wearability lab’. Rather than a design
solution, the garment is a sort of working paper
prototype. Indeed, even the aesthetic and fabric (an off-
white cotton muslin) were chosen so the jacket could
double as a working document: notations of sensor
configurations are written directly on the jacket for quick
visual reference in user tests and a lasting archive.
Addressing questions of sensor choice, locations, and
combinations, we have crafted a modular scheme for
sensor configuration. Each sensor is encased in a plastic
patch with Velcro backing and a Velcro grid was sewn on
both sides of the garment fabric. Thus, each sensor can be
systematically tested nearly anywhere on the upper body.
This was particularly important in orienting the
accelerometer and microphone to test out variations in the
user experience – for instance, the difference in directing
the mic toward the feet or the breath.
Dynamic physical movement and mobile factors were
of primary factors on designing the form and shape of the
garment. A comfortable fit, weight distribution of the
electronics, and a wide range of upper-body movement
had to be accommodated for users of various shapes and
sizes. For maximum maneuverability and comfort, all
fixed electronic components were fitted into custom
pockets at the back, with Velcro tabs to secure sensor
cables along the seams. Adjustable straps and Velcro
closures were used to alter the fit of the jacket and ensure
a smooth surface so as not to interfere with sensor
placement. The arms of the garment were left semi-open
at the seam and intentionally elongated to aid with sizing
and thermal considerations and it effectively extended
sensor placement down to the fingertips. Finally, the
garment was sewn as independent, modular pieces to
support easy disassembly and reuse.
4.3 Designing the testing situation
 We have performed initial, in-depth user tests with
the wearable prototype in the streets of Göteborg. In order
to thoroughly investigate the multiple variables involved,
we have designed an interview-style test that was held
with two users, in two sites, over two days. Written
documentation consists of worksheets to record the path,
context factors, sensor placement, and user feedback.
Roles among the project testing team were clearly
defined in order to explore as much of the user’s
experience as possible. One role involved technical
calibration and configuration of the hardware and software
options. An interviewer gathered and documented user
     
Figures 6 & 7. Documenting jacket & worksheet
feedback. Two camera people observed and captured the
user’s behavior and the local environment including
reactions of bystanders. The role of analysis took place
afterwards, once the recorded musical soundtrack is finally
united with video footage and user worksheets.
6. Initial Results and Future Implications
Initial results of the user tests have been very positive.
Results are primarily qualititative and embedded in
experiential feedback though revealing clear implications
for the technical side. In terms of the context perception
and dynamic mapping strategy, user feedback suggested
successful maintenance of interest over the duration of a
walk and clear perception of difference between contexts
and curiosity about the variable factors. Musical
interaction was explored even when standing still – while
waiting for a traffic light, one user began improvising
physically by swinging his arms and jumping around,
smiling spontaneously.
There were also implications for future design
directions. There was a good response to flexible sensor
placement – for instance, the choice between locating the
light sensor on a highly dynamic part of the body such as
the hand or a more static location where it might remain
shaded and ‘muted’ for extended periods. These findings
point toward potential end-user configurability in the
future – this could enable users to customize and refresh
the role of Sonic City for their everyday use.
Significant user input will continue to be gathered and
these results probed further in future tests, potentially in
other cities. We are also planning a ‘street-performance’
style test in order to learn about general public
perceptions and social implications. Future wearable
prototype iterations will be developed to reflect emerging
criteria and priorities.
In the near future, test results and the user experience
scenarios will be incorporated into workshops with
fashion and textile designers as vehicles for developing
future visions and exploring industrially-relevant or
commercially-viable aspects of Sonic City. As we
consider increasingly sophisticated iterations, our ongoing
challenge will be to develop and apply relevant design
and evaluation methods for wearable user experiences.
7. Acknowledgements
The core and original project team includes Lalya Gaye
(Future Applications Lab, Viktoria Institute), who among
many things was responsible for and implemented the
technology. We would like to give special thanks to
Daniel Skoglund of 8Tunnel2 for contributing to the
sound design, Nikolas Makelberge, Daniel Rehn, and
Hanna Landin for testing, Magnus Johansson and Sara
Lerén for help with methods. This project is funded by
the Swedish Foundation for Strategic Research through
the Future Applications Lab and the Interactive Institute.
8. References
[1] M. Buchenau and J. Suri, “Experience Prototyping”,
Proc. of DIS , ACM Press, 2000, pp. 424-433.
[2] J. Farringdon, A. Moore, N. Tilbury, J. Church, and
P. Biemond. “Wearable Sensor Badge & Sensor Jacket
for Context Awareness”, Proc. of the 3rd Intl. Symosium.
on Wearable Computers. IEEE, 1999, pp. 107-113.
[3] S.K. Feiner, “The Importance of Being Mobile: Some
Social Consequences of Wearable Augmented Reality
Systems”,  Proc. of the 2nd Intl. Workshop on Augmented
Reality, IEEE and ACM, 1999, pp. 145-148.
[4] L. Gaye, R. Mazé, and L.E. Holmquist, "Sonic City:
The Urban Environment as a Musical Interface",
forthcoming Proc. of NIME, 2003.
[5] H.W. Gellersen, A. Schmidt, and M. Beigl, “Multi-
Sensor Context-Awareness in Mobile Devices and Smart
Artifacts”, Mobile Networks and Applications, Kluwer
Academic Publishers, MA, USA, 2002, pp. 341-351.
[6] F. Gemperle, C. Kasabach, J. Stivoric, M. Bauer, and
R. Martin, “Design for Wearability”, Proc. of the 2nd Intl.
Symp. on Wearable Computers, IEEE, 1998, pp.116-122.
[7] S. Howard et al. “Endowed props in scenario based
design”, Proc.of NordiCHI, 2002, pp. 1-10.
[8] J. Redström, Designing Everyday Computational
Things. PhD. Thesis, Gothenburg Studies in Informatics
No. 20, Göteborg University, Sweden, 2001.
[9] Smart-Its, information at www.smart-its.org
[10] M. Wanderley and M. Battier, eds., Trends in
Gestural Control of Music. IRCAM, Paris, 2000.


Swedish Institute of Computer Science Publications Database

Design for Wearability”,

Designing Everyday Computational Things.

Endowed props in scenario based design”,

Experience Prototyping”,

Sonic City: The Urban Environment as a Musical Interface&quot;, forthcoming

The Importance of Being Mobile: Some Social Consequences of Wearable Augmented Reality Systems”,

Sonic City: Prototyping a wearable experience

Abstract

Similar works

Full text

Available Versions

Digitala Vetenskapliga Arkivet - Academic Archive On-line

Software institutes' Online Digital Archive

Swedish Institute of Computer Science Publications Database

RISE – Research Institutes of Sweden

Crossref