This demonstration showcases the current state of the art for the levitating particle display from the Levitate Project. In this demonstration, we show a new type of display consisting of floating voxels, small levitating particles that can be positioned and moved independently in 3D space. Phased ultrasound arrays are used to acoustically levitate the particles. Users can interact directly with each particle using pointing gestures. This allows users to walk-up and interact without any user instrumentation, creating an exciting opportunity to deploy these tangible displays in public spaces in the future. This demonstration explores the design potential of floating voxels and how these may be used to create new types of user interfaces

Brewster, Stephen

Freeman, Euan

Williamson, Julie

English

Enlighten: Publications

 
 
 
 
 
Williamson, J., Freeman, E.  and Brewster, S. (2017) Levitate: Interaction 
with Floating Particle Displays. In: 6th ACM International Symposium on 
Pervasive Displays, Lugano, Switzerland, 7-9 June 2017, Article 24. ISBN 
9781450350457(doi:10.1145/3078810.3084347) 
 
 
This is the author’s final accepted version. 
 
There may be differences between this version and the published version. 
You are advised to consult the publisher’s version if you wish to cite from 
it. 
 
http://eprints.gla.ac.uk/142897/ 
     
 
 
 
 
 
 
Deposited on: 27 June 2017 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Enlighten – Research publications by members of the University of Glasgow 
http://eprints.gla.ac.uk33640 
Levitate: Interaction with Floating Particle Displays 
Julie R. Williamson, Euan Freeman, and Stephen Brewster 
University of Glasgow 
Glasgow, Scotland 
{FirstName.LastName}@glasgow.ac.uk 
 
ABSTRACT 
This demonstration will showcase the current state of the 
art levitating particle display from the Levitate Project.  In 
this demonstration, we show a new type of display 
consisting of floating voxels, small levitating particles that 
can be positioned and moved independently in 3D space. 
Phased ultrasound arrays are used to acoustically levitate 
the particles and users can interaction with each particle 
individually using pointing gestures.  Users can interact 
with the system in a walk-up-and-use manner without any 
user instrumentation, creating an exciting opportunity to 
deploy these tangible displays in public spaces in the future.  
This demonstration explores the design potential of floating 
voxels and how these may be used to create new types of 
user interfaces. 
Author Keywords 
Tangible Displays, Ultrasonic Levitation, Multimodal 
Displays. 
ACM Classification Keywords 
H.5.2. User Interfaces (Input devices and strategies). 
INTRODUCTION 
Novel display form factors are becoming more increasingly 
prevalent in public displays as technology because more 
robust and new input techniques are developed to enhance 
interaction at public displays.  Levitating particle displays 
are a promising approach to create tangible displays to 
support direct manipulation.  Sound can be used to levitate 
objects in mid-air by trapping them in the low-pressure 
nodes of standing waves.  Recent research has enhanced 
this simple concept with the ability to levitate multiple 
objects in a plane [4], move multiple objects in 3D [5], and 
rotate them as well [2,6]. 
This demonstration explores different techniques for 
interacting with individual pixels on a levitation particle 
display. We consider levitated objects as floating voxels, 
and see their potential as a new type of display. Displays 
created from floating voxels do not require any user 
instrumentation (e.g., head-mounted displays), can be 
viewed from many angles, and users can see through them, 
since the objects are held in mid-air by sound. This means 
they naturally support collaboration and could be used to 
create new multi-user interactions and experiences.  In this 
demonstration, we describe a design space for floating 
voxels and how these can be enhanced by multiple 
modalities.  
 
 
Figure 1. Arrays of ultrasonic transducers can levitate 
particles to create tangible displays.  We are exploring 
different techniques for interacting with these displays.  Note 
that the blue background is only included to increase visibility 
of the levitating particles and is removed during 
demonstration so that users can look through the display. Top:  
Two floating particles create a two pixel display suspended in 
mid-air.  Bottom: Users can select pixels by pointing without 
wearing any additional sensors. 
RELATED WORK 
Tangible displays made from dynamically controlled 
particles have been attempted using a variety of techniques.  
Ishii's radical atoms vision, which describes human-
material interaction through material that can change form 
and appearance dynamically, has inspired many different 
interfaces.  Levitation has been achieved using magnetic 
forces [1] and acoustic fields [3–5].   
One area in need of additional research is how input and 
interaction should be design for levitating displays using 
acoustic fields.  Although acoustic field levitation provides 
a highly flexible and promising approach, direct input can 
disrupt fields and make levitation difficult.  New techniques 
must be designed to provide a rich experience without 
disrupting levitation. 
LEVITATE: CREATING DYNAMIC DISPLAYS 
Our acoustic levitation system (Figure 1) uses two 8x4 
arrays of 40kHz ultrasound transducers (Murata MA40S4S, 
10mm). The arrays face each other and are vertically 
separated by 65mm, a similar arrangement to LeviPath [5] 
and Floating Charts [3].  The arrays are controlled by a 
custom board with two XMOS processors; the board uses 
information from a PC to generate the voltage necessary to 
drive the transducers to produce the required output. Note 
that 40kHz ultrasound is inaudible to humans. We used 
expanded polystyrene beads (1–2mm) for the levitating 
objects because of their suitable size and low density. 
We use a Leap Motion sensor to track user input. We use 
the position and direction of an extended index finger to 
determine which levitating object the user is pointing at. 
Since our system uses small objects (1–2mm), we place a 
virtual sphere at each object position; a user is pointing at 
an object when a vector in the direction of their finger 
intersects a sphere.  
The demonstration will present floating voxels, individual 
pixels that can be interacted with using touch.  For each 
pixel, users can select a pixel and drag it through 3D space 
to position it anywhere within the display area.  These 
manipulations are supported by visual, tactile, and audio 
feedback to create a dynamic multimodal experience.  
Key Issues for Levitating Displays 
Demonstrating the hardware at this time will be useful to 
the pervasive displays community to help explore the 
current design considerations for levitating particle displays 
and discovering potential use cases for this technology.  We 
are currently exploring the following key issues: 
Feedback techniques for physical interactions. Because 
physical manipulation of the particles in the display could 
disrupt levitation, feedback must be designed to create the 
sensation of direct manipulation without actual physical 
contact.  Additionally, in order to maximize the realism of 
the display, we must explore different combinations of 
modalities. We must design techniques that work within the 
constraints of the hard to create realistic experiences with 
the floating particles. 
User experience and acceptability.  The user experience 
and user expectations of these devices is still an open issue.  
We are still determining key factors in walk up experience, 
control and usability, social interactions, and user 
expectations. For example, while some users may be 
intrigued, others may be uncomfortable with the technology 
if it is not immediately apparent how it works.  
Applications areas for levitating displays.  Given this 
new form factor, new applications areas need to be 
developed that fully exploit the dynamic nature of the 
display to achieve new interactions not possible on 
traditional displays.  Related work such as Floating Charts 
[3] demonstrate how such displays can be used, but more 
research is needed to develop a range of application areas. 
VISION AND FUTURE DIRECTIONS 
There are numerous applications for this display technology 
that could be used in public spaces.  In our long-term 
vision, the computer can control the existence, form, and 
visual appearance of levitating objects of any given size 
composed of tangible "levitating particles". Users can reach 
into the levitating matter, feel it and manipulate it, with all 
feedback originating from the levitating object's position in 
mid-air.   For example, instead of having to reach for an 
iDrive dial in a car, users may just reach out and the dial is 
created directly under their hand. The flexible medium of 
floating particles could be used by artists to create new 
forms of digital interactive installations for public spaces.  
CONCLUSION 
This demonstration explores input techniques for a 
levitating particle display using acoustic fields.  With 
pointing gestures, users can select and manipulate particles 
in the display. 
ACKNOWLEDGMENTS 
This research is funded by the European Union Horizon 
2020 research and innovation programme (#737087: 
Levitate). 
REFERENCES 
1. Jinha Lee, Rehmi Post, and Hiroshi Ishii. 2011. 
ZeroN!: Mid-Air Tangible Interaction Enabled by 
Computer Controlled Magnetic Levitation. UIST 
2011. 
2. Asier Marzo, Sue Ann Seah, Bruce W. Drinkwater, 
Deepak Ranjan Sahoo, Benjamin Long, and Sriram 
Subramanian. 2015. Holographic acoustic elements 
for manipulation of levitated objects. Nature 
Communications, May: 1–7. 
3. Asier Marzo, Perez Themis, and Sriram 
Subramanian. 2016. Floating charts!: Data plotting 
using free-floating acoustically levitated 
representations Floating Charts!: Data Plotting 
using Free-Floating. IEEE Symposium on 3D User 
Interfaces 2016, January 2017: 3–7. 
4. Yoichi Ochiai, Hoshi Takayuki, and Jun Rekimoto. 
2014. Pixie Dust!: Graphics Generated by Levitated 
and Animated Objects in Computational Acoustic-
Potential Feild. ACM Trans. Graph. 
5. Themis Omirou, Asier Marzo, Sue Ann Seah, and 
Sriram Subramanian. 2015. LeviPath!: Modular 
Acoustic Levitation for 3D Path Visualisations. 
309–312. 
6. Deepak Ranjan Sahoo, Takuto Nakamura, and 
Asier Marzo. 2016. JOLED!: A Mid-air Display 
based on Electrostatic Rotation of Levitated Janus 
Objects. In UIST, 437–448. 


Levitate: Interaction with Floating Particle Displays

Enlighten

     Williamson, J., Freeman, E.  and Brewster, S. (2017) Levitate: Interaction with Floating Particle Displays. In: 6th ACM International Symposium on Pervasive Displays, Lugano, Switzerland, 7-9 June 2017, Article 24. ISBN 9781450350457(doi:10.1145/3078810.3084347)   This is the author’s final accepted version.  There may be differences between this version and the published version. You are advised to consult the publisher’s version if you wish to cite from it.  http://eprints.gla.ac.uk/142897/            Deposited on: 27 June 2017                Enlighten – Research publications by members of the University of Glasgow http://eprints.gla.ac.uk33640 Levitate: Interaction with Floating Particle Displays Julie R. Williamson, Euan Freeman, and Stephen Brewster University of Glasgow Glasgow, Scotland {FirstName.LastName}@glasgow.ac.uk  ABSTRACT This demonstration will showcase the current state of the art levitating particle display from the Levitate Project.  In this demonstration, we show a new type of display consisting of floating voxels, small levitating particles that can be positioned and moved independently in 3D space. Phased ultrasound arrays are used to acoustically levitate the particles and users can interaction with each particle individually using pointing gestures.  Users can interact with the system in a walk-up-and-use manner without any user instrumentation, creating an exciting opportunity to deploy these tangible displays in public spaces in the future.  This demonstration explores the design potential of floating voxels and how these may be used to create new types of user interfaces. Author Keywords Tangible Displays, Ultrasonic Levitation, Multimodal Displays. ACM Classification Keywords H.5.2. User Interfaces (Input devices and strategies). INTRODUCTION Novel display form factors are becoming more increasingly prevalent in public displays as technology because more robust and new input techniques are developed to enhance interaction at public displays.  Levitating particle displays are a promising approach to create tangible displays to support direct manipulation.  Sound can be used to levitate objects in mid-air by trapping them in the low-pressure nodes of standing waves.  Recent research has enhanced this simple concept with the ability to levitate multiple objects in a plane [4], move multiple objects in 3D [5], and rotate them as well [2,6]. This demonstration explores different techniques for interacting with individual pixels on a levitation particle display. We consider levitated objects as floating voxels, and see their potential as a new type of display. Displays created from floating voxels do not require any user instrumentation (e.g., head-mounted displays), can be viewed from many angles, and users can see through them, since the objects are held in mid-air by sound. This means they naturally support collaboration and could be used to create new multi-user interactions and experiences.  In this demonstration, we describe a design space for floating voxels and how these can be enhanced by multiple modalities.    Figure 1. Arrays of ultrasonic transducers can levitate particles to create tangible displays.  We are exploring different techniques for interacting with these displays.  Note that the blue background is only included to increase visibility of the levitating particles and is removed during demonstration so that users can look through the display. Top:  Two floating particles create a two pixel display suspended in mid-air.  Bottom: Users can select pixels by pointing without wearing any additional sensors. RELATED WORK Tangible displays made from dynamically controlled particles have been attempted using a variety of techniques.  Ishii's radical atoms vision, which describes human-material interaction through material that can change form and appearance dynamically, has inspired many different interfaces.  Levitation has been achieved using magnetic forces [1] and acoustic fields [3–5].   One area in need of additional research is how input and interaction should be design for levitating displays using acoustic fields.  Although acoustic field levitation provides a highly flexible and promising approach, direct input can disrupt fields and make levitation difficult.  New techniques must be designed to provide a rich experience without disrupting levitation. LEVITATE: CREATING DYNAMIC DISPLAYS Our acoustic levitation system (Figure 1) uses two 8x4 arrays of 40kHz ultrasound transducers (Murata MA40S4S, 10mm). The arrays face each other and are vertically separated by 65mm, a similar arrangement to LeviPath [5] and Floating Charts [3].  The arrays are controlled by a custom board with two XMOS processors; the board uses information from a PC to generate the voltage necessary to drive the transducers to produce the required output. Note that 40kHz ultrasound is inaudible to humans. We used expanded polystyrene beads (1–2mm) for the levitating objects because of their suitable size and low density. We use a Leap Motion sensor to track user input. We use the position and direction of an extended index finger to determine which levitating object the user is pointing at. Since our system uses small objects (1–2mm), we place a virtual sphere at each object position; a user is pointing at an object when a vector in the direction of their finger intersects a sphere.  The demonstration will present floating voxels, individual pixels that can be interacted with using touch.  For each pixel, users can select a pixel and drag it through 3D space to position it anywhere within the display area.  These manipulations are supported by visual, tactile, and audio feedback to create a dynamic multimodal experience.  Key Issues for Levitating Displays Demonstrating the hardware at this time will be useful to the pervasive displays community to help explore the current design considerations for levitating particle displays and discovering potential use cases for this technology.  We are currently exploring the following key issues: Feedback techniques for physical interactions. Because physical manipulation of the particles in the display could disrupt levitation, feedback must be designed to create the sensation of direct manipulation without actual physical contact.  Additionally, in order to maximize the realism of the display, we must explore different combinations of modalities. We must design techniques that work within the constraints of the hard to create realistic experiences with the floating particles. User experience and acceptability.  The user experience and user expectations of these devices is still an open issue.  We are still determining key factors in walk up experience, control and usability, social interactions, and user expectations. For example, while some users may be intrigued, others may be uncomfortable with the technology if it is not immediately apparent how it works.  Applications areas for levitating displays.  Given this new form factor, new applications areas need to be developed that fully exploit the dynamic nature of the display to achieve new interactions not possible on traditional displays.  Related work such as Floating Charts [3] demonstrate how such displays can be used, but more research is needed to develop a range of application areas. VISION AND FUTURE DIRECTIONS There are numerous applications for this display technology that could be used in public spaces.  In our long-term vision, the computer can control the existence, form, and visual appearance of levitating objects of any given size composed of tangible "levitating particles". Users can reach into the levitating matter, feel it and manipulate it, with all feedback originating from the levitating object's position in mid-air.   For example, instead of having to reach for an iDrive dial in a car, users may just reach out and the dial is created directly under their hand. The flexible medium of floating particles could be used by artists to create new forms of digital interactive installations for public spaces.  CONCLUSION This demonstration explores input techniques for a levitating particle display using acoustic fields.  With pointing gestures, users can select and manipulate particles in the display. ACKNOWLEDGMENTS This research is funded by the European Union Horizon 2020 research and innovation programme (#737087: Levitate). REFERENCES 1. Jinha Lee, Rehmi Post, and Hiroshi Ishii. 2011. ZeroN!: Mid-Air Tangible Interaction Enabled by Computer Controlled Magnetic Levitation. UIST 2011. 2. Asier Marzo, Sue Ann Seah, Bruce W. Drinkwater, Deepak Ranjan Sahoo, Benjamin Long, and Sriram Subramanian. 2015. Holographic acoustic elements for manipulation of levitated objects. Nature Communications, May: 1–7. 3. Asier Marzo, Perez Themis, and Sriram Subramanian. 2016. Floating charts!: Data plotting using free-floating acoustically levitated representations Floating Charts!: Data Plotting using Free-Floating. IEEE Symposium on 3D User Interfaces 2016, January 2017: 3–7. 4. Yoichi Ochiai, Hoshi Takayuki, and Jun Rekimoto. 2014. Pixie Dust!: Graphics Generated by Levitated and Animated Objects in Computational Acoustic-Potential Feild. ACM Trans. Graph. 5. Themis Omirou, Asier Marzo, Sue Ann Seah, and Sriram Subramanian. 2015. LeviPath!: Modular Acoustic Levitation for 3D Path Visualisations. 309–312. 6. Deepak Ranjan Sahoo, Takuto Nakamura, and Asier Marzo. 2016. JOLED!: A Mid-air Display based on Electrostatic Rotation of Levitated Janus Objects. In UIST, 437–448. 

http://eprints.gla.ac.uk/142897/7/142897.pdf

Levitate: Interaction with Floating Particle Displays

Abstract

Similar works

Full text

Available Versions

Enlighten: Publications

Enlighten