We present an overview of the approach we used and the challenges we encountered while designing software for smartphones to facilitate communication and improve social skills of children with severe autism spectrum disorder (ASD). We employed participatory design, using special education teachers of children with ASD as proxies for our target population

De Leo, Gianluca

Leroy, Gondy

English

Scholarship@Claremont

Claremont CollegesScholarship @ ClaremontCGU Faculty Publications and Research CGU Faculty Scholarship6-1-2008Smartphones to Facilitate Communication andImprove Social Skills of Children with SevereAutism Spectrum Disorder: Special EducationTeachers as ProxiesGondy LeroyClaremont Graduate UniversityGianluca De LeoOld Dominion UniversityThis Conference Proceeding is brought to you for free and open access by the CGU Faculty Scholarship at Scholarship @ Claremont. It has beenaccepted for inclusion in CGU Faculty Publications and Research by an authorized administrator of Scholarship @ Claremont. For more information,please contact scholarship@cuc.claremont.edu.Recommended CitationGianluca De Leo and Gondy Leroy. 2008. Smartphones to facilitate communication and improve social skills of children with severeautism spectrum disorder: special education teachers as proxies. In Proceedings of the 7th international conference on Interactiondesign and children (IDC '08). ACM, New York, NY, USA, 45-48. DOI=10.1145/1463689.1463715 http://doi.acm.org/10.1145/1463689.1463715G. De Leo and G. Leroy, Smartphones to Faciliate Communication and Improve Social Skills of Children with Severe Autism Spectrum Disorder: Special Education Teachers as Proxies, 7th International Conference on Interaction Design & Children – Workshop on Designing for Children with Special Needs, Chicago, June 11-13, 2008. 1 Smartphones to Facilitate Communication and Improve Social Skills of Children with Severe Autism Spectrum Disorder:  Special Education Teachers as Proxies   Abstract We present an overview of the approach we used and the challenges we encountered while designing software for smartphones to facilitate communication and improve social skills of children with severe autism spectrum disorder (ASD). We employed participatory design, using special education teachers of children with ASD as proxies for our target population. Keywords Assisted cognition, autism, health informatics, design with proxies ACM Classification Keywords K.4.2 Computers and Society: Social Issues—Assistive technologies for persons with disabilities; H.5.2 [Information Interfaces and Presentation]: User Interfaces - Evaluation/ methodology, graphical user interfaces, prototyping, user-centered design. Introduction Autism spectrum disorder (ASD) is one of the most prevalent neurodevelopment disorders and its incidence Copyright is held by the author/owner(s). Gianluca De Leo, PhD MBA Virginia Modeling Analysis and Simulation Center Old Dominion University 1030 University Blvd. Suffolk, VA 23435 USA gdeleo@odu.edu  Gondy Leroy, PhD School of Information Systems and Technology Claremont Graduate University 130 E. Ninth Street Claremont, CA 91711, gondy.leroy@cgu.edu  2 continues to grow. It currently affects 1 in every 150 American children [1]. The main symptoms of ASD are impairments in reciprocal social interaction and communication, and restricted and stereotyped patterns of interests [2]. Functional communication and forming social relationships are the most difficult and dominant problems in ASD [3]. About a third to half of individuals with autism never develop sufficient natural speech to meet their daily communication needs [4].  One of the most popular treatment approaches among special education teachers is to teach children with ASD to communicate and interact with others using images. The most commonly system used is the Picture Exchange Communication System (PECS) [5-7].   Software can be developed to help people with severe autism communicate. However, since good software design requires good communication with users, designing for children with ASD requires a special approach. Since we can not engage the final user (the children with ASD) in the design process due to communication barriers, we involve special education teachers in the design in two ways: they help design the product in their role as teachers, allowing us to do participatory design, and they act as proxy users for autistic children. This ensures that the software will be nearly optimal before testing with autistic persons. Picture Exchange Communication System (PECS) PECS utilizes printed and laminated pictures and applied behavior analysis is commonly used to teach the picture exchange. There are several phases which increasingly encourage independent communication. Although very popular, low-tech approaches such as PECS are cumbersome, time-consuming, and inefficient. Teachers with multiple ASD children in their classrooms spend a significant amount of time printing, cutting, and laminating pictures for all of their students. It becomes increasingly difficult to customize the set of pictures based on personal desire, interests, and the communication skills of each child. For a child, the length of time needed to find a picture and then show it may discourage the expression of emotions, such as happiness, which can rise after an activity the child enjoyed.  Existing Technical Solutions Existing electronic communication devices such as DynaVox [8], Palmtop Impact [8] and Cyrano Communicator [9] have been designed for high functioning learners. Other devices such as Visual Assistant, Pocket Reader, Pocket Coach, and Pocket Compass [10] have been designed to provide multi-modal assistance to individuals with moderately impaired intellectual performance. The electronic communication devices available on the market have three primary problems: 1) not easily customized to the needs of each child (no possibility to use personal or ad hoc images), 2) no qualitative and quantitative usage data for teachers, 3) and no analysis tools to measure the effects on child development.  Our Smartphone Solution We have designed and developed software for smartphones that offers a new media for communicating and socializing while overcoming the shortcomings with paper pictures. There are two modes that the smartphone can be in: operational and display mode. In operational mode, users can form a message (Figure 1a) by touching images with their finger. The  3 selected images are combined. Then, clicking a button places the message in the middle of the screen and the handheld device goes into display mode. In display mode, the images are enlarged as much as possible (Figure 1b) and the touch-screen does not respond to touch anymore, allowing others to hold the device without changing the message.    Figure 1: Smatphone application Our solution provides two augmentations that may have an enormous impact on therapy. First, we can track how often the child uses the smartphone to form messages. Today there are no convenient and systematic approaches to gather data for these very young children with ASD. It has been impractical and very difficult to do when using PECS. The second enhancement is that images shown using the smartphone can be easily customized based on the needs, interests, and preferences of each child, but also for groups of children without extra effort. It is possible to use comic based pictures with and without text on the top, real-life photos of items such as an apple, as well as personal images such as the child’s house, parents, and family members. Our solution can be easily adapted to children with other communication impairments. As of March 2008, a first functional prototype has been developed and its evaluation is in progress. Participatory Design and Designing by Proxy Because the final users of our smartphone are the children with severe ASD, we employed participatory design using special education teachers as proxies to our target population. We work with special education teachers in multiple public and private schools in North America. We are expecting enrolling in the study 10 teachers. Initial design insights are based on interviews with therapists, parents, and teachers in Southern California and Arizona. Then, a first low-end prototype was developed at Claremont Graduate University and compared with paper-images [11]. We continued refining the design for message forming and tracking in interaction with special education teachers by means of semi-structured interviews.   The teachers adopt two roles in the design. First, they help design the software from a teacher perspective so that the software optimizes their use of time when  4 teaching children. They do not want to spend extra time preparing images for multiple children but are eager to be able to monitor and analyze image usage. Second, teachers also serve as proxy users for autistic children and help design the software keeping in mind strengths and limitations of the children they work with. They spend time with children with autism on a daily basis and are familiar with the specific problems that will be encountered. Conclusion Designing health informatics system for a vulnerable group such children with ASD has additional difficulties. We propose to overcome these barriers by having teacher assuming different roles: they are participants in the design but also serve as proxy users. We are currently pursuing additional funding to test our smarphone software with children with ASD in controlled clinical trials. Acknowledgements This work was sponsored by a grant from Microsoft Research. References and Citations [1] Prevalence of the Autism Spectrum Disorders in Multiple Areas of the United States, Surveillance Years 2000 and 2002. Retrieved February 13, 2008, from the http://www.cdc.gov/ncbddd/dd/addmprevalence.htm. [2] Kanner L. Autistic disturbances of affective contact. Nerv Child. 1943; 2: 217–50. Acta Paedopsychiatr. Reprint. 1968; 35 (4): 100–136. [3] Kabot S, Masi W, Segal M. Advances in the Diagnosis and Treatment of Autism Spectrum Disorders. Professional Psychology: Research and Practice. 2003; 34: 26-33. [4] Noens I, Van Berckelaer-Onnes I, Verpoorten R, Van Duijn G. The ComFor: an instrument for the indication of augmentative communication in people with autism and intellectual disability. J Intellect Disabil Res. 2006; 50 (9): 621–632. [5] Bondy AS, Frost LA. The Picture Exchange Communication System. Focus on Autistic Behavior. 1994; 1-19. [6] Bondy AS, Frost L. The Picture Exchange Communication System. Behav Modif. 2001; 25(5):725-744. [7] Bondy AS. PECS: Potential Benefits and Risks. The Behavior Analyst Today. 2001; 2: 127-132. [8] Dynavox Technologies. Retrieved February 13, 2008, from the Dynavox Technologies Website: http://www.dynavoxtech.com/. [9] Cyrano Communicator. Retrieved February 13, 2008, from the Cyrano Communicator Website: http://www.cyranocommunicator.com/. [10] Visual Assistant. Retrieved February 13, 2008, from the AbleLink Technologies Website: http://www.ablelinktech.com/_handhelds/visualassistant.asp. [11] T. Miller, G. Leroy, J. Huang, S. Chuang, and M. H. Charlop-Christy, "Using a Digital Library of Images for Communication: Comparison of a Card-Based System to PDA Software," presented at First International Conference on Design Science Research in Information Systems and Technology, Claremont, CA, 2006 

Smartphones to Facilitate Communication and Improve Social Skills of Children with Severe Autism Spectrum Disorder: Special Education Teachers as Proxies

Keck Graduate Institute

Claremont Colleges
Scholarship @ Claremont
CGU Faculty Publications and Research CGU Faculty Scholarship
6-1-2008
Smartphones to Facilitate Communication and
Improve Social Skills of Children with Severe
Autism Spectrum Disorder: Special Education
Teachers as Proxies
Gondy Leroy
Claremont Graduate University
Gianluca De Leo
Old Dominion University
This Conference Proceeding is brought to you for free and open access by the CGU Faculty Scholarship at Scholarship @ Claremont. It has been
accepted for inclusion in CGU Faculty Publications and Research by an authorized administrator of Scholarship @ Claremont. For more information,
please contact scholarship@cuc.claremont.edu.
Recommended Citation
Gianluca De Leo and Gondy Leroy. 2008. Smartphones to facilitate communication and improve social skills of children with severe
autism spectrum disorder: special education teachers as proxies. In Proceedings of the 7th international conference on Interaction
design and children (IDC '08). ACM, New York, NY, USA, 45-48. DOI=10.1145/1463689.1463715 http://doi.acm.org/10.1145/
1463689.1463715
G. De Leo and G. Leroy, Smartphones to Faciliate Communication and Improve Social Skills of Children with Severe Autism Spectrum Disorder: 
Special Education Teachers as Proxies, 7th International Conference on Interaction Design & Children – Workshop on Designing for Children with 
Special Needs, Chicago, June 11-13, 2008. 1 
Smartphones to Facilitate 
Communication and Improve Social 
Skills of Children with Severe Autism 
Spectrum Disorder:  Special Education 
Teachers as Proxies 
 
 
Abstract 
We present an overview of the approach we used and 
the challenges we encountered while designing 
software for smartphones to facilitate communication 
and improve social skills of children with severe autism 
spectrum disorder (ASD). We employed participatory 
design, using special education teachers of children 
with ASD as proxies for our target population. 
Keywords 
Assisted cognition, autism, health informatics, design 
with proxies 
ACM Classification Keywords 
K.4.2 Computers and Society: Social Issues—Assistive 
technologies for persons with disabilities; H.5.2 
[Information Interfaces and Presentation]: User 
Interfaces - Evaluation/ methodology, graphical user 
interfaces, prototyping, user-centered design. 
Introduction 
Autism spectrum disorder (ASD) is one of the most 
prevalent neurodevelopment disorders and its incidence Copyright is held by the author/owner(s). 
Gianluca De Leo, PhD MBA 
Virginia Modeling Analysis and Simulation Center 
Old Dominion University 
1030 University Blvd. 
Suffolk, VA 23435 USA 
gdeleo@odu.edu 
 
Gondy Leroy, PhD 
School of Information Systems and Technology 
Claremont Graduate University 
130 E. Ninth Street 
Claremont, CA 91711, 
gondy.leroy@cgu.edu 
 2 
continues to grow. It currently affects 1 in every 150 
American children [1]. The main symptoms of ASD are 
impairments in reciprocal social interaction and 
communication, and restricted and stereotyped 
patterns of interests [2]. Functional communication and 
forming social relationships are the most difficult and 
dominant problems in ASD [3]. About a third to half of 
individuals with autism never develop sufficient natural 
speech to meet their daily communication needs [4].  
One of the most popular treatment approaches among 
special education teachers is to teach children with ASD 
to communicate and interact with others using images. 
The most commonly system used is the Picture 
Exchange Communication System (PECS) [5-7].   
Software can be developed to help people with severe 
autism communicate. However, since good software 
design requires good communication with users, 
designing for children with ASD requires a special 
approach. Since we can not engage the final user (the 
children with ASD) in the design process due to 
communication barriers, we involve special education 
teachers in the design in two ways: they help design 
the product in their role as teachers, allowing us to do 
participatory design, and they act as proxy users for 
autistic children. This ensures that the software will be 
nearly optimal before testing with autistic persons. 
Picture Exchange Communication System 
(PECS) 
PECS utilizes printed and laminated pictures and 
applied behavior analysis is commonly used to teach 
the picture exchange. There are several phases which 
increasingly encourage independent communication. 
Although very popular, low-tech approaches such as 
PECS are cumbersome, time-consuming, and 
inefficient. Teachers with multiple ASD children in their 
classrooms spend a significant amount of time printing, 
cutting, and laminating pictures for all of their students. 
It becomes increasingly difficult to customize the set of 
pictures based on personal desire, interests, and the 
communication skills of each child. For a child, the 
length of time needed to find a picture and then show it 
may discourage the expression of emotions, such as 
happiness, which can rise after an activity the child 
enjoyed.  
Existing Technical Solutions 
Existing electronic communication devices such as 
DynaVox [8], Palmtop Impact [8] and Cyrano 
Communicator [9] have been designed for high 
functioning learners. Other devices such as Visual 
Assistant, Pocket Reader, Pocket Coach, and Pocket 
Compass [10] have been designed to provide multi-
modal assistance to individuals with moderately 
impaired intellectual performance. The electronic 
communication devices available on the market have 
three primary problems: 1) not easily customized to 
the needs of each child (no possibility to use personal 
or ad hoc images), 2) no qualitative and quantitative 
usage data for teachers, 3) and no analysis tools to 
measure the effects on child development.  
Our Smartphone Solution 
We have designed and developed software for 
smartphones that offers a new media for 
communicating and socializing while overcoming the 
shortcomings with paper pictures. There are two modes 
that the smartphone can be in: operational and display 
mode. In operational mode, users can form a message 
(Figure 1a) by touching images with their finger. The 
 3 
selected images are combined. Then, clicking a button 
places the message in the middle of the screen and the 
handheld device goes into display mode. In display 
mode, the images are enlarged as much as possible 
(Figure 1b) and the touch-screen does not respond to 
touch anymore, allowing others to hold the device 
without changing the message.   
 
Figure 1: Smatphone application 
Our solution provides two augmentations that may 
have an enormous impact on therapy. First, we can 
track how often the child uses the smartphone to form 
messages. Today there are no convenient and 
systematic approaches to gather data for these very 
young children with ASD. It has been impractical and 
very difficult to do when using PECS. The second 
enhancement is that images shown using the 
smartphone can be easily customized based on the 
needs, interests, and preferences of each child, but also 
for groups of children without extra effort. It is possible 
to use comic based pictures with and without text on 
the top, real-life photos of items such as an apple, as 
well as personal images such as the child’s house, 
parents, and family members. Our solution can be 
easily adapted to children with other communication 
impairments. As of March 2008, a first functional 
prototype has been developed and its evaluation is in 
progress. 
Participatory Design and Designing by Proxy 
Because the final users of our smartphone are the 
children with severe ASD, we employed participatory 
design using special education teachers as proxies to 
our target population. We work with special education 
teachers in multiple public and private schools in North 
America. We are expecting enrolling in the study 10 
teachers. 
Initial design insights are based on interviews with 
therapists, parents, and teachers in Southern California 
and Arizona. Then, a first low-end prototype was 
developed at Claremont Graduate University and 
compared with paper-images [11]. We continued 
refining the design for message forming and tracking in 
interaction with special education teachers by means of 
semi-structured interviews.   
The teachers adopt two roles in the design. First, they 
help design the software from a teacher perspective so 
that the software optimizes their use of time when 
 4 
teaching children. They do not want to spend extra 
time preparing images for multiple children but are 
eager to be able to monitor and analyze image usage. 
Second, teachers also serve as proxy users for autistic 
children and help design the software keeping in mind 
strengths and limitations of the children they work 
with. They spend time with children with autism on a 
daily basis and are familiar with the specific problems 
that will be encountered. 
Conclusion 
Designing health informatics system for a vulnerable 
group such children with ASD has additional difficulties. 
We propose to overcome these barriers by having 
teacher assuming different roles: they are participants 
in the design but also serve as proxy users. We are 
currently pursuing additional funding to test our 
smarphone software with children with ASD in 
controlled clinical trials. 
Acknowledgements 
This work was sponsored by a grant from Microsoft 
Research. 
References and Citations 
[1] Prevalence of the Autism Spectrum Disorders in 
Multiple Areas of the United States, Surveillance Years 
2000 and 2002. Retrieved February 13, 2008, from the 
http://www.cdc.gov/ncbddd/dd/addmprevalence.htm. 
[2] Kanner L. Autistic disturbances of affective contact. 
Nerv Child. 1943; 2: 217–50. Acta Paedopsychiatr. 
Reprint. 1968; 35 (4): 100–136. 
[3] Kabot S, Masi W, Segal M. Advances in the 
Diagnosis and Treatment of Autism Spectrum 
Disorders. Professional Psychology: Research and 
Practice. 2003; 34: 26-33. 
[4] Noens I, Van Berckelaer-Onnes I, Verpoorten R, 
Van Duijn G. The ComFor: an instrument for the 
indication of augmentative communication in people 
with autism and intellectual disability. J Intellect Disabil 
Res. 2006; 50 (9): 621–632. 
[5] Bondy AS, Frost LA. The Picture Exchange 
Communication System. Focus on Autistic Behavior. 
1994; 1-19. 
[6] Bondy AS, Frost L. The Picture Exchange 
Communication System. Behav Modif. 2001; 
25(5):725-744. 
[7] Bondy AS. PECS: Potential Benefits and Risks. The 
Behavior Analyst Today. 2001; 2: 127-132. 
[8] Dynavox Technologies. Retrieved February 13, 
2008, from the Dynavox Technologies Website: 
http://www.dynavoxtech.com/. 
[9] Cyrano Communicator. Retrieved February 13, 
2008, from the Cyrano Communicator Website: 
http://www.cyranocommunicator.com/. 
[10] Visual Assistant. Retrieved February 13, 2008, 
from the AbleLink Technologies Website: 
http://www.ablelinktech.com/_handhelds/visualassista
nt.asp. 
[11] T. Miller, G. Leroy, J. Huang, S. Chuang, and M. H. 
Charlop-Christy, "Using a Digital Library of Images for 
Communication: Comparison of a Card-Based System 
to PDA Software," presented at First International 
Conference on Design Science Research in Information 
Systems and Technology, Claremont, CA, 2006 


https://scholarship.claremont.edu/cgi/viewcontent.cgi?article=1104&amp;context=cgu_fac_pub

Smartphones to Facilitate Communication and Improve Social Skills of Children with Severe Autism Spectrum Disorder: Special Education Teachers as Proxies

Abstract

Similar works

Full text

Available Versions

Scholarship@Claremont

Keck Graduate Institute