We propose that accessibility research can lay the foundation for technology
that can be used to augment the perception of everyone. To show how this can be
achieved, we present three case studies of our research in which we demonstrate
our approaches for impaired colour vision, situational visual impairments and
situational hearing impairment.Comment: 5 pages, 4 figure

Flatla, David R.

Gorman, Benjamin M.

Mauderer, Michael

Tigwell, Garreth W.

arXiv

We propose that accessibility research can lay the foundation for technology that can be used to augment the perception of everyone. To show how this can be achieved, we present three case studies of our research in which we demonstrate our approaches for impaired colour vision, situational visual impairments and situational hearing impairment

University of Dundee Online Publications

                                                              University of DundeeBeyond AccessibilityMauderer, Michael; Tigwell, Garreth; Gorman, Benjamin M.; Flatla, DavidPublication date:2017Document VersionFinal published versionLink to publication in Discovery Research PortalCitation for published version (APA):Mauderer, M., Tigwell, G. W., Gorman, B. M., & Flatla, D. R. (2017). Beyond Accessibility: Lifting PerceptualLimitations for Everyone. Paper presented at CHI '17, Denver, United States.General rightsCopyright and moral rights for the publications made accessible in Discovery Research Portal are retained by the authors and/or othercopyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated withthese rights. • Users may download and print one copy of any publication from Discovery Research Portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain. • You may freely distribute the URL identifying the publication in the public portal.Take down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Download date: 01. Jul. 2017Beyond Accessibility: LiftingPerceptual Limitations for EveryoneMichael MaudererUniversity of DundeeDundee, UKm.mauderer@dundee.ac.ukGarreth W. TigwellUniversity of DundeeDundee, UKg.w.tigwell@dundee.ac.ukBenjamin M. GormanUniversity of DundeeDundee, UKb.gorman@dundee.ac.ukDavid R. FlatlaUniversity of DundeeDundee, UKd.flatla@dundee.ac.ukProceedings of the CHI 2017 Workshop on Amplification and Augmentation of HumanPerception, May 07, 2017, Denver, CO, USA. Copyright is held by theowner/author(s).AbstractWe propose that accessibility research can lay the founda-tion for technology that can be used to augment the per-ception of everyone. To show how this can be achieved,we present three case studies of our research in which wedemonstrate our approaches for impaired colour vision,situational visual impairments and situational hearing im-pairment.Author Keywordsperception, augmentation, vision, hearing, accessibilityACM Classification KeywordsH.5.m [Information interfaces & presentation (HCI)]: Misc.IntroductionAccessibility research aims to create technologies that ben-efit people with disabilities. These technologies can rangefrom prosthesis for amputees to digital tools to increase dis-play contrast for people with low vision. Often the goal isto alleviate challenges brought on due to the impairmentand achieve equality to people with typical abilities. In somecases, however, the tools that are created to deal with animpairment allow the people that use them to exceed typi-cal abilities, or potentially provide a benefit to people of allabilities.Neil Harbisson, who was born with total colour blindness,uses a device that records colour images in front of him andturn them into vibrations in his skull allowing him to ”hear“the colours in the image. The device is set up in a way thatallows him to not only see what is in front of him but alsoreceive images from other sources, i.e., pictures sent to himfrom people in other locations [11], thus in a way expandinghis perceptual capabilities beyond those of a typical person.An example of accessible technology that helps everyoneare design guidelines for websites that address colours thatare hard to differentiate for people with colour vision defi-ciencies and contrast for text which can be hard to read forpeople with low vision. While initially intended to alleviatechallenges arising from impairments, these guidelines re-sult in website designs that benefit everyone viewing thewebsite through clearer colours and increased contrast [18].The remainder of the paper will present digital assistive sys-tems that deal with perceptual issues and show how frominvestigating (situational) disabilities we can develop tech-nologies, tools, or design guidelines that can benefit andenhance everyone’s perception. To do this, we presentthree case studies that showcase our results for (1) im-paired colour vision, (2) situational visual impairments, and(3) situational hearing impairment.Case Study 1: Impaired Colour VisionFigure 1: Examples of colouridentification techniques for thetask “select the yellow pencil”. Topleft to bottom right: typical vision,impaired colour vision (ICV)simulation showing difficulty, ourColourNames, ColourMeters, andColourPopper technique.Colour vision is a vital component of day-to-day living; itsloss can severely limit a person’s livelihood (e.g., airplanepilots, electricians, dentists, and visual artists all requireaccurate colour vision), threaten health and safety (e.g.,not spotting sunburns and rashes, misidentifying medica-tion), make food preparation dangerous (e.g., undercookingmeat, detecting mouldiness), and even challenge some-one’s social acceptability (e.g., by inhibiting clothing coordi-nation and home decorating) [3].To help address the challenges imposed by impaired colourvision (ICV), we developed the concept of Situation-SpecificModelling (SSM) and used it to construct a model of colourperception tunable to an individual’s unique abilities andenvironment [5, 6, 7]. We also developed a recolouring toolfor improving web colour differentiability for people with ICVthat was designed to maintain the subjective experience ofthe original colour scheme (e.g., by keeping ‘warm’ colourschemes ‘warm’) while improving colour accessibility [8].Next, we went beyond recolouring to focus on improvingcolour identification and discrimination. We developed re-visualization techniques for mapping colour informationto other visual properties (ColourID) [4], which improvedcolour identification to nearly 100% for participants withICV (see Figure 1 for examples of the techniques). We alsoinvestigated the use of eye tracking based manipulationsof colours to exploit with the eventual aim to make coloursmore differentiable to the observer [15].Every technique we have developed has the characteristicof enabling people to perceive visual properties that theytypically cannot (i.e., we make the invisible visible – in thiscase invisible colour). However, the same techniques canbe used to let people with typical abilities ‘see the invisible’.Next, we plan to extend our techniques to give people theability to perceive light outside of the visible spectrum: in-frared and ultraviolet. We will apply the same techniquesthat work for people with ICV to enable all people to per-ceive information they normally wouldn’t be able to see atall. This has the potential to greatly improve health (e.g., byenabling rapid visual triage for fever in epidemic situations)and safety (e.g., giving airport security the ability to ‘see’emitted heat).Case Study 2: Situational Visual ImpairmentA situational impairment is when a person experiences diffi-culty completing tasks that would not typically present prob-lems (e.g., the difference when typing a message while sta-tionary vs. running, or reading a message on a display of amobile device on an overcast vs. sunny day). Various fac-tors can cause a situational impairment, and it is not limitedto environmental conditions. This concept within computinghas been discussed by Newell [16] and Sears et al. [17].The portability and computing power of mobile devices hasextended our freedom for completing tasks at our conve-nience. However, with this freedom there are substantialissues; situational contexts can become difficult to con-trol and can also be unpredictable. We focus specificallyon situational visual impairment (SVI) when using mobiledevices, which describes situations in which it becomesdifficult to see content on the mobile device when it wouldotherwise not be an issue (e.g., bright light reflecting on thedisplay and low contrast text as demonstrated in Figure 2).Mobile screens are used in diverse situations and for di-verse purposes, e.g., car navigation, in the medical industry,and education. Currently, we are looking at the SVIs expe-rienced when using mobile devices under bright lighting.Although it is well documented that mobile screens typicallydegrade in quality as ambient light increase (e.g., [13, 14]),we do not know what tasks people are doing on their mobiledevices when experiencing SVI, what other factors are in-volved that can exacerbate the problem, and what solutionspeople currently employ to overcome the problem.Figure 2: A smartphone in abrightly lit room, with half thedevice in shade. The low contrastmenu text at the bottom of thescreen becomes difficult to read.Figure 3: Our speechreadingacquisition framework, with twodimensions: Type of Skill andAmount of Information, each splitinto three levels(Analytic/Hybrid /Synthetic andLow /Medium/High, respectively)Figure 4: PhonemeViz mockupviewed through Epson Moverioglasses for the word ”bat“.We predict that there are different solutions required to re-duce the occurrence of SVI since it is likely to involve mul-tiple factors at any one time. For example, if the contentbeing viewed has a low contrast design then it will becomeeven harder to read on a display in direct sunlight. One so-lution would be to provide designers with a tool that helpsthem assess the robustness of their design’s perceptibilityunder increasing ambient light, similar to an accessibilitytool that we have previously developed to allow web de-signers to select a colour scheme for people with ICV, whilemaintaining the designers’ creative freedom [19].Solving this issue of situational visual impairment would bea benefit for everyone who is using mobile devices whereoverly bright conditions occur.Case Study 3: Situational Hearing ImpairmentThe WHO estimates that on a global scale 360 million peo-ple (~5%) worldwide have disabling hearing loss1 [21].Hearing loss is an everyday problem as it interferes withconversations by making it difficult to understanding whatothers are saying [20].People who are deaf or have a hearing loss find that speechread-ing (SR) (commonly called lipreading) can overcome manyof the barriers when communicating with others [2]. SRhelps to improve conversational confidence (thereby reduc-ing social isolation), enhance employability, and improveeducational outcomes [20].However, SR is a skill that takes considerable practice andtraining to acquire [12]. One reason for this is caused bynearly identical mouth shapes (visemes) producing severalspeech sounds (phonemes); there is not a one-to-manymapping from visemes to phonemes. This decreases com-prehension and causes confusion and frustration duringconversations. Publicly-funded SR classes are sometimesprovided and have been shown to improve SR acquisi-tion [1]. However, classes are only provided in few countries1Hearing loss of more than 40 dB in the better hearing ear in adultsand more than 30 dB in the better hearing ear in children.around the world; there is an insufficient number of classesrunning in areas in which they are provided (e.g., only 50 ofan estimated 325 required classes are currently running inScotland [1]) and classes require mobility to attend.To help expand SR training, we developed a novel frame-work [10](Figure 3) that can be used to develop Speech-reading Acquisition Tools (SATs) – a new type of technologydesigned specifically to improve SR acquisition. Throughthe development and release of SATs, people with hear-ing loss will be able to augment their class-based learningor learn on their own if no suitable classes are available.These SATs would also benefit those who are experiencinga situational hearing impairment.PhonemeViz is a SAT we have developed using our frame-work. It places the character(s) of the most recently spo-ken initial phoneme just to the side of a speaker’s lips.This design should enable a speechreader to focus onthe speaker’s eyes and lip movements (as in traditionalspeechreading), while also monitoring changes in Phone-meViz’s state using their peripheral vision to help disam-biguate confusing visemes. PhonemeViz can be overlaidonto video or displayed on a transparent head mounted dis-play (Figure 4, left) to augment natural speechreading andenhance speechreading acquisition. In a pilot evaluation ofa PhonemeViz prototype [9] we found it enabled all partici-pants to achieve 100% word recognition.When people encounter listening issues due to noise orother factors in their environment this constitutes a situ-ational hearing impairment (e.g., in a noisy restaurant oron a construction site). Through the use of SATs, such asPhonemeViz, people who experience situational hearingimpairment can then rely on SR in these challenging situa-tions to facilitate communication.ConclusionWith our case studies, we have shown how we can developperceptual augmentations from research on assistive sys-tems that target disabilities. From research on impairedcolour vision, we have arrived at techniques that allow theperception of colour and information outside of the capabili-ties of the human visual system. From looking at situationalvisual impairments, we will develop tools and guidelinesthat can make content on mobile devices easier to perceive.Finally, from investigating hearing impairment, we have ar-rived at tools that can help people in situations that cancause situational hearing impairment, e.g., noisy environ-ments. While initially each case study focuses on a specificaccessibility problem, the results can be generalised andapplied to the benefit of everyone.REFERENCES1. L Armstrong. 2015. On everybody’s lips.http://www.scotlipreading.org.uk/files/1914/2686/1587/On_everybodys_lips_-_report.pdf.(2015).2. Ruth Campbell, Barbara Dodd, and Denis K Burnham.1998. Hearing by eye II. Vol. 2. Psychology Press.3. Barry L Cole. 2004. The Handicap of Abnormal ColourVision. Clinical and Experimental Optometry (2004).4. David R Flatla, Alan R Andrade, Ross D Teviotdale,Dylan L Knowles, and Craig Stewart. 2015. ColourID:Improving Colour Identification for People with ImpairedColour Vision. In CHI ’15.5. David R Flatla and Carl Gutwin. Individual models ofcolor differentiation to improve interpretability ofinformation visualization. In CHI’10.6. David R Flatla and Carl Gutwin. Situation-specificmodels of color differentiation. In TACCESS’12.7. David R Flatla and Carl Gutwin. So that’s what you see:building understanding with personalized simulations ofcolour vision deficiency. In SIGACCESS’12.8. David R Flatla, Katharina Reinecke, Carl Gutwin, andKrzysztof Z Gajos. SPRWeb: Preserving subjectiveresponses to website colour schemes throughautomatic recolouring. In CHI’13.9. Benjamin M. Gorman. Reducing viseme confusion inspeech-reading. In SIGACCESS’16.10. Benjamin M. Gorman and David R. Flatla. AFramework for Speechreading Acquisition Tools. InCHI’17.11. Neil Harbisson. 2012. I listen to color. TEDXTalk,http://www. ted.com/talks/neil_harbisson_i_listen_to_color (2012).12. Harriet Kaplan, Scott J Bally, and Carol Garretson.1985. Speechreading: A way to improveunderstanding. Gallaudet University Press.13. Youn Jin Kim, M Ronnier Luo, Wonhee Choe,Hong Suk Kim, Seung Ok Park, Yeseul Baek, PeterRhodes, Seongdeok Lee, and Chang Yeong Kim. 2008.Factors affecting the psychophysical image qualityevaluation of mobile phone displays: the case oftransmissive liquid-crystal displays. JOSA A (2008).14. Peter Liu, Fahad Zafar, and Aldo Badano. 2014. Theeffect of ambient illumination on handheld displayimage quality. Journal of digital imaging (2014).15. Michael Mauderer, David R. Flatla, and Miguel A.Nacenta. Gaze-Contingent Manipulation of ColorPerception. In CHI’16.16. Alan F Newell. 1995. Extra-ordinary human-computerinteraction. In Extra-ordinary human-computerinteraction, Alistair D. N. Edwards (Ed.). CambridgeUniversity Press.17. Andrew Sears, Min Lin, Julie Jacko, and Yan Xiao.2003. When computers fade: Pervasive computing andsituationally-induced impairments and disabilities. InHCI International.18. John M. Slatin and Sharron Rush. 2002. MaximumAccessibility: Making Your Web Site More Usable forEveryone.19. Garreth W. Tigwell, David R. Flatla, and Neil D.Archibald. ACE: A Colour Palette Design Tool forBalancing Aesthetics and Accessibility. InTACCESS’17.20. Nancy Tye-Murray. 2014. Foundations of auralrehabilitation: Children, adults, and their familymembers. Nelson Education.21. WHO 2015. Deafness and hearing loss, Fact sheetN.300. http://www.who.int/mediacentre/factsheets/fs300/en/.(2015).

Beyond Accessibility:Lifting Perceptual Limitations for Everyone

                                                                    University of DundeeBeyond AccessibilityMauderer, Michael; Tigwell, Garreth W.; Gorman, Benjamin M.; Flatla, David R.Publication date:2017Licence:No Licence / UnknownDocument VersionPublisher's PDF, also known as Version of recordLink to publication in Discovery Research PortalCitation for published version (APA):Mauderer, M., Tigwell, G. W., Gorman, B. M., & Flatla, D. R. (2017). Beyond Accessibility: Lifting PerceptualLimitations for Everyone. Paper presented at CHI '17, Denver, United States. http://www.hcilab.org/amplify-chi17/wp-content/uploads/sites/13/2017/04/AmplifyCHI17_Mauderer.pdfGeneral rightsCopyright and moral rights for the publications made accessible in Discovery Research Portal are retained by the authors and/or othercopyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated withthese rights. • Users may download and print one copy of any publication from Discovery Research Portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain. • You may freely distribute the URL identifying the publication in the public portal.Take down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Download date: 30. Jul. 2023Beyond Accessibility: LiftingPerceptual Limitations for EveryoneMichael MaudererUniversity of DundeeDundee, UKm.mauderer@dundee.ac.ukGarreth W. TigwellUniversity of DundeeDundee, UKg.w.tigwell@dundee.ac.ukBenjamin M. GormanUniversity of DundeeDundee, UKb.gorman@dundee.ac.ukDavid R. FlatlaUniversity of DundeeDundee, UKd.flatla@dundee.ac.ukProceedings of the CHI 2017 Workshop on Amplification and Augmentation of HumanPerception, May 07, 2017, Denver, CO, USA. Copyright is held by theowner/author(s).AbstractWe propose that accessibility research can lay the founda-tion for technology that can be used to augment the per-ception of everyone. To show how this can be achieved,we present three case studies of our research in which wedemonstrate our approaches for impaired colour vision,situational visual impairments and situational hearing im-pairment.Author Keywordsperception, augmentation, vision, hearing, accessibilityACM Classification KeywordsH.5.m [Information interfaces & presentation (HCI)]: Misc.IntroductionAccessibility research aims to create technologies that ben-efit people with disabilities. These technologies can rangefrom prosthesis for amputees to digital tools to increase dis-play contrast for people with low vision. Often the goal isto alleviate challenges brought on due to the impairmentand achieve equality to people with typical abilities. In somecases, however, the tools that are created to deal with animpairment allow the people that use them to exceed typi-cal abilities, or potentially provide a benefit to people of allabilities.Neil Harbisson, who was born with total colour blindness,uses a device that records colour images in front of him andturn them into vibrations in his skull allowing him to ”hear“the colours in the image. The device is set up in a way thatallows him to not only see what is in front of him but alsoreceive images from other sources, i.e., pictures sent to himfrom people in other locations [11], thus in a way expandinghis perceptual capabilities beyond those of a typical person.An example of accessible technology that helps everyoneare design guidelines for websites that address colours thatare hard to differentiate for people with colour vision defi-ciencies and contrast for text which can be hard to read forpeople with low vision. While initially intended to alleviatechallenges arising from impairments, these guidelines re-sult in website designs that benefit everyone viewing thewebsite through clearer colours and increased contrast [18].The remainder of the paper will present digital assistive sys-tems that deal with perceptual issues and show how frominvestigating (situational) disabilities we can develop tech-nologies, tools, or design guidelines that can benefit andenhance everyone’s perception. To do this, we presentthree case studies that showcase our results for (1) im-paired colour vision, (2) situational visual impairments, and(3) situational hearing impairment.Case Study 1: Impaired Colour VisionFigure 1: Examples of colouridentification techniques for thetask “select the yellow pencil”. Topleft to bottom right: typical vision,impaired colour vision (ICV)simulation showing difficulty, ourColourNames, ColourMeters, andColourPopper technique.Colour vision is a vital component of day-to-day living; itsloss can severely limit a person’s livelihood (e.g., airplanepilots, electricians, dentists, and visual artists all requireaccurate colour vision), threaten health and safety (e.g.,not spotting sunburns and rashes, misidentifying medica-tion), make food preparation dangerous (e.g., undercookingmeat, detecting mouldiness), and even challenge some-one’s social acceptability (e.g., by inhibiting clothing coordi-nation and home decorating) [3].To help address the challenges imposed by impaired colourvision (ICV), we developed the concept of Situation-SpecificModelling (SSM) and used it to construct a model of colourperception tunable to an individual’s unique abilities andenvironment [5, 6, 7]. We also developed a recolouring toolfor improving web colour differentiability for people with ICVthat was designed to maintain the subjective experience ofthe original colour scheme (e.g., by keeping ‘warm’ colourschemes ‘warm’) while improving colour accessibility [8].Next, we went beyond recolouring to focus on improvingcolour identification and discrimination. We developed re-visualization techniques for mapping colour informationto other visual properties (ColourID) [4], which improvedcolour identification to nearly 100% for participants withICV (see Figure 1 for examples of the techniques). We alsoinvestigated the use of eye tracking based manipulationsof colours to exploit with the eventual aim to make coloursmore differentiable to the observer [15].Every technique we have developed has the characteristicof enabling people to perceive visual properties that theytypically cannot (i.e., we make the invisible visible – in thiscase invisible colour). However, the same techniques canbe used to let people with typical abilities ‘see the invisible’.Next, we plan to extend our techniques to give people theability to perceive light outside of the visible spectrum: in-frared and ultraviolet. We will apply the same techniquesthat work for people with ICV to enable all people to per-ceive information they normally wouldn’t be able to see atall. This has the potential to greatly improve health (e.g., byenabling rapid visual triage for fever in epidemic situations)and safety (e.g., giving airport security the ability to ‘see’emitted heat).Case Study 2: Situational Visual ImpairmentA situational impairment is when a person experiences diffi-culty completing tasks that would not typically present prob-lems (e.g., the difference when typing a message while sta-tionary vs. running, or reading a message on a display of amobile device on an overcast vs. sunny day). Various fac-tors can cause a situational impairment, and it is not limitedto environmental conditions. This concept within computinghas been discussed by Newell [16] and Sears et al. [17].The portability and computing power of mobile devices hasextended our freedom for completing tasks at our conve-nience. However, with this freedom there are substantialissues; situational contexts can become difficult to con-trol and can also be unpredictable. We focus specificallyon situational visual impairment (SVI) when using mobiledevices, which describes situations in which it becomesdifficult to see content on the mobile device when it wouldotherwise not be an issue (e.g., bright light reflecting on thedisplay and low contrast text as demonstrated in Figure 2).Mobile screens are used in diverse situations and for di-verse purposes, e.g., car navigation, in the medical industry,and education. Currently, we are looking at the SVIs expe-rienced when using mobile devices under bright lighting.Although it is well documented that mobile screens typicallydegrade in quality as ambient light increase (e.g., [13, 14]),we do not know what tasks people are doing on their mobiledevices when experiencing SVI, what other factors are in-volved that can exacerbate the problem, and what solutionspeople currently employ to overcome the problem.Figure 2: A smartphone in abrightly lit room, with half thedevice in shade. The low contrastmenu text at the bottom of thescreen becomes difficult to read.Figure 3: Our speechreadingacquisition framework, with twodimensions: Type of Skill andAmount of Information, each splitinto three levels(Analytic/Hybrid /Synthetic andLow /Medium/High, respectively)Figure 4: PhonemeViz mockupviewed through Epson Moverioglasses for the word ”bat“.We predict that there are different solutions required to re-duce the occurrence of SVI since it is likely to involve mul-tiple factors at any one time. For example, if the contentbeing viewed has a low contrast design then it will becomeeven harder to read on a display in direct sunlight. One so-lution would be to provide designers with a tool that helpsthem assess the robustness of their design’s perceptibilityunder increasing ambient light, similar to an accessibilitytool that we have previously developed to allow web de-signers to select a colour scheme for people with ICV, whilemaintaining the designers’ creative freedom [19].Solving this issue of situational visual impairment would bea benefit for everyone who is using mobile devices whereoverly bright conditions occur.Case Study 3: Situational Hearing ImpairmentThe WHO estimates that on a global scale 360 million peo-ple (~5%) worldwide have disabling hearing loss1 [21].Hearing loss is an everyday problem as it interferes withconversations by making it difficult to understanding whatothers are saying [20].People who are deaf or have a hearing loss find that speechread-ing (SR) (commonly called lipreading) can overcome manyof the barriers when communicating with others [2]. SRhelps to improve conversational confidence (thereby reduc-ing social isolation), enhance employability, and improveeducational outcomes [20].However, SR is a skill that takes considerable practice andtraining to acquire [12]. One reason for this is caused bynearly identical mouth shapes (visemes) producing severalspeech sounds (phonemes); there is not a one-to-manymapping from visemes to phonemes. This decreases com-prehension and causes confusion and frustration duringconversations. Publicly-funded SR classes are sometimesprovided and have been shown to improve SR acquisi-tion [1]. However, classes are only provided in few countries1Hearing loss of more than 40 dB in the better hearing ear in adultsand more than 30 dB in the better hearing ear in children.around the world; there is an insufficient number of classesrunning in areas in which they are provided (e.g., only 50 ofan estimated 325 required classes are currently running inScotland [1]) and classes require mobility to attend.To help expand SR training, we developed a novel frame-work [10](Figure 3) that can be used to develop Speech-reading Acquisition Tools (SATs) – a new type of technologydesigned specifically to improve SR acquisition. Throughthe development and release of SATs, people with hear-ing loss will be able to augment their class-based learningor learn on their own if no suitable classes are available.These SATs would also benefit those who are experiencinga situational hearing impairment.PhonemeViz is a SAT we have developed using our frame-work. It places the character(s) of the most recently spo-ken initial phoneme just to the side of a speaker’s lips.This design should enable a speechreader to focus onthe speaker’s eyes and lip movements (as in traditionalspeechreading), while also monitoring changes in Phone-meViz’s state using their peripheral vision to help disam-biguate confusing visemes. PhonemeViz can be overlaidonto video or displayed on a transparent head mounted dis-play (Figure 4, left) to augment natural speechreading andenhance speechreading acquisition. In a pilot evaluation ofa PhonemeViz prototype [9] we found it enabled all partici-pants to achieve 100% word recognition.When people encounter listening issues due to noise orother factors in their environment this constitutes a situ-ational hearing impairment (e.g., in a noisy restaurant oron a construction site). Through the use of SATs, such asPhonemeViz, people who experience situational hearingimpairment can then rely on SR in these challenging situa-tions to facilitate communication.ConclusionWith our case studies, we have shown how we can developperceptual augmentations from research on assistive sys-tems that target disabilities. From research on impairedcolour vision, we have arrived at techniques that allow theperception of colour and information outside of the capabili-ties of the human visual system. From looking at situationalvisual impairments, we will develop tools and guidelinesthat can make content on mobile devices easier to perceive.Finally, from investigating hearing impairment, we have ar-rived at tools that can help people in situations that cancause situational hearing impairment, e.g., noisy environ-ments. While initially each case study focuses on a specificaccessibility problem, the results can be generalised andapplied to the benefit of everyone.REFERENCES1. L Armstrong. 2015. On everybody’s lips.http://www.scotlipreading.org.uk/files/1914/2686/1587/On_everybodys_lips_-_report.pdf.(2015).2. Ruth Campbell, Barbara Dodd, and Denis K Burnham.1998. Hearing by eye II. Vol. 2. Psychology Press.3. Barry L Cole. 2004. The Handicap of Abnormal ColourVision. Clinical and Experimental Optometry (2004).4. David R Flatla, Alan R Andrade, Ross D Teviotdale,Dylan L Knowles, and Craig Stewart. 2015. ColourID:Improving Colour Identification for People with ImpairedColour Vision. In CHI ’15.5. David R Flatla and Carl Gutwin. Individual models ofcolor differentiation to improve interpretability ofinformation visualization. In CHI’10.6. David R Flatla and Carl Gutwin. Situation-specificmodels of color differentiation. In TACCESS’12.7. David R Flatla and Carl Gutwin. So that’s what you see:building understanding with personalized simulations ofcolour vision deficiency. In SIGACCESS’12.8. David R Flatla, Katharina Reinecke, Carl Gutwin, andKrzysztof Z Gajos. SPRWeb: Preserving subjectiveresponses to website colour schemes throughautomatic recolouring. In CHI’13.9. Benjamin M. Gorman. Reducing viseme confusion inspeech-reading. In SIGACCESS’16.10. Benjamin M. Gorman and David R. Flatla. AFramework for Speechreading Acquisition Tools. InCHI’17.11. Neil Harbisson. 2012. I listen to color. TEDXTalk,http://www. ted.com/talks/neil_harbisson_i_listen_to_color (2012).12. Harriet Kaplan, Scott J Bally, and Carol Garretson.1985. Speechreading: A way to improveunderstanding. Gallaudet University Press.13. Youn Jin Kim, M Ronnier Luo, Wonhee Choe,Hong Suk Kim, Seung Ok Park, Yeseul Baek, PeterRhodes, Seongdeok Lee, and Chang Yeong Kim. 2008.Factors affecting the psychophysical image qualityevaluation of mobile phone displays: the case oftransmissive liquid-crystal displays. JOSA A (2008).14. Peter Liu, Fahad Zafar, and Aldo Badano. 2014. Theeffect of ambient illumination on handheld displayimage quality. Journal of digital imaging (2014).15. Michael Mauderer, David R. Flatla, and Miguel A.Nacenta. Gaze-Contingent Manipulation of ColorPerception. In CHI’16.16. Alan F Newell. 1995. Extra-ordinary human-computerinteraction. In Extra-ordinary human-computerinteraction, Alistair D. N. Edwards (Ed.). CambridgeUniversity Press.17. Andrew Sears, Min Lin, Julie Jacko, and Yan Xiao.2003. When computers fade: Pervasive computing andsituationally-induced impairments and disabilities. InHCI International.18. John M. Slatin and Sharron Rush. 2002. MaximumAccessibility: Making Your Web Site More Usable forEveryone.19. Garreth W. Tigwell, David R. Flatla, and Neil D.Archibald. ACE: A Colour Palette Design Tool forBalancing Aesthetics and Accessibility. InTACCESS’17.20. Nancy Tye-Murray. 2014. Foundations of auralrehabilitation: Children, adults, and their familymembers. Nelson Education.21. WHO 2015. Deafness and hearing loss, Fact sheetN.300. http://www.who.int/mediacentre/factsheets/fs300/en/.(2015).

English

https://discovery.dundee.ac.uk/files/13257544/AugmentedPerception_CHI_Workshop.pdf

Beyond Accessibility:Lifting Perceptual Limitations for Everyone

Abstract

Similar works

Full text

Available Versions

University of Dundee Online Publications

University of Dundee Online Publications