A workbook documenting the development of prototypes exploring a voice enabled internet

Clarke, Loraine

Rogers, Jon

Skelly, Martin

Taylor, Nick

Thomas, Peter

English

University of Dundee Online Publications

                                                                    University of DundeePrototyping a Voice Enabled InternetSkelly, Martin; Rogers, Jon; Clarke, Loraine; Thomas, Peter; Taylor, NickPublication date:2017Document VersionPublisher's PDF, also known as Version of recordLink to publication in Discovery Research PortalCitation for published version (APA):Skelly, M., Rogers, J., Clarke, L., Thomas, P., & Taylor, N. (2017, Jun). Prototyping a Voice Enabled Internet: apractitioner's guide . Mozilla Foundation.General rightsCopyright and moral rights for the publications made accessible in Discovery Research Portal are retained by the authors and/or othercopyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated withthese rights. • Users may download and print one copy of any publication from Discovery Research Portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain. • You may freely distribute the URL identifying the publication in the public portal.Take down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Download date: 28. Dec. 2021PROTOTYPING  A VOICE ENABLED INTERNETA WORKBOOK DOCUMENTING THE DEVELOPMENT OF PROTOTYPES EXPLORING  A VOICE ENABLED INTERNETThe Open IoT Studio is a collaboration between the Mozilla Foundation and the University of Dundee.The studio seeks to advance responsible open IoT through professional practices and a network of IoT practitioners who conduct research, make prototypes and build meaningful collaborations.Mozilla All Hands June, 2017TOWARDS A VOICE ENABLED INTERNETIoT amplifies major Internet health issues such as decentralisation, trust, privacy and security. Voice is already a leading category of connected devices in people’s homes, with approximately 11,000,000* pairs of Alexa’s ears listening in peoples bedrooms, kitchens and living rooms right now.‘Always on’ is becoming the accepted social norm, but what do people expect from this new category of connected product? The market is focused on domestic speakers with cloud artificial intelligence, but the future of integrating connected microphones and voice services into other product categories, including public spaces, has huge potential to change society. In this project, we explore how product design principles, such as form, materiality, intuition and design can be used to create an alternative voice future - one of interoperability, transparency, and ethical values.always on ubiquitousinvisible designclosed loop systems little interoperability  transparencysecurity* http://www.seattletimes.com/business/amazon/amazon-has-sold-more-than-11-million-echo-devices-morgan-stanley-says/PROVOCATIVE PROTOTYPINGProvocative prototyping is the creation of objects or artefacts that ask questions, explore design issues and enable open discussion around a topic.The purpose of these artefacts is to raise awareness of potential issues, encourage debate and enable change by empowering people to form a point of view.Provocative prototypes don’t aim to create a market ready solution, solve engineering challenges or become robust technical working ‘products’. In this project we have created research prototypes that deliberately exaggerate and draw attention to issues around IoT and a voice enabled internet but this methodology can be used for all sorts of business challenges.These objects were designed to be used in a series of facilitated workshops with members of the public as well as among professional designers and engineers to debate the ethics and current direction of the voice enabled internet.IDEA GENERATIONWe began by sketching out three prototype ideas. Each was kept separate to allow flexibility when facilitating the workshop and allow for the possibility of digging deeper into one area with participants. These ideas focused on the three design issues that were identified; lack of interoperability, invisible data transfer and de-mystifying the complexity of artificial intelligence systems.The AgentThe ActorThe AdvisorAgent acts as a translator between a user and different voice platforms. It keeps you in control of your own voice data, and the valuable deep learning stays on your local device. Your agent acts on your behalf in interactions with the siloed voice platforms of other companies. It can be used to mask or change your voice when speaking with conversational interfaces to stop them gaining data that you would prefer not to offer. Actor will explore how a product’s form, interactions and material properties can shape our experiences when speaking with machines. What does listening look like? We all know a good listener, but how can we embody human listening cues in the behaviour  of conversational interfaces? Advisor raises questions around ethics, inclusion, localisation & diversity issues within voice and IoT. Advisor encourages visitors to record labelled speech data into the object and displays the percentage accuracy of typical services. It also poses ethical questions around voice and allows users to voice their opinions.THEMESWhen presenting the sketch ideas to project stakeholders we realised that the ambiguity of the concepts was causing problems. The prototype directions were deliberately open and they didn’t answer many of the questions that they posed. The sketches were starting points for concept development and the prototyping stage would force us to make decisions about prototype functionality and define the concepts fully.We knew at this point that we had to demonstrate the functionality rather than trying to explain the objects with on screen presentations, so we began prototyping the interactions. NOTESTECHNOLOGY PROTOTYPINGStep one was to create a basic breadboard circuit of each concept and work out technically what was feasible in a rapid prototype with off the shelf components. A low fidelity card model was then created to test the functionality of the ideas. For example, a cardboard version of the Agent’s Alexa hat was made to test a light sensor positioned on the light ring. Speed is key at this stage and it’s important not to get hung up on creating complex features if they detract from the overall experience of the prototype. It’s not important to nail all of the possible features, but it is important to nail the core functionality so people can believe that the object could exist.Once each idea had the basic functionality working on a breadboard, it was time to explore creating the 3D design and a look and feel for each object.It was important that the three research prototypes were separate to allow exploration of the three design issues but had a shared look and feel. We created a common design language to act as brand guidelines for the three physical objects. We wanted the prototypes to feel believable, but not so finished that they didn’t have a prototype aesthetic. We also had to utilise readily available materials and making processes to keep it simple and allow them to be produced in house. We decided to use readily available, familiar domestic materials for example fabrics and woods with some 3D printed highlights. A single colour, orange, was also selected and used to tie each object together visually. PROTOTYPE DEVELOPMENTTHE AGENTThe Agent prototype works with the Amazon Alexa and can detect when the speaker hears a wake word.  It displays further information about what happens behind the scenes when voice assistants share data with the cloud.  “I’m the Agent. I can translate and speak on your behalf  to commercial voice platforms“  “Your voice is being recorded and sent to the companies  cloud for processing““It takes 10,000 hours of recorded speech to train a voice  assistant““Nearly 11,000,000 devices are listening to conversations  right now using Amazon Alexa’s technology““It’s hard to make different voice assistants talk to each  other, since many companies close their systems to keep  out competitors““Alexa is now active listening“THE ACTORThe actor prototype explores a transparent visual language for machines to show when and how they are listening. Each of the cones contain a microphone, and when an increase in volume is detected, the top part of the actor rotates to demonstrate it’s listening to you. The orange stick nods in time with what you are saying, giving a mirroring personality to the object. Both behaviours are a bit clumsy at this stage.THE ADVISORThe advisor prototype poses ethical dilemmas through a question and voting system. It asks participants ethical questions around internet enabled voice and tells you how many people have responded in the same way.NOTES NOTESTESTING THE PROTOTYPESWe ran a workshop at Dundee Design Festival in June 2017 to explore these design issues with around 15 participants. The workshop was split into three sections and each prototype informed a section of the workshop. The theme of the festival was ‘factory floor’ so we emphasised exploration of the value of personal data, and debated the issues around it being tracked, filtered and sold on. NOTES NOTESPROVOCATIVE PROTOTYPINGThroughout the project, eight principles of provocative prototyping emerged. ‘the function of the object should leave a little room for imagination’ ‘the object should suspend disbelief that it could exist, but should not look so similar to market solutions that it is accepted’ ‘weirdness should be encouraged’ ‘humour is useful in opening up design questions’‘objects should leave a bit of mystery and ambiguity about their purpose’ ‘objects will usually require facilitation and discussion to support useful findings’ ‘objects should encourage the questioning of norms’‘the technology should be focused on the core experience, and extra features should be stripped back’ 

Prototyping a Voice Enabled Internet: a practitioner's guide

University of St. Andrews - Pure

                                                                    University of DundeePrototyping a Voice Enabled InternetSkelly, Martin; Rogers, Jon; Clarke, Loraine; Thomas, Peter; Taylor, NickPublication date:2017Document VersionPublisher's PDF, also known as Version of recordLink to publication in Discovery Research PortalCitation for published version (APA):Skelly, M., Rogers, J., Clarke, L., Thomas, P., & Taylor, N. (2017, Jun). Prototyping a Voice Enabled Internet: apractitioner's guide . Mozilla Foundation.General rightsCopyright and moral rights for the publications made accessible in Discovery Research Portal are retained by the authors and/or othercopyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated withthese rights. • Users may download and print one copy of any publication from Discovery Research Portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain. • You may freely distribute the URL identifying the publication in the public portal.Take down policyIf you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediatelyand investigate your claim.Download date: 31. Jul. 2023PROTOTYPING  A VOICE ENABLED INTERNETA WORKBOOK DOCUMENTING THE DEVELOPMENT OF PROTOTYPES EXPLORING  A VOICE ENABLED INTERNETThe Open IoT Studio is a collaboration between the Mozilla Foundation and the University of Dundee.The studio seeks to advance responsible open IoT through professional practices and a network of IoT practitioners who conduct research, make prototypes and build meaningful collaborations.Mozilla All Hands June, 2017TOWARDS A VOICE ENABLED INTERNETIoT amplifies major Internet health issues such as decentralisation, trust, privacy and security. Voice is already a leading category of connected devices in people’s homes, with approximately 11,000,000* pairs of Alexa’s ears listening in peoples bedrooms, kitchens and living rooms right now.‘Always on’ is becoming the accepted social norm, but what do people expect from this new category of connected product? The market is focused on domestic speakers with cloud artificial intelligence, but the future of integrating connected microphones and voice services into other product categories, including public spaces, has huge potential to change society. In this project, we explore how product design principles, such as form, materiality, intuition and design can be used to create an alternative voice future - one of interoperability, transparency, and ethical values.always on ubiquitousinvisible designclosed loop systems little interoperability  transparencysecurity* http://www.seattletimes.com/business/amazon/amazon-has-sold-more-than-11-million-echo-devices-morgan-stanley-says/PROVOCATIVE PROTOTYPINGProvocative prototyping is the creation of objects or artefacts that ask questions, explore design issues and enable open discussion around a topic.The purpose of these artefacts is to raise awareness of potential issues, encourage debate and enable change by empowering people to form a point of view.Provocative prototypes don’t aim to create a market ready solution, solve engineering challenges or become robust technical working ‘products’. In this project we have created research prototypes that deliberately exaggerate and draw attention to issues around IoT and a voice enabled internet but this methodology can be used for all sorts of business challenges.These objects were designed to be used in a series of facilitated workshops with members of the public as well as among professional designers and engineers to debate the ethics and current direction of the voice enabled internet.IDEA GENERATIONWe began by sketching out three prototype ideas. Each was kept separate to allow flexibility when facilitating the workshop and allow for the possibility of digging deeper into one area with participants. These ideas focused on the three design issues that were identified; lack of interoperability, invisible data transfer and de-mystifying the complexity of artificial intelligence systems.The AgentThe ActorThe AdvisorAgent acts as a translator between a user and different voice platforms. It keeps you in control of your own voice data, and the valuable deep learning stays on your local device. Your agent acts on your behalf in interactions with the siloed voice platforms of other companies. It can be used to mask or change your voice when speaking with conversational interfaces to stop them gaining data that you would prefer not to offer. Actor will explore how a product’s form, interactions and material properties can shape our experiences when speaking with machines. What does listening look like? We all know a good listener, but how can we embody human listening cues in the behaviour  of conversational interfaces? Advisor raises questions around ethics, inclusion, localisation & diversity issues within voice and IoT. Advisor encourages visitors to record labelled speech data into the object and displays the percentage accuracy of typical services. It also poses ethical questions around voice and allows users to voice their opinions.THEMESWhen presenting the sketch ideas to project stakeholders we realised that the ambiguity of the concepts was causing problems. The prototype directions were deliberately open and they didn’t answer many of the questions that they posed. The sketches were starting points for concept development and the prototyping stage would force us to make decisions about prototype functionality and define the concepts fully.We knew at this point that we had to demonstrate the functionality rather than trying to explain the objects with on screen presentations, so we began prototyping the interactions. NOTESTECHNOLOGY PROTOTYPINGStep one was to create a basic breadboard circuit of each concept and work out technically what was feasible in a rapid prototype with off the shelf components. A low fidelity card model was then created to test the functionality of the ideas. For example, a cardboard version of the Agent’s Alexa hat was made to test a light sensor positioned on the light ring. Speed is key at this stage and it’s important not to get hung up on creating complex features if they detract from the overall experience of the prototype. It’s not important to nail all of the possible features, but it is important to nail the core functionality so people can believe that the object could exist.Once each idea had the basic functionality working on a breadboard, it was time to explore creating the 3D design and a look and feel for each object.It was important that the three research prototypes were separate to allow exploration of the three design issues but had a shared look and feel. We created a common design language to act as brand guidelines for the three physical objects. We wanted the prototypes to feel believable, but not so finished that they didn’t have a prototype aesthetic. We also had to utilise readily available materials and making processes to keep it simple and allow them to be produced in house. We decided to use readily available, familiar domestic materials for example fabrics and woods with some 3D printed highlights. A single colour, orange, was also selected and used to tie each object together visually. PROTOTYPE DEVELOPMENTTHE AGENTThe Agent prototype works with the Amazon Alexa and can detect when the speaker hears a wake word.  It displays further information about what happens behind the scenes when voice assistants share data with the cloud.  “I’m the Agent. I can translate and speak on your behalf  to commercial voice platforms“  “Your voice is being recorded and sent to the companies  cloud for processing““It takes 10,000 hours of recorded speech to train a voice  assistant““Nearly 11,000,000 devices are listening to conversations  right now using Amazon Alexa’s technology““It’s hard to make different voice assistants talk to each  other, since many companies close their systems to keep  out competitors““Alexa is now active listening“THE ACTORThe actor prototype explores a transparent visual language for machines to show when and how they are listening. Each of the cones contain a microphone, and when an increase in volume is detected, the top part of the actor rotates to demonstrate it’s listening to you. The orange stick nods in time with what you are saying, giving a mirroring personality to the object. Both behaviours are a bit clumsy at this stage.THE ADVISORThe advisor prototype poses ethical dilemmas through a question and voting system. It asks participants ethical questions around internet enabled voice and tells you how many people have responded in the same way.NOTES NOTESTESTING THE PROTOTYPESWe ran a workshop at Dundee Design Festival in June 2017 to explore these design issues with around 15 participants. The workshop was split into three sections and each prototype informed a section of the workshop. The theme of the festival was ‘factory floor’ so we emphasised exploration of the value of personal data, and debated the issues around it being tracked, filtered and sold on. NOTES NOTESPROVOCATIVE PROTOTYPINGThroughout the project, eight principles of provocative prototyping emerged. ‘the function of the object should leave a little room for imagination’ ‘the object should suspend disbelief that it could exist, but should not look so similar to market solutions that it is accepted’ ‘weirdness should be encouraged’ ‘humour is useful in opening up design questions’‘objects should leave a bit of mystery and ambiguity about their purpose’ ‘objects will usually require facilitation and discussion to support useful findings’ ‘objects should encourage the questioning of norms’‘the technology should be focused on the core experience, and extra features should be stripped back’ 

https://discovery.dundee.ac.uk/files/48251290/voice_workbook_1_.pdf

Prototyping a Voice Enabled Internet: a practitioner's guide

Abstract

Similar works

Full text

Available Versions

University of Dundee Online Publications

University of St. Andrews - Pure

University of Dundee Online Publications

University of St. Andrews - Pure