Implementing a Wizard of Oz Tool for Augmented Reality

This thesis aims to explore Wizard of Oz testing in conjunction with Augmented Reality (AR) and focus has been put on testing AR with Head Mounted Displays. The recent increase of interest in HMDs with products such as MOD Live from Recon Instruments and Google's Project Glass puts new demands and possibilities on human-computer interaction. Since the commercial market for HMDs is still in its infancy the need to explore different design approaches is very much present. One way to conduct experiments on human-machine interaction is with the help of a Wizard of Oz tool. During the thesis we have developed such a tool to support designers in researching usability and interaction. The tool provides a user friendly framework to carry out user case studies focused on AR with HMDs. After input and feedback from stakeholders and experts we believe that, even though the tool is mainly meant to be used in conjunction with AR in HMDs, the tool can be applied to other areas as well

Improving digital object handoff using the space above the table

Object handoff – that is, passing an object or tool to another person – is an extremely common activity in collaborative tabletop work. On digital tables, object handoff is typically accomplished by sliding the object on the table surface – but surface-only interactions can be slow and error-prone, particularly when there are multiple people carrying out multiple handoffs. An alternative approach is to use the space above the table for object handoff; this provides more room to move, but requires above-surface tracking. I developed two above-the-surface handoff techniques that use simple and inexpensive tracking: a force-field technique that uses a depth camera to determine hand proximity, and an electromagnetic-field technique called ElectroTouch that provides positive indication when people touch hands over the table. These new techniques were compared to three kinds of existing surface-only handoff (sliding, flicking, and surface-only Force-Fields). The study showed that the above-surface techniques significantly improved both speed and accuracy, and that ElectroTouch was the best technique overall. Also, as object interactions are moved above-the-surface of the table the representation of off-table objects becomes crucial. To address the issue of off-table digital object representation several object designs were created an evaluated. The result of the present research provides designers with practical new techniques for substantially increasing performance and interaction richness on digital tables

Development and evaluation of a smartphone-based system for inspection of road maintenance work

Abstract. In the road construction industry, doing work inspection is a laborious and resource-consuming job because of the distributed work site. Contractors in Finland require to capture photos of every road fix they have done as proof of their work. It is well-established that with the help of smartphone technology, these kinds of manual work can be reduced. This thesis aims to develop and evaluate a smartphone-based system to capture video evidence of task completion. The system, designed and developed in this thesis, consists of an Android application named ’Road Recorder’ and a web tool for managing the content collected by Road Recorder. While mounted to a vehicle’s dashboard used in construction work, the Road Recorder can record the videos of road surface and geo-location information and some other metadata and send them to a remote server that is inspected using the web tool. Users of different backgrounds were given the system to accomplish some tasks and were observed closely. The users were interviewed at the end, and responses were analyzed to find the usability of the applications. The results indicate the high usability of the Road Recorder application and reveal possible improvements for the Road Recorder management web application. Overall, Road Recorder is a great step towards the automation of such construction work inspection. Though there were some limitations in the evaluation process, it demonstrates that Road Recorder is easy to use and can be a useful tool in the industry

'탑승자'의 관점의 시간, 위치 기반 차량 클러스터 UI 디자인 프레임 제안 연구

학위논문(석사)--서울대학교 대학원 :미술대학 디자인학부 디자인전공,2019. 8. 정의철.One important design issue is the examination of how the user interface (UI) supports the new user role in future mobility. However, there are few design studies on the passengers cognitive needs and behavior in Autonomous Vehicles (AVs) based on empirical data. There is no doubt that autonomous mobility technologies are growing. The technology is already aiding the driving experience, and it will change the mobility culture and the transition of driver into passenger. This study is based on the premise that future AV is capable of performing all driving tasks. It proposes a set of passenger-centered automotive cluster UI designs for future mobility employing two factors: time and path. A set of empirical data is provided to understand the passengers perspective. In this study, a solid set of empirical data on the cognitive needs of passengers is collected. Human cognitive characteristics and driving tasks are investigated from various viewpoints to understand the passengers iii perspective. The cognitive relationship in the driving environment is analyzed through a literature review on situation awareness (SA) and structuring of the data flow framework. The framework is further explored by connecting the technological role transformation to the passenger. To construct the empirical database on the passenger, three sets of user tests and in-depth interviews were undertaken. The user tests were designed employing the Wizard of Oz method, and the results were summarized using descriptive and exploratory analysis. Based on these insights, a set of UI designs from the perspective of the passenger was proposed, and usability tests were conducted to verify its effectiveness and usability. The results of the tests demonstrate that a major percentage of the information request was related to time (current time and duration) and path (vehicle location and surroundings). Based on the data, a UI framework was built. Two usage scenarios were designed, time-full and time-less, for better in-situation comprehension. Time- and path-based UI were proposed to flow with the scenarios. A usability test was conducted, and a passengers cognitive framework was defined. There are two aspects to this study: the data flow frameworks of the driver/passenger, and the UI design proposal. Situational precision from the perspective of the driver was analyzed to understand the relationship between the user, the vehicle and the road conditions. Further, the cognitive framework of the passenger was proposed based on the data. This study provides a solid understanding of drivers emerging needs when they are relieved of the cognitive burden of driving tasks. The UI features for AV are introduced based on the empirical data and research related to the provision of better situation awareness, focusing on time and location. This study contributes to the extant literature by observing the iv perspective of passengers in Autonomous vehicles based on a qualitative study. The proposed UI design will be further explored as a communication method between the system and the passive user in future mobility.사용자 인터페이스가 (UI) 미래 이동성에서 새로운 사용자 역할을 지지하는 디자인 도출은 미래 이동성 분야에서 중요한 디자인 이슈이다. 그러나 사용자 실험에 근거하여 자율주행차량 (AV) 의 탑승자인지 욕구와 행동에 대한 디자인 연구는 미미하다. 자율주행이 기술의 발전과 그 영역은 점차 넓어지고 있다. 해당 기술은 이미 운전 환경에 적용되고 있으며, 이로 인해 미래 이동문화에서 사용자의 역할은 '운전자'에서 '탑승자'로 변화한다. 본 연구는 미래 자율주행차량이 모든 운전 상황에 대처할 수 있다는 것을 전제로 한다. 사용자 실험을 통해 탑승자의 관점에 대한 분석을 진행하였고, 이를 기반으로 미래 모빌리티 환경에 적용될 사용자 인터페이스를 제안한다. 제안된 디자인은 운전자 중심의 상황인지에서 벗어나 탑승자 중심 인지 정보 요소를 분석하였고, 시간과 경로 두 가지 요소를 강조한 UI 를 제안한다. 본 연구에서 탑승자의 인지 정보 요구에 대한 실험적 데이터를 수집하였다. 탑승자의 관점을 이해하기 위해 다양한 관점에서 인간의 인지적 특성 및 운전 태스크를 관찰하였고, 상황인지 (SA) 에 관한 문헌 연구와 데이터 프레임워크 구조화를 통해 운전 환경에서 발생하는 인지적 요소 관계를 분석하였다. 제안된 프레임워크는 기술 변화에 따라 운전자가 탑승자로 변화되었을 때 운전 환경에서의 데이터 관계 변화를 시각적으로 구조화하여 심층적으로 탐구되었다. 탑승자의 인지 니즈 대한 실험적 데이터베이스를 수집하기 위해 총 3 세트의 유저 테스트와 심층 인터뷰가 수반되었다. 유저 테스트는 Wizard of Oz 방법을 사용하여 설계되었으며 실험 결과는 질적 연구방법론의 분석 방법을 통해 분석되었다. 실험을 통해 얻은 인사이트를 바탕으로 탑승자 관점에서 UI 디자인을 제안하고 사용성 테스트를 통해 효율성과 유용성을 5 점 리 커트 스케일로써 검증하였다. 실험 결과에 따르면 탑승자가 요청한 인지 정보는 시간 (현재 시각 및 기간)과 경로 (차량 위치 및 주변 환경)에 집중된 것을 관찰할 수 있었다. 이와 같은 데이터를 기반으로 UI 프레임워크를 구성하였다. 상황 속의 사용례를 제시하기 위하여도 가지 time-full 과 time-less 의 사용 시나리오를 구축하고, 제안된 시나리오에 따라 시간과 위치에 기반한 UI 를 제안하였다. 제안된 UI 에 대한 사용성 테스트를 진행하였고, 탑승자 관점에서의 운전상황 인지 워크 프레임을 완성하였다. 본 연구의 가치는 두 가지로 정리될 수 있다. 하나는 운전자 / 탑승자의 데이터 플로우 프레임워크를 제안하였다는 것과 두 번째는 탑승자의 관점을 지지하는 UI 디자인 제안에 있다. 운전자의 관점에서의 운전 상황을 분석하여 사용자, 차량, 그리고 도로 상태 간의 관계를 시각화하였고, 이는 탑승자인지 플로우 프레임워크를 제안하는데 기조적인 틀로써 사용되었다. 본 연구는 운전 태스크를 수행하는 데에 필요했던 인지 부담에서 벗어났을 때의 운전자가 필요로 하는 복합적인 니즈에 대해 관찰하고 미래 모빌리티 환경에 적합한 UI 의 디자인 요소에 대한 연구논문이다. 미래 자율주행차량 안의 사용자 인터페이스가 갖추어야 하는 요소를 실험적 데이터에 근거하여 제시하며, 시간과 루트를 강조하여 향상된 상황 인지를 제공하는 방법에 대한 심도있는 관찰을 기록한다. 본 연구는 질적 연구에 기초한 자율 차량의 탑승자 관점을 관찰함으로써 기존 자율주행이 디자인 연구에 기여할 것이다. 제안된 UI 디자인 미래 이동 성안에서 시스템과 탑승자 간의 커뮤니케이션 방법에 대한 연구로써 그 의의가 있다.ABSTRACT ...................................................................................................................... II CHAPTER 1. INTRODUCTION......................................................................................... １ 1.1. BACKGROUND ..............................................................................................................１ 1.2. PURPOSE .....................................................................................................................７ 1.3. RESEARCH QUESTION.....................................................................................................８ CHAPTER 2. LITERATURE REVIEW ..............................................................................１１ 2.1. SITATION AWARENESS (SA) ........................................................................................１１ 2.2. HUMAN INFORMATION PROCESSING MODEL..................................................................１５ 2.3. DRIVING SITUATION AWARENESS AND PERSPECTIVE.........................................................２０ 2.4. DRIVING TASK AND SENSORY INTERACTION ....................................................................２２ CHAPTER 3. COGNITIVE NEEDS IN AUTONOMOUS.....................................................２７ 3.1. DRIVING BEHAVIOR TRANSFORMATION AND CLUSTER UI..................................................２７ 3.2. COGNITIVE FRAMEWORK TRANSFORMATION ..................................................................３３ CHAPTER 4. USER TESTS ............................................................................................３６ 4.1. WIZARD OF OZ PROTOTYPING .....................................................................................３８ 4.2. PILOT TEST 1............................................................................................................４０ 4.2.1. Experiment Design & Laboratory Setting.................................................４０ 4.2.2. Persona Scenario & Task Design ..............................................................４２ 4.2.3. Preparation of Driving situation...............................................................４５ 4.2.4. Procedure.................................................................................................４７ 4.2.5. Data Analysis & Insight............................................................................４８ 4.3. PILOT TEST 2............................................................................................................５１ 4.3.1. Amendment: Experiment Design & Laboratory Setting ...........................５２ 4.3.2. Amendment: Task Scenario & Command Cue..........................................５４ 4.3.3. Amendment: Perform Role and preparation of driving situation ............５７ 4.3.4. Amendment: Procedure ...........................................................................５９ 4.3.5. Data Analysis & Insight............................................................................６２ 4.4. MAIN TEST ..............................................................................................................６５ 4.4.1. Experiment Design & Laboratory setting .................................................６６ 4.4.2. Task Design ..............................................................................................６９ 4.4.3. Procedure.................................................................................................７１ 4.4.4. Result Analysis & Insight..........................................................................７４ CHAPTER 5. UI CONCEPT DEVELOPMENT...................................................................８１ 5.1. UI DESIGN METHOD..................................................................................................８１ 5.2. DESIGN PROPOSAL ....................................................................................................８４ 5.3. USER SCENARIOS ......................................................................................................８６ 5.3.1 Scenario 1. Time-less: Late for a morning meeting..................................８６ 5.3.2 Scenario 2.Time-full: Leisure driving on weekends ..................................９３ CHAPTER 6. USABILITY TEST ......................................................................................９８ 6.1. USABILITY TEST GUIDE ...............................................................................................９８ 6.2. ASSESSMENT USABILITY TEST ..................................................................................１００ 6.2.1 Test planning........................................................................................１００ 6.2.2 Laboratory setting................................................................................１０２ 6.2.3 Test conduct and debriefing.................................................................１０６ 6.3. RESULT ANALYSIS ..................................................................................................１０６ CHAPTER 7. CONCLUSION......................................................................................１０７ APPENDIX 1...........................................................................................................１１０ APPENDIX 2...........................................................................................................１１１ APPENDIX 3...........................................................................................................１１３ APPENDIX 4...........................................................................................................１２１ APPENDIX 5...........................................................................................................１２４ APPENDIX 6...........................................................................................................１２８ APPENDIX 7...........................................................................................................１３３ BIBLIOGRAPHY ......................................................................................................１３６ 국문 초록 ............................................................................................................１４３Maste

Application of Machine Learning within Visual Content Production

We are living in an era where digital content is being produced at a dazzling pace. The heterogeneity of contents and contexts is so varied that a numerous amount of applications have been created to respond to people and market demands. The visual content production pipeline is the generalisation of the process that allows a content editor to create and evaluate their product, such as a video, an image, a 3D model, etc. Such data is then displayed on one or more devices such as TVs, PC monitors, virtual reality head-mounted displays, tablets, mobiles, or even smartwatches. Content creation can be simple as clicking a button to film a video and then share it into a social network, or complex as managing a dense user interface full of parameters by using keyboard and mouse to generate a realistic 3D model for a VR game. In this second example, such sophistication results in a steep learning curve for beginner-level users. In contrast, expert users regularly need to refine their skills via expensive lessons, time-consuming tutorials, or experience. Thus, user interaction plays an essential role in the diffusion of content creation software, primarily when it is targeted to untrained people. In particular, with the fast spread of virtual reality devices into the consumer market, new opportunities for designing reliable and intuitive interfaces have been created. Such new interactions need to take a step beyond the point and click interaction typical of the 2D desktop environment. The interactions need to be smart, intuitive and reliable, to interpret 3D gestures and therefore, more accurate algorithms are needed to recognise patterns. In recent years, machine learning and in particular deep learning have achieved outstanding results in many branches of computer science, such as computer graphics and human-computer interface, outperforming algorithms that were considered state of the art, however, there are only fleeting efforts to translate this into virtual reality. In this thesis, we seek to apply and take advantage of deep learning models to two different content production pipeline areas embracing the following subjects of interest: advanced methods for user interaction and visual quality assessment. First, we focus on 3D sketching to retrieve models from an extensive database of complex geometries and textures, while the user is immersed in a virtual environment. We explore both 2D and 3D strokes as tools for model retrieval in VR. Therefore, we implement a novel system for improving accuracy in searching for a 3D model. We contribute an efficient method to describe models through 3D sketch via an iterative descriptor generation, focusing both on accuracy and user experience. To evaluate it, we design a user study to compare different interactions for sketch generation. Second, we explore the combination of sketch input and vocal description to correct and fine-tune the search for 3D models in a database containing fine-grained variation. We analyse sketch and speech queries, identifying a way to incorporate both of them into our system's interaction loop. Third, in the context of the visual content production pipeline, we present a detailed study of visual metrics. We propose a novel method for detecting rendering-based artefacts in images. It exploits analogous deep learning algorithms used when extracting features from sketches

An educational game to teach children about air quality using augmented reality and tangible interaction with sensors

Air pollution is known to be one of the main causes of injuries to the respiratory system and even premature death. Gases, particles, and biological compounds affect not only the air we breathe outdoors, but also indoors. Children are highly affected by the poor quality of the air they breathe because their organs and immune systems are still in the developmental stages. To contribute to raising children’s awareness to these concerns, this article presents the design, implementation, and experimental validation of an serious augmented reality game for children to playfully learn about air quality by interacting with physical sensor nodes. The game presents visual representations of the pollutants measured by the sensor node, rendering tangible the invisible. Causal knowledge is elicited by stimulating the children to expose real-life objects (e.g., candles) to the sensor node. The playful experience is amplified by letting children play in pairs. The game was evaluated using the Wizard of Oz method in a sample of 27 children aged between 7 and 11 years. The results show that the proposed game, in addition to improving children’s knowledge about indoor air pollution, is also perceived by them as easy to use and a useful learning tool that they would like to continue using, even in other educational contexts.info:eu-repo/semantics/publishedVersio

Introducing a Socio-Technical Perspective on Digital Competence Education Through Co-design

Digital competence and programming have been part of the Swedish school curricula since 2018. This paper demonstrates how co-design design activities can be conducted as a way to provide digital competence education from a socio-technical perspective. Such activities were conducted with novice designers in three sessions, each with two school classes, from a Swedish engineering upper secondary school program in 2019. The students gained hands-on experience with interaction design and co-design as they designed a mobile application using a Wizard-of-Oz prototyping system and evaluated the prototype with users. This trial is used to argue that time allocation for socio-technical perspectives on digital competence in the Swedish school curricula can be expanded to be able to provide students with a more holistic view of digitization beyond technical issues