Context-Aware Mobile Apps using iBeacons: Towards Smarter Interactions

In this paper we describe four mobile apps for iOS devices that use Bluetooth Low Energy iBeacons to provide contextual relevance and personalized experiences for the user. The applications span a number of vertical markets including asset tracking, food transportation logistics and health care. We developed these apps in collaboration with an industry partner located in Mississauga, Ontario, Canada. In this paper we present the relevant background of work in this area, the architectural framework that we designed and developed to support these context-aware apps, the apps themselves, and report on the findings of real use test case scenarios

'탑승자'의 관점의 시간, 위치 기반 차량 클러스터 UI 디자인 프레임 제안 연구

학위논문(석사)--서울대학교 대학원 :미술대학 디자인학부 디자인전공,2019. 8. 정의철.One important design issue is the examination of how the user interface (UI) supports the new user role in future mobility. However, there are few design studies on the passengers cognitive needs and behavior in Autonomous Vehicles (AVs) based on empirical data. There is no doubt that autonomous mobility technologies are growing. The technology is already aiding the driving experience, and it will change the mobility culture and the transition of driver into passenger. This study is based on the premise that future AV is capable of performing all driving tasks. It proposes a set of passenger-centered automotive cluster UI designs for future mobility employing two factors: time and path. A set of empirical data is provided to understand the passengers perspective. In this study, a solid set of empirical data on the cognitive needs of passengers is collected. Human cognitive characteristics and driving tasks are investigated from various viewpoints to understand the passengers iii perspective. The cognitive relationship in the driving environment is analyzed through a literature review on situation awareness (SA) and structuring of the data flow framework. The framework is further explored by connecting the technological role transformation to the passenger. To construct the empirical database on the passenger, three sets of user tests and in-depth interviews were undertaken. The user tests were designed employing the Wizard of Oz method, and the results were summarized using descriptive and exploratory analysis. Based on these insights, a set of UI designs from the perspective of the passenger was proposed, and usability tests were conducted to verify its effectiveness and usability. The results of the tests demonstrate that a major percentage of the information request was related to time (current time and duration) and path (vehicle location and surroundings). Based on the data, a UI framework was built. Two usage scenarios were designed, time-full and time-less, for better in-situation comprehension. Time- and path-based UI were proposed to flow with the scenarios. A usability test was conducted, and a passengers cognitive framework was defined. There are two aspects to this study: the data flow frameworks of the driver/passenger, and the UI design proposal. Situational precision from the perspective of the driver was analyzed to understand the relationship between the user, the vehicle and the road conditions. Further, the cognitive framework of the passenger was proposed based on the data. This study provides a solid understanding of drivers emerging needs when they are relieved of the cognitive burden of driving tasks. The UI features for AV are introduced based on the empirical data and research related to the provision of better situation awareness, focusing on time and location. This study contributes to the extant literature by observing the iv perspective of passengers in Autonomous vehicles based on a qualitative study. The proposed UI design will be further explored as a communication method between the system and the passive user in future mobility.사용자 인터페이스가 (UI) 미래 이동성에서 새로운 사용자 역할을 지지하는 디자인 도출은 미래 이동성 분야에서 중요한 디자인 이슈이다. 그러나 사용자 실험에 근거하여 자율주행차량 (AV) 의 탑승자인지 욕구와 행동에 대한 디자인 연구는 미미하다. 자율주행이 기술의 발전과 그 영역은 점차 넓어지고 있다. 해당 기술은 이미 운전 환경에 적용되고 있으며, 이로 인해 미래 이동문화에서 사용자의 역할은 '운전자'에서 '탑승자'로 변화한다. 본 연구는 미래 자율주행차량이 모든 운전 상황에 대처할 수 있다는 것을 전제로 한다. 사용자 실험을 통해 탑승자의 관점에 대한 분석을 진행하였고, 이를 기반으로 미래 모빌리티 환경에 적용될 사용자 인터페이스를 제안한다. 제안된 디자인은 운전자 중심의 상황인지에서 벗어나 탑승자 중심 인지 정보 요소를 분석하였고, 시간과 경로 두 가지 요소를 강조한 UI 를 제안한다. 본 연구에서 탑승자의 인지 정보 요구에 대한 실험적 데이터를 수집하였다. 탑승자의 관점을 이해하기 위해 다양한 관점에서 인간의 인지적 특성 및 운전 태스크를 관찰하였고, 상황인지 (SA) 에 관한 문헌 연구와 데이터 프레임워크 구조화를 통해 운전 환경에서 발생하는 인지적 요소 관계를 분석하였다. 제안된 프레임워크는 기술 변화에 따라 운전자가 탑승자로 변화되었을 때 운전 환경에서의 데이터 관계 변화를 시각적으로 구조화하여 심층적으로 탐구되었다. 탑승자의 인지 니즈 대한 실험적 데이터베이스를 수집하기 위해 총 3 세트의 유저 테스트와 심층 인터뷰가 수반되었다. 유저 테스트는 Wizard of Oz 방법을 사용하여 설계되었으며 실험 결과는 질적 연구방법론의 분석 방법을 통해 분석되었다. 실험을 통해 얻은 인사이트를 바탕으로 탑승자 관점에서 UI 디자인을 제안하고 사용성 테스트를 통해 효율성과 유용성을 5 점 리 커트 스케일로써 검증하였다. 실험 결과에 따르면 탑승자가 요청한 인지 정보는 시간 (현재 시각 및 기간)과 경로 (차량 위치 및 주변 환경)에 집중된 것을 관찰할 수 있었다. 이와 같은 데이터를 기반으로 UI 프레임워크를 구성하였다. 상황 속의 사용례를 제시하기 위하여도 가지 time-full 과 time-less 의 사용 시나리오를 구축하고, 제안된 시나리오에 따라 시간과 위치에 기반한 UI 를 제안하였다. 제안된 UI 에 대한 사용성 테스트를 진행하였고, 탑승자 관점에서의 운전상황 인지 워크 프레임을 완성하였다. 본 연구의 가치는 두 가지로 정리될 수 있다. 하나는 운전자 / 탑승자의 데이터 플로우 프레임워크를 제안하였다는 것과 두 번째는 탑승자의 관점을 지지하는 UI 디자인 제안에 있다. 운전자의 관점에서의 운전 상황을 분석하여 사용자, 차량, 그리고 도로 상태 간의 관계를 시각화하였고, 이는 탑승자인지 플로우 프레임워크를 제안하는데 기조적인 틀로써 사용되었다. 본 연구는 운전 태스크를 수행하는 데에 필요했던 인지 부담에서 벗어났을 때의 운전자가 필요로 하는 복합적인 니즈에 대해 관찰하고 미래 모빌리티 환경에 적합한 UI 의 디자인 요소에 대한 연구논문이다. 미래 자율주행차량 안의 사용자 인터페이스가 갖추어야 하는 요소를 실험적 데이터에 근거하여 제시하며, 시간과 루트를 강조하여 향상된 상황 인지를 제공하는 방법에 대한 심도있는 관찰을 기록한다. 본 연구는 질적 연구에 기초한 자율 차량의 탑승자 관점을 관찰함으로써 기존 자율주행이 디자인 연구에 기여할 것이다. 제안된 UI 디자인 미래 이동 성안에서 시스템과 탑승자 간의 커뮤니케이션 방법에 대한 연구로써 그 의의가 있다.ABSTRACT ...................................................................................................................... II CHAPTER 1. INTRODUCTION......................................................................................... １ 1.1. BACKGROUND ..............................................................................................................１ 1.2. PURPOSE .....................................................................................................................７ 1.3. RESEARCH QUESTION.....................................................................................................８ CHAPTER 2. LITERATURE REVIEW ..............................................................................１１ 2.1. SITATION AWARENESS (SA) ........................................................................................１１ 2.2. HUMAN INFORMATION PROCESSING MODEL..................................................................１５ 2.3. DRIVING SITUATION AWARENESS AND PERSPECTIVE.........................................................２０ 2.4. DRIVING TASK AND SENSORY INTERACTION ....................................................................２２ CHAPTER 3. COGNITIVE NEEDS IN AUTONOMOUS.....................................................２７ 3.1. DRIVING BEHAVIOR TRANSFORMATION AND CLUSTER UI..................................................２７ 3.2. COGNITIVE FRAMEWORK TRANSFORMATION ..................................................................３３ CHAPTER 4. USER TESTS ............................................................................................３６ 4.1. WIZARD OF OZ PROTOTYPING .....................................................................................３８ 4.2. PILOT TEST 1............................................................................................................４０ 4.2.1. Experiment Design & Laboratory Setting.................................................４０ 4.2.2. Persona Scenario & Task Design ..............................................................４２ 4.2.3. Preparation of Driving situation...............................................................４５ 4.2.4. Procedure.................................................................................................４７ 4.2.5. Data Analysis & Insight............................................................................４８ 4.3. PILOT TEST 2............................................................................................................５１ 4.3.1. Amendment: Experiment Design & Laboratory Setting ...........................５２ 4.3.2. Amendment: Task Scenario & Command Cue..........................................５４ 4.3.3. Amendment: Perform Role and preparation of driving situation ............５７ 4.3.4. Amendment: Procedure ...........................................................................５９ 4.3.5. Data Analysis & Insight............................................................................６２ 4.4. MAIN TEST ..............................................................................................................６５ 4.4.1. Experiment Design & Laboratory setting .................................................６６ 4.4.2. Task Design ..............................................................................................６９ 4.4.3. Procedure.................................................................................................７１ 4.4.4. Result Analysis & Insight..........................................................................７４ CHAPTER 5. UI CONCEPT DEVELOPMENT...................................................................８１ 5.1. UI DESIGN METHOD..................................................................................................８１ 5.2. DESIGN PROPOSAL ....................................................................................................８４ 5.3. USER SCENARIOS ......................................................................................................８６ 5.3.1 Scenario 1. Time-less: Late for a morning meeting..................................８６ 5.3.2 Scenario 2.Time-full: Leisure driving on weekends ..................................９３ CHAPTER 6. USABILITY TEST ......................................................................................９８ 6.1. USABILITY TEST GUIDE ...............................................................................................９８ 6.2. ASSESSMENT USABILITY TEST ..................................................................................１００ 6.2.1 Test planning........................................................................................１００ 6.2.2 Laboratory setting................................................................................１０２ 6.2.3 Test conduct and debriefing.................................................................１０６ 6.3. RESULT ANALYSIS ..................................................................................................１０６ CHAPTER 7. CONCLUSION......................................................................................１０７ APPENDIX 1...........................................................................................................１１０ APPENDIX 2...........................................................................................................１１１ APPENDIX 3...........................................................................................................１１３ APPENDIX 4...........................................................................................................１２１ APPENDIX 5...........................................................................................................１２４ APPENDIX 6...........................................................................................................１２８ APPENDIX 7...........................................................................................................１３３ BIBLIOGRAPHY ......................................................................................................１３６ 국문 초록 ............................................................................................................１４３Maste

Multi-Dimensional-Personalization in mobile contexts

During the dot com era the word "personalisation” was a hot buzzword. With the fall of the dot com companies the topic has lost momentum. As the killer application for UMTS or the mobile internet has yet to be identified, the concept of Multi-Dimensional-Personalisation (MDP) could be a candidate. Using this approach, a recommendation of mobile advertisement or marketing (i.e., recommendations or notifications), online content, as well as offline events, can be offered to the user based on their known interests and current location. Instead of having to request or pull this information, the new service concept would proactively provide the information and services – with the consequence that the right information or service could therefore be offered at the right place, at the right time. The growing availability of "Location-based Services“ for mobile phones is a new target for the use of personalisation. "Location-based Services“ are information, for example, about restaurants, hotels or shopping malls with offers which are in close range / short distance to the user. The lack of acceptance for such services in the past is based on the fact that early implementations required the user to pull the information from the service provider. A more promising approach is to actively push information to the user. This information must be from interest to the user and has to reach the user at the right time and at the right place. This raises new requirements on personalisation which will go far beyond present requirements. It will reach out from personalisation based only on the interest of the user. Besides the interest, the enhanced personalisation has to cover the location and movement patterns, the usage and the past, present and future schedule of the user. This new personalisation paradigm has to protect the user’s privacy so that an approach supporting anonymous recommendations through an extended "Chinese Wall“ will be described

Hybrid wheelchair controller for handicapped and quadriplegic patients

In this dissertation, a hybrid wheelchair controller for handicapped and quadriplegic patient is proposed. The system has two sub-controllers which are the voice controller and the head tilt controller. The system aims to help quadriplegic, handicapped, elderly and paralyzed patients to control a robotic wheelchair using voice commands and head movements instead of a traditional joystick controller. The multi-input design makes the system more flexible to adapt to the available body signals. The low-cost design is taken into consideration as it allows more patients to use this system

Crossmodal displays : coordinated crossmodal cues for information provision in public spaces

PhD ThesisThis thesis explores the design of Crossmodal Display, a new kind of display-based interface that aims to help prevent information overload and support information presentation for multiple simultaneous people who share a physical space or situated interface but have different information needs and privacy concerns. By exploiting the human multimodal perception and utilizing the synergy of both existing public displays and personal displays, crossmodal displays avoid numerous drawbacks associated with previous approaches, including a reliance on tracking technologies, weak protection for user‟s privacy, small user capacity and high cognitive load demands. The review of the human multimodal perception in this thesis, especially multimodal integration and crossmodal interaction, has many design implications for the design of crossmodal displays and constitutes the foundation for our proposed conceptual model. Two types of crossmodal display prototype applications are developed: CROSSFLOW for indoor navigation and CROSSBOARD for information retrieval on high-density information display; both of these utilize coordinated crossmodal cues to guide multiple simultaneous users‟ attention to publicly visible information relevant to each user timely. Most of the results of single-user and multi-user lab studies on the prototype systems we developed in this research demonstrate the effectiveness and efficiency of crossmodal displays and validate several significant advantages over the previous solutions. However, the results also reveal that more detailed usability and user experience of crossmodal displays as well as the human perception of crossmodal cues should be investigated and improved. This thesis is the first exploration into the design of crossmodal displays. A set of design suggestions and a lifecycle model of crossmodal display development have been produced, and can be used by designers or other researchers who wish to develop crossmodal displays for their applications or integrate crossmodal cues in their interfaces

The development and evaluation of a prototyping environment for context-sensitive mobile computing interaction

Recent developments in wireless communication, mobile computing, and sensor technologies have prompted a new vision of the world in which we live. As witnesses the effects of Moore's law, which are evident in many aspects of innovative technical opportunity, such as cost, size, capacity, bandwidth, etc. These advances allow us to build new types of human-computer-environment interaction in augmented physical spaces. Ideally, mobile computing devices can go with people so that they can access information on the move as being constantly connected to the digital space. Sensor technologies enable mobile computing devices to sense their users and environments. This increases the interaction bandwidth between a human and a mobile computing device. The development of context-sensitive mobile computing systems requires considerable engineering skills. None of the existing approaches provides an effective means of obtaining location and environmental information using "standard" hardware and software. This raises the entry level of discovering more about this type of interaction to the designers. In addition, it is important to stress that relatively little is known about the usability problems that might arise from interaction with these different context-sensitive mobile computing applications. The focus of this thesis is on the development of a prototyping environment for context-sensitive mobile computing. This thesis makes two contributions. The most significant contribution is the presentation of the Glasgow Context Server (GCS). It has been specifically designed to address the concerns mentioned above. It successfully integrates an off-the-shelf radio Local Area Network (LAN) with the infrared sensors that have been a feature of many previous context-sensitive mobile computing applications. The GCS is intended to help interface designers validate the claimed benefits of location sensing, location disclosing and environment sensing applications. The second contribution is the working applications, in particular, a web-based annotation system for physical objects and a shopping assistant built upon the GCS environment. These demonstrations are used to evaluate the GCS approach and point out the challenging issues in computing technology as well as usability concern. The hope is that this research can provide interface designers with an in-depth reference to a prototyping environment for context-sensitive mobile computing applications

Developing Unobtrusive Mobile Interactions: a Model Driven Engineering approach

In Ubiquitous computing environments, people are surrounded by a lot of embedded services. With the inclusion of pervasive technologies such as sensors or GPS receivers, mobile devices turn into an effective communication tool between users and the services embedded in their environment. All these services compete for the attentional resources of the user. Thus, it is essential to consider the degree in which each service intrudes the user mind when services are designed. In order to prevent service behavior from becoming overwhelming, this work, based on Model Driven Engineering foundations, is devoted to develop services according to user needs. In this thesis, we provide a systematic method for the development of mobile services that can be adapted in terms of obtrusiveness. That is, services can be developed to provide their functionality at different obtrusiveness levels by minimizing the duplication of efforts. For the system specification, a modeling language is defined to cope with the particular requirements of the context-aware user interface domain. From this specification, following a sequence of well-defined steps, a software solution is obtained.Gil Pascual, M. (2010). Developing Unobtrusive Mobile Interactions: a Model Driven Engineering approach. http://hdl.handle.net/10251/12745Archivo delegad