Conducting visitor studies using smartphone-based location sensing

Visitor studies explore human experiences within museums, cultural heritage sites, and other informal learning settings to inform decisions. Smartphones offer novel opportunities for extending the depth and breadth of visitor studies while considerably reducing their cost and their demands on specialist human resources. By enabling the collection of significantly higher volumes of data, they also make possible the application of advanced machine-learning and visualization techniques, potentially leading to the discovery of new patterns and behaviors that cannot be captured by simple descriptive statistics. In this article, we present a principled approach to the use of smartphones for visitor studies, in particular proposing a structured methodology and associated methods that enable its effective use in this context. We discuss specific methodological considerations that have to be addressed for effective data collection, preprocessing, and analysis and identify the limitations in the applicability of these tools using family visits to the London Zoo as a case study. We conclude with a discussion of the wider opportunities afforded by the introduction of smartphones and related technologies and outline the steps toward establishing them as a standard tool for visitor studies

Location-based Data Analysis of Visitor Structure for Recreational Area Management

This work presents a location-based data analysis framework for profiling visitors structures. In terms of recreational area management, understanding visitors’ structure and popularity is important. Traditionally, visitors monitoring with automatic counting devices has drawbacks of inaccurate visitors counting. In this work, compared to automatic counting devices, we use Wi-Fi tracking as the main method to count visitors, which provides a fairly precise picture of visitor structures. Moreover, we deliver rich analytic functions in this framework and we present the functionality with visitor data collected from Guanyinshan Visitor Center. This framework not only standardizes visitor counting process but also facilitates a profound analysis of visitor structures. Key Words: Guanyinshan Visitor Center, Wi-Fi trackin

Deep Room Recognition Using Inaudible Echos

Recent years have seen the increasing need of location awareness by mobile applications. This paper presents a room-level indoor localization approach based on the measured room's echos in response to a two-millisecond single-tone inaudible chirp emitted by a smartphone's loudspeaker. Different from other acoustics-based room recognition systems that record full-spectrum audio for up to ten seconds, our approach records audio in a narrow inaudible band for 0.1 seconds only to preserve the user's privacy. However, the short-time and narrowband audio signal carries limited information about the room's characteristics, presenting challenges to accurate room recognition. This paper applies deep learning to effectively capture the subtle fingerprints in the rooms' acoustic responses. Our extensive experiments show that a two-layer convolutional neural network fed with the spectrogram of the inaudible echos achieve the best performance, compared with alternative designs using other raw data formats and deep models. Based on this result, we design a RoomRecognize cloud service and its mobile client library that enable the mobile application developers to readily implement the room recognition functionality without resorting to any existing infrastructures and add-on hardware. Extensive evaluation shows that RoomRecognize achieves 99.7%, 97.7%, 99%, and 89% accuracy in differentiating 22 and 50 residential/office rooms, 19 spots in a quiet museum, and 15 spots in a crowded museum, respectively. Compared with the state-of-the-art approaches based on support vector machine, RoomRecognize significantly improves the Pareto frontier of recognition accuracy versus robustness against interfering sounds (e.g., ambient music).Comment: 29 page

User Experience Enhancement on Smartphones using Wireless Communication Technologies

학위논문 (박사) -- 서울대학교 대학원 : 공과대학 전기·정보공학부, 2020. 8. 박세웅.Recently, various sensors as well as wireless communication technologies such as Wi-Fi and Bluetooth Low Energy (BLE) have been equipped with smartphones. In addition, in many cases, users use a smartphone while on the move, so if a wireless communication technologies and various sensors are used for a mobile user, a better user experience can be provided. For example, when a user moves while using Wi-Fi, the user experience can be improved by providing a seamless Wi-Fi service. In addition, it is possible to provide a special service such as indoor positioning or navigation by estimating the users mobility in an indoor environment, and additional services such as location-based advertising and payment systems can also be provided. Therefore, improving the user experience by using wireless communication technology and smartphones sensors is considered to be an important research field in the future. In this dissertation, we propose three systems that can improve the user experience or convenience by usingWi-Fi, BLE, and smartphones sensors: (i) BLEND: BLE beacon-aided fast Wi-Fi handoff for smartphones, (ii) PYLON: Smartphone based Indoor Path Estimation and Localization without Human Intervention, (iii) FINISH: Fully-automated Indoor Navigation using Smartphones with Zero Human Assistance. First, we propose fast handoff scheme called BLEND exploiting BLE as secondary radio. We conduct detailed analysis of the sticky client problem on commercial smartphones with experiment and close examination of Android source code. We propose BLEND, which exploits BLE modules to provide smartphones with prior knowledge of the presence and information of APs operating at 2.4 and 5 GHz Wi-Fi channels. BLEND operating with only application requires no hardware and Android source code modification of smartphones.We prototype BLEND with commercial smartphones and evaluate the performance in real environment. Our measurement results demonstrate that BLEND significantly improves throughput and video bitrate by up to 61% and 111%, compared to a commercial Android application, respectively, with negligible energy overhead. Second, we design a path estimation and localization system, termed PYLON, which is plug-and-play on Android smartphones. PYLON includes a novel landmark correction scheme that leverages real doors of indoor environments consisting of floor plan mapping, door passing time detection and correction. It operates without any user intervention. PYLON relaxes some requirements for localization systems. It does not require any modifications to hardware or software of smartphones, and the initial location of WiFi APs, BLE beacons, and users. We implement PYLON on five Android smartphones and evaluate it on two office buildings with the help of three participants to prove applicability and scalability. PYLON achieves very high floor plan mapping accuracy with a low localization error. Finally, We design a fully-automated navigation system, termed FINISH, which addresses the problems of existing previous indoor navigation systems. FINISH generates the radio map of an indoor building based on the localization system to determine the initial location of the user. FINISH relaxes some requirements for current indoor navigation systems. It does not require any human assistance to provide navigation instructions. In addition, it is plug-and-play on Android smartphones. We implement FINISH on five Android smartphones and evaluate it on five floors of an office building with the help of multiple users to prove applicability and scalability. FINISH determines the location of the user with extremely high accuracy with in one step. In summary, we propose systems that enhance the users convenience and experience by utilizing wireless infrastructures such as Wi-Fi and BLE and various smartphones sensors such as accelerometer, gyroscope, and barometer equipped in smartphones. Systems are implemented on commercial smartphones to verify the performance through experiments. As a result, systems show the excellent performance that can enhance the users experience.1 Introduction 1 1.1 Motivation 1 1.2 Overview of Existing Approaches 3 1.2.1 Wi-Fi handoff for smartphones 3 1.2.2 Indoor path estimation and localization 4 1.2.3 Indoor navigation 5 1.3 Main Contributions 7 1.3.1 BLEND: BLE Beacon-aided Fast Handoff for Smartphones 7 1.3.2 PYLON: Smartphone Based Indoor Path Estimation and Localization with Human Intervention 8 1.3.3 FINISH: Fully-automated Indoor Navigation using Smartphones with Zero Human Assistance 9 1.4 Organization of Dissertation 10 2 BLEND: BLE Beacon-Aided FastWi-Fi Handoff for Smartphones 11 2.1 Introduction 11 2.2 Related Work 14 2.2.1 Wi-Fi-based Handoff 14 2.2.2 WPAN-aided AP Discovery 15 2.3 Background 16 2.3.1 Handoff Procedure in IEEE 802.11 16 2.3.2 BSS Load Element in IEEE 802.11 16 2.3.3 Bluetooth Low Energy 17 2.4 Sticky Client Problem 17 2.4.1 Sticky Client Problem of Commercial Smartphone 17 2.4.2 Cause of Sticky Client Problem 20 2.5 BLEND: Proposed Scheme 21 2.5.1 Advantages and Necessities of BLE as Secondary Low-Power Radio 21 2.5.2 Overall Architecture 22 2.5.3 AP Operation 23 2.5.4 Smartphone Operation 24 2.5.5 Verification of aTH estimation 28 2.6 Performance Evaluation 30 2.6.1 Implementation and Measurement Setup 30 2.6.2 Saturated Traffic Scenario 31 2.6.3 Video Streaming Scenario 35 2.7 Summary 38 3 PYLON: Smartphone based Indoor Path Estimation and Localization without Human Intervention 41 3.1 Introduction 41 3.2 Background and Related Work 44 3.2.1 Infrastructure-Based Localization 44 3.2.2 Fingerprint-Based Localization 45 3.2.3 Model-Based Localization 45 3.2.4 Dead Reckoning 46 3.2.5 Landmark-Based Localization 47 3.2.6 Simultaneous Localization and Mapping (SLAM) 47 3.3 System Overview 48 3.3.1 Notable RSSI Signature 49 3.3.2 Smartphone Operation 50 3.3.3 Server Operation 51 3.4 Path Estimation 52 3.4.1 Step Detection 52 3.4.2 Step Length Estimation 54 3.4.3 Walking Direction 54 3.4.4 Location Update 55 3.5 Landmark Correction Part 1: Virtual Room Generation 56 3.5.1 RSSI Stacking Difference 56 3.5.2 Virtual Room Generation 57 3.5.3 Virtual Graph Generation 59 3.5.4 Physical Graph Generation 60 3.6 Landmark Correction Part 2: From Floor Plan Mapping to Path Correction 60 3.6.1 Candidate Graph Generation 60 3.6.2 Backbone Node Mapping 62 3.6.3 Dead-end Node Mapping 65 3.6.4 Final Candidate Graph Selection 66 3.6.5 Door Passing Time Detection 68 3.6.6 Path Correction 70 3.7 Particle Filter 71 3.8 Performance Evaluation 73 3.8.1 Implementation and Measurement Setup 73 3.8.2 Step Detection Accuracy 77 3.8.3 Floor Plan Mapping Accuracy 77 3.8.4 Door Passing Time 78 3.8.5 Walking Direction and Localization Performance 81 3.8.6 Impact of WiFi AP and BLE Beacon Number 84 3.8.7 Impact of Walking Distance and Speed 84 3.8.8 Performance on Different Areas 87 3.9 Summary 87 4 FINISH: Fully-automated Indoor Navigation using Smartphones with Zero Human Assistance 91 4.1 Introduction 91 4.2 Related Work 92 4.2.1 Localization-based Navigation System 92 4.2.2 Peer-to-peer Navigation System 93 4.3 System Overview 93 4.3.1 System Architecture 93 4.3.2 An Example for Navigation 95 4.4 Level Change Detection and Floor Decision 96 4.4.1 Level Change Detection 96 4.5 Real-time navigation 97 4.5.1 Initial Floor and Location Decision 97 4.5.2 Orientation Adjustment 98 4.5.3 Shortest Path Estimation 99 4.6 Performance Evaluation 99 4.6.1 Initial Location Accuracy 99 4.6.2 Real-Time Navigation Accuracy 100 4.7 Summary 101 5 Conclusion 102 5.1 Research Contributions 102 5.2 Future Work 103 Abstract (In Korean) 118 감사의 글Docto

Mobile Sensing, BYOD and Big Data Analytics: New technologies for audience research in museums

Over the past four decades, varied information technology and communication tools were incorporated in standard audience research methodologies thus becoming a ubiquitous feature of fieldwork and data analysis. In this paper, we highlight the costs and benefits of the application of these technologies to audience research through specific case studies set within varied museum environments. Collectively the findings of these studies suggest that mobile sensing supports the acquisition of richer and deeperdata sets relating to the visitor experience that can potentially reveal a greater diversity of patterns of visitor behaviour as well as the changing dynamics of visiting over time. This can considerably extend our capability to conduct audience research and rethink the way museums conceptualize and deliver their interpretation. This paper also discusses the methodological, ethical and practical challenges emerging from the use of these technologies in the field as well as the limits of their applicatio

Usable privacy and security in smart homes

Ubiquitous computing devices increasingly dominate our everyday lives, including our most private places: our homes. Homes that are equipped with interconnected, context-aware computing devices, are considered “smart” homes. To provide their functionality and features, these devices are typically equipped with sensors and, thus, are capable of collecting, storing, and processing sensitive user data, such as presence in the home. At the same time, these devices are prone to novel threats, making our homes vulnerable by opening them for attackers from outside, but also from within the home. For instance, remote attackers who digitally gain access to presence data can plan for physical burglary. Attackers who are physically present with access to devices could access associated (sensitive) user data and exploit it for further cyberattacks. As such, users’ privacy and security are at risk in their homes. Even worse, many users are unaware of this and/or have limited means to take action. This raises the need to think about usable mechanisms that can support users in protecting their smart home setups. The design of such mechanisms, however, is challenging due to the variety and heterogeneity of devices available on the consumer market and the complex interplay of user roles within this context. This thesis contributes to usable privacy and security research in the context of smart homes by a) understanding users’ privacy perceptions and requirements for usable mechanisms and b) investigating concepts and prototypes for privacy and security mechanisms. Hereby, the focus is on two specific target groups, that are inhabitants and guests of smart homes. In particular, this thesis targets their awareness of potential privacy and security risks, enables them to take control over their personal privacy and security, and illustrates considerations for usable authentication mechanisms. This thesis provides valuable insights to help researchers and practitioners in designing and evaluating privacy and security mechanisms for future smart devices and homes, particularly targeting awareness, control, and authentication, as well as various roles.Computer und andere „intelligente“, vernetzte Geräte sind allgegenwärtig und machen auch vor unserem privatesten Zufluchtsort keinen Halt: unserem Zuhause. Ein „intelligentes Heim“ verspricht viele Vorteile und nützliche Funktionen. Um diese zu erfüllen, sind die Geräte mit diversen Sensoren ausgestattet – sie können also in unserem Zuhause sensitive Daten sammeln, speichern und verarbeiten (bspw. Anwesenheit). Gleichzeitig sind die Geräte anfällig für (neuartige) Cyberangriffe, gefährden somit unser Zuhause und öffnen es für potenzielle – interne sowie externe – Angreifer. Beispielsweise könnten Angreifer, die digital Zugriff auf sensitive Daten wie Präsenz erhalten, einen physischen Überfall in Abwesenheit der Hausbewohner planen. Angreifer, die physischen Zugriff auf ein Gerät erhalten, könnten auf assoziierte Daten und Accounts zugreifen und diese für weitere Cyberangriffe ausnutzen. Damit werden die Privatsphäre und Sicherheit der Nutzenden in deren eigenem Zuhause gefährdet. Erschwerend kommt hinzu, dass viele Nutzenden sich dessen nicht bewusst sind und/oder nur limitierte Möglichkeiten haben, effiziente Gegenmaßnahmen zu ergreifen. Dies macht es unabdingbar, über benutzbare Mechanismen nachzudenken, die Nutzende beim Schutz ihres intelligenten Zuhauses unterstützen. Die Umsetzung solcher Mechanismen ist allerdings eine große Herausforderung. Das liegt unter anderem an der großen Vielfalt erhältlicher Geräte von verschiedensten Herstellern, was das Finden einer einheitlichen Lösung erschwert. Darüber hinaus interagieren im Heimkontext meist mehrere Nutzende in verschieden Rollen (bspw. Bewohner und Gäste), was die Gestaltung von Mechanismen zusätzlich erschwert. Diese Doktorarbeit trägt dazu bei, benutzbare Privatsphäre- und Sicherheitsmechanismen im Kontext des „intelligenten Zuhauses“ zu entwickeln. Insbesondere werden a) die Wahrnehmung von Privatsphäre sowie Anforderungen an potenzielle Mechanismen untersucht, sowie b) Konzepte und Prototypen für Privatsphäre- und Sicherheitsmechanismen vorgestellt. Der Fokus liegt hierbei auf zwei Zielgruppen, den Bewohnern sowie den Gästen eines intelligenten Zuhauses. Insbesondere werden in dieser Arbeit deren Bewusstsein für potenzielle Privatsphäre- und Sicherheits-Risiken adressiert, ihnen Kontrolle über ihre persönliche Privatsphäre und Sicherheit ermöglicht, sowie Möglichkeiten für benutzbare Authentifizierungsmechanismen für beide Zielgruppen aufgezeigt. Die Ergebnisse dieser Doktorarbeit legen den Grundstein für zukünftige Entwicklung und Evaluierung von benutzbaren Privatsphäre und Sicherheitsmechanismen im intelligenten Zuhause

Context-Based Cultural Visits

Over the last two decades, there have been tremendous advances in mobile technologies, which have increased the interest in studying and developing mobile augmented reality systems, especially in the field of Cultural Heritage. Nowadays, people rely even more on smartphones, for example, when visiting a new city to search for information about monuments and landmarks, and the visitor expects precise and tailored information to his needs. Therefore, researchers started to investigate innovative approaches for presenting and suggesting digital content related to cultural and historical places around the city, incorporating contextual information about the visitor and his needs. This document presents a novel mobile augmented reality application, NearHeritage, that was developed within the scope of the master's thesis on Electrical and Computers Engineering from the Faculty of Engineering of Porto University (FEUP), in collaboration with INESC TEC. The research carried out was focused on the importance of utilising modern technologies to assist the visitors in finding and exploring Cultural Heritage. In this way, it is provided not only the nearby points-of-interest of a city but also detailed information about each POI. The solution presented uses built-in sensors and hardware of Android devices and takes advantage of various APIs (Foursquare API, Google Maps API and IntelContextSensing) to retrieve information about the landmarks and the visitor context. Also, these are crucial hardware components for implementing the full potential of augmented reality tools to create innovative contents that increase the overall user experience. All the experiments were conducted in Porto, Portugal, and the final results showcase that the concept of a MAR application can improve the user experience in discovering and learning more about Cultural Heritage around the world, creating an interactive, enjoyable and unforgettable adventure

‘Smart’ Visitor Mobility Management in the Tourist-Historic City

The multi-disciplinary study tries to establish interrelations between online mobile technology used for heritage interpretation and visitor mobility behaviour, exploring the implications that the changed behaviour patterns have for applied visitor management in UNESCO World Heritage listed historic city centres. Ubiquitous online mobile technology disperses delimitations between activities, time and space which affects and fundamentally changes everyday life including touristic utilized spaces and activities. In this ‘liquid modernity’, tourism is perceived as an essential form of mobility, increasingly changing under the influence of online mobile technologies. With this changing landscape of both, tourism and online mobile technology, interpretation of space, communication of information and the ensuing movement within a destination have to be amended accordingly. Tourism is often considered as one of the few options for economic vitality in historic cities. However, there is a fine line between tourism being an added value and support to conservation, and tourism becoming the reason for conservation. This study scrutinizes prevailing visitor mobility management strategies in historic cities in the context of changing visitor movement triggered by the increasing influence of online mobile media. For this purpose the technology awareness and acceptance of destination management organisations and the utilization of interpretive online mobile media is explored by conducting an online survey in 127 UNESCO World Heritage historic city centres, followed by an in-depth analysis in selected study areas of the compatibility of currently applied visitor mobility management strategies with prevailing behaviour of ‘connected’ visitors using sentiment analysis on data generated from social media mining

Improving Visitor Evaluation at the British Museum

The purpose of this project was to collaborate with the Interpretation Team at the British Museum to streamline the process of visitor evaluation. We reviewed best practices and technologies used in visitor evaluation by museums in general and the British Museum in particular. We developed an innovative toolkit for conducting visitor evaluation studies and visualizing the results in easy and intuitive ways. We used a temporary display room, Room 3, as the baseline for our approach. The team devised a README file to instruct the Museum staff and volunteers how to bridge the tools together. We recommend the British Museum use tools like Microsoft Excel and Visio and SurveyJS to obtain the necessary materials to gather, analyze, and visualize data using our prototype toolkit application

都市の持続可能性に向けた旅行行動と知的移動データ統合に関する包括的研究

過去数十年にわたり世界中で都市の持続可能性がトレンドとなり研究対象となっている．人々は，非効率な天然資源の消費や社会経済活動による環境破壊など，地球環境に有害な活動を行い，これには都市計画や交通計画を始め，多くの分野が密接に関係している．現在では，これらを解決する新技術の開発や応用が広範囲な研究分野で日々取り組まれている．本研究では観光に関する問題を，交通と都市の研究の観点からさまざまなビックデータを使用し，持続可能な都市開発を目標とした具体的な解決策を示した．本研究では都市や地域の持続可能性に資するデータの活用方法として，Wi-Fiパケットセンサーを使用した旅行者にとって魅力的な観光目的地マネジメントに関する研究，およびETCプローブデータを使用した旅行時間の信頼性の観測における天候の影響に関する分析を組み合わせて示した．本論文では，都市の移動性の認知に対して以下に示す3つの研究から，特徴的な結果と有効な分析手法を確立した．1）Wi-Fiパッケージセンシング調査を使用した，広域観光エリアでの周遊パターンのマイニングベースの関連法則の調査，2）Wi-Fi追跡データでの大規模な観光地の持続可能な開発に向けた魅力的な目的地の抽出，3）ETC2.0プローブデータを使用して，様々な道路タイプを考慮した旅行の信頼性に対する降雪の影響の評価．以上の研究から，複数視点の考察を積み重ね，包括的な評価と提案を行い，いくつかの重要な結果が得られた．この論文の貢献は，より良い社会への問題解決への糸口となり，今後の政策立案者にとって有意義な内容となるだろう．According to sustainability, the trend is spreading out around the world for past decades. There are many area subjects involved, such as city planning, transportation planning, and so on, because people realized human activities harmful to the environment by consuming natural resources with less efficiency process or damage environment by social and economic movements. Currently, emerging technologies considered for the proactive procedure in extensive study areas regarding new technology application and knowledge based. In term of transport and urban study, including tourism concerns, we used intelligent data from deferent sources to be demonstrating the possible solutions which involve sustainable urban development concept. In this study, as a method of utilizing data that contributes to the sustainability of cities and regions, consideration of attractive destination management for tourists by using wireless probe data, and the weather impact on travel time reliability observation by using electronic toll collection probe data, it represented as combination experiments throughout comprehensive study. This dissertation addressed three contribution studies to the composed acknowledgment of urban mobility, and it obtained the intelligent data and specific method of research-based. It consists of; 1) an association rule mining-based exploration of travel patterns in wide tourism areas using a Wi-Fi package sensing survey, 2) Attractive destinations mining towards massive tourism area sustainable development on Wi-Fi tracking data, and 3) Assessment of the impact of snowfall on travel reliability considering different road types using ETC2.0 probe data. Hence, a stack of varying viewpoints researches provided a comprehensive review and suggestion throughout significant results. The contribution of this dissertation could be an advantage substance for strategy and policies planner to recognize alternative solutions leading to a better society.室蘭工業大学 (Muroran Institute of Technology)博士（工学