A Study and Estimation a Lost Person Behavior in Crowded Areas Using Accelerometer Data from Smartphones

As smartphones become more popular, applications are being developed with new and innovative ways to solve problems in the day-to-day lives of users. One area of smartphone technology that has been developed in recent years is human activity recognition (HAR). This technology uses various sensors that are built into the smartphone to sense a person\u27s activity in real time. Applications that incorporate HAR can be used to track a person\u27s movements and are very useful in areas such as health care. We use this type of motion sensing technology, specifically, using data collected from the accelerometer sensor. The purpose of this study is to study and estimate the person who may become lost in a crowded area. The application is capable of estimating the movements of people in a crowded area, and whether or not the person is lost in a crowded area based on his/her movements as detected by the smartphone. This will be a great benefit to anyone interested in crowd management strategies. In this paper, we review related literature and research that has given us the basis for our own research. We also detail research on lost person behavior. We looked at the typical movements a person will likely make when he/she is lost and used these movements to indicate lost person behavior. We then evaluate and describe the creation of the application, all of its components, and the testing process

Ubitrack: A Study on Lost Person Activity Estimation Using Accelerometer Data from Smartphones

As smartphones become very more popular, applications are being developed with new and innovative ways to solve problems faced in the day-to-day lives of users. One area of smartphone technology that has been developed in recent years is human activity recognition HAR. This technology uses various sensors that are built into the smartphone to sense a person\u27s activity in real time. Applications that incorporate HAR can be used to track a person\u27s movements and are very useful in areas such as health care. In our research, we use this type of motion sensing technology, specifically, using data collected from the accelerometer sensor. The purpose of this study is to estimate the pilgrim who may become lost on the annual pilgrimage to Hajj. The application is capable of estimating the movements of people in a crowded area, and of indicating whether or not the person is lost in a crowded area based on his/her movements as detected by the smartphone. This will be a great benefit to anyone interested in crowd management strategies, specifically regarding Hajj. In this thesis, we review related literature and research that has given us the basis for our own research. For example, we could not create this application without the use of HAR technology and without specific classification algorithms. We also detail research on lost person behavior. We looked at the typical movements a person will likely make when he/she is lost and used these movements to indicate lost person behavior. We then describe the creation of the application, all of its components, and the testing process. Finally, we discuss the results of our trials and plans for future work

A Novel Approach to Complex Human Activity Recognition

Human activity recognition is a technology that offers automatic recognition of what a person is doing with respect to body motion and function. The main goal is to recognize a person\u27s activity using different technologies such as cameras, motion sensors, location sensors, and time. Human activity recognition is important in many areas such as pervasive computing, artificial intelligence, human-computer interaction, health care, health outcomes, rehabilitation engineering, occupational science, and social sciences. There are numerous ubiquitous and pervasive computing systems where users\u27 activities play an important role. The human activity carries a lot of information about the context and helps systems to achieve context-awareness. In the rehabilitation area, it helps with functional diagnosis and assessing health outcomes. Human activity recognition is an important indicator of participation, quality of life and lifestyle. There are two classes of human activities based on body motion and function. The first class, simple human activity, involves human body motion and posture, such as walking, running, and sitting. The second class, complex human activity, includes function along with simple human activity, such as cooking, reading, and watching TV. Human activity recognition is an interdisciplinary research area that has been active for more than a decade. Substantial research has been conducted to recognize human activities, but, there are many major issues still need to be addressed. Addressing these issues would provide a significant improvement in different aspects of the applications of the human activity recognition in different areas. There has been considerable research conducted on simple human activity recognition, whereas, a little research has been carried out on complex human activity recognition. However, there are many key aspects (recognition accuracy, computational cost, energy consumption, mobility) that need to be addressed in both areas to improve their viability. This dissertation aims to address the key aspects in both areas of human activity recognition and eventually focuses on recognition of complex activity. It also addresses indoor and outdoor localization, an important parameter along with time in complex activity recognition. This work studies accelerometer sensor data to recognize simple human activity and time, location and simple activity to recognize complex activity

A Light Weight Smartphone Based Human Activity Recognition System with High Accuracy

With the pervasive use of smartphones, which contain numerous sensors, data for modeling human activity is readily available. Human activity recognition is an important area of research because it can be used in context-aware applications. It has significant influence in many other research areas and applications including healthcare, assisted living, personal fitness, and entertainment. There has been a widespread use of machine learning techniques in wearable and smartphone based human activity recognition. Despite being an active area of research for more than a decade, most of the existing approaches require extensive computation to extract feature, train model, and recognize activities. This study presents a computationally efficient smartphone based human activity recognizer, based on dynamical systems and chaos theory. A reconstructed phase space is formed from the accelerometer sensor data using time-delay embedding. A single accelerometer axis is used to reduce memory and computational complexity. A Gaussian mixture model is learned on the reconstructed phase space. A maximum likelihood classifier uses the Gaussian mixture model to classify ten different human activities and a baseline. One public and one collected dataset were used to validate the proposed approach. Data was collected from ten subjects. The public dataset contains data from 30 subjects. Out-of-sample experimental results show that the proposed approach is able to recognize human activities from smartphones’ one-axis raw accelerometer sensor data. The proposed approach achieved 100% accuracy for individual models across all activities and datasets. The proposed research requires 3 to 7 times less amount of data than the existing approaches to classify activities. It also requires 3 to 4 times less amount of time to build reconstructed phase space compare to time and frequency domain features. A comparative evaluation is also presented to compare proposed approach with the state-of-the-art works

Vehicular Networks and Outdoor Pedestrian Localization

This thesis focuses on vehicular networks and outdoor pedestrian localization. In particular, it targets secure positioning in vehicular networks and pedestrian localization for safety services in outdoor environments. The former research topic must cope with three major challenges, concerning users’ privacy, computational costs of security and the system trust on user correctness. This thesis addresses those issues by proposing a new lightweight privacy-preserving framework for continuous tracking of vehicles. The proposed solution is evaluated in both dense and sparse vehicular settings through simulation and experiments in real-world testbeds. In addition, this thesis explores the benefit given by the use of low frequency bands for the transmission of control messages in vehicular networks. The latter topic is motivated by a significant number of traffic accidents with pedestrians distracted by their smartphones. This thesis proposes two different localization solutions specifically for pedestrian safety: a GPS-based approach and a shoe-mounted inertial sensor method. The GPS-based solution is more suitable for rural and suburban areas while it is not applicable in dense urban environments, due to large positioning errors. Instead the inertial sensor approach overcomes the limitations of previous technique in urban environments. Indeed, by exploiting accelerometer data, this architecture is able to precisely detect the transitions from safe to potentially unsafe walking locations without the need of any absolute positioning systems

Hajj crowd management via a mobile augmented reality application: a case of The Hajj event, Saudi Arabia

Hajj event is considered as one of the Islamic pillars that each Muslim, who could afford its’ expenses and are well bodied, should perform its’ rituals at least once in a lifetime. Therefore, they could travel to Mecca city, in the Kingdom of Saudi Arabia to perform Hajj rituals. This holy city hosts this event annually in the last month of the Arabic calendar, which is Dhul Hijjah, and it lasts for 6 days. In addition, those Muslim visitors or pilgrims are obligated to be accommodated at Hajj ritual places, which are Arafat, Mina and Muzdalifah. However, in the last ten years, it was noticed that the Hajj events are crowded every year. Therefore, Hajj crowd management is being a complex task, due to the huge number of the pilgrims as they are crowded at the Hajj ritual places. This huge number is causing many problems, and Hajj authorities are facing difficulties in managing those crowded pilgrims. As a result, this research focuses on three main problems that occur at Hajj events. First, difficulties in organizing the crowds’ movements of the pilgrims, as Hajj events host enormous number of pilgrims in limited geographical spaces at the ritual places. This problem leads to overcrowdings, congestions and stampedes. Second, the pilgrims could get lost at Hajj ritual places, especially when they are moving between these places. Third, lack of directional information and guidance for those lost pilgrims. This problem leads to difficulties in finding their groups at the ritual sites, because the huge number of the pilgrims. Thus, this research proposes to deploy a technology, such as a Mobile Augmented Reality application. This application would assist the Hajj authorities (staff and operators) in managing the pilgrims’ movements between the ritual places, and to provide directions to the lost pilgrims. In addition, it would help those lost pilgrims by alerting, and sending their location information to their group guide. On the other hand, the research literature review covers previous studies about the Hajj crowd management, as it is divided into two perspectives. The theoretical perspective, which explains the crowd management steps that should be followed and applied, as these steps would help the Hajj authorities to succeed in crowd management at Hajj events. The practical perspective presents some studies that are related to the Hajj events. Those studies offered some solutions to manage crowded pilgrims, to avoid overcrowdings and stampedes, and to identify, locate and guide lost pilgrims. The solutions were Radio Frequency Identification (RFID) systems, Global Positioning Systems (GPS) deceives and monitoring cameras. In addition, this research conducted and distributed questionnaires on 104 respondents. They were selected as they are related to Hajj events. The results of this research method confirmed that the Hajj events face problems. For example, overcrowdings, congestions and stampedes that occur at the ritual places, due to lack of pilgrims organization in limited spaces at these places. In addition, foreign pilgrims face difficulties in guidance, due to lack of directional information, and they could get lost from their groups at the Hajj events. In addition, the respondents suggested using technology to assist Hajj authorities in Hajj crowd management. Therefore, deploying MAR application is suggested, as a solution to solve or at least reduce the Hajj problems. The proposed application could help the Hajj authorities to manage the crowded pilgrims at the Hajj ritual places as this research illustrates two scenarios in Hajj crowd management. In conclusion, this application is beneficial and significant in crowd management at Hajj events, as it could provide instant information using high-speed process in sending and receiving information. In addition, the information about the pilgrims’ movements could be gathered, presented on smart devices and shared between applications’ users. Those users will be the Hajj staff on the ground and the Hajj operators in the control room of Hajj operations

An inertial motion capture framework for constructing body sensor networks

Motion capture is the process of measuring and subsequently reconstructing the movement of an animated object or being in virtual space. Virtual reconstructions of human motion play an important role in numerous application areas such as animation, medical science, ergonomics, etc. While optical motion capture systems are the industry standard, inertial body sensor networks are becoming viable alternatives due to portability, practicality and cost. This thesis presents an innovative inertial motion capture framework for constructing body sensor networks through software environments, smartphones and web technologies. The first component of the framework is a unique inertial motion capture software environment aimed at providing an improved experimentation environment, accompanied by programming scaffolding and a driver development kit, for users interested in studying or engineering body sensor networks. The software environment provides a bespoke 3D engine for kinematic motion visualisations and a set of tools for hardware integration. The software environment is used to develop the hardware behind a prototype motion capture suit focused on low-power consumption and hardware-centricity. Additional inertial measurement units, which are available commercially, are also integrated to demonstrate the functionality the software environment while providing the framework with additional sources for motion data. The smartphone is the most ubiquitous computing technology and its worldwide uptake has prompted many advances in wearable inertial sensing technologies. Smartphones contain gyroscopes, accelerometers and magnetometers, a combination of sensors that is commonly found in inertial measurement units. This thesis presents a mobile application that investigates whether the smartphone is capable of inertial motion capture by constructing a novel omnidirectional body sensor network. This thesis proposes a novel use for web technologies through the development of the Motion Cloud, a repository and gateway for inertial data. Web technologies have the potential to replace motion capture file formats with online repositories and to set a new standard for how motion data is stored. From a single inertial measurement unit to a more complex body sensor network, the proposed architecture is extendable and facilitates the integration of any inertial hardware configuration. The Motion Cloud’s data can be accessed through an application-programming interface or through a web portal that provides users with the functionality for visualising and exporting the motion data

Programming frameworks for mobile sensing

The proliferation of smart mobile devices in people’s daily lives is making context-aware computing a reality. A plethora of sensors available in these devices can be utilized to understand users’ context better. Apps can provide more relevant data or services to the user based on improved understanding of user’s context. With the advent of cloud-assisted mobile platforms, apps can also perform collaborative computation over the sensing data collected from a group of users. However, there are still two main issues: (1) A lack of simple and effective personal sensing frameworks: existing frameworks do not provide support for real-time fusing of data from motion and visual sensors in a simple manner, and no existing framework collectively utilizes sensors from multiple personal devices and personal IoT sensors, and (2) a lack of collaborative/distributed computing frameworks for mobile users. This dissertation presents solutions for these two issues. The first issue is addressed by TagPix and Sentio, two frameworks for mobile sensing. The second issue is addressed by Moitree, a middleware for mobile distributed computing, and CASINO, a collaborative sensor-driven offloading system. TagPix is a real-time, privacy preserving photo tagging framework, which works locally on the phones and consumes little resources (e.g., battery). It generates relevant tags for landscape photos by utilizing sensors of a mobile device and it does not require any previous training or indexing. When a user aims the mobile camera to a particular landmark, the framework uses accelerometer and geomagnetic field sensor to identify in which direction the user is aiming the camera at. It then uses a landmark database and employs a smart distance estimation algorithm to identify which landmark(s) is targeted by the user. The framework then generates relevant tags for the captured photo using these information. A more versatile sensing framework can be developed using sensors from multiple devices possessed by a user. Sentio is such a framework which enables apps to seamlessly utilize the collective sensing capabilities of the user’s personal devices and of the IoT sensors located in the proximity of the user. With Sentio, an app running on any personal mobile/wearable device can access any sensor of the user in real-time using the same API, can selectively switch to the most suitable sensor of a particular type when multiple sensors of this type are available at different devices, and can build composite sensors. Sentio offers seamless connectivity to sensors even if the sensor-accessing code is offloaded to the cloud. Sentio provides these functionalities with a high-level API and a distributed middleware that handles all low-level communication and sensor management tasks. This dissertation also proposes Moitree, a middleware for the mobile cloud platforms where each mobile device is augmented by an avatar, a per-user always-on software entity that resides in the cloud. Mobile-avatar pairs participate in distributed computing as a unified computing entity. Moitree provides a common programming and execution framework for mobile distributed apps. Moitree allows the components of a distributed app to execute seamlessly over a set of mobile/avatar pairs, with the provision of offloading computation and communication to the cloud. The programming framework has two key features: user collaborations are modeled using group semantics - groups are created dynamically based on context and are hierarchical; data communication among group members is offloaded to the cloud through high-level communication channels. Finally, this dissertation presents and discusses CASINO, a collaborative sensor-driven computation offloading framework which can be used alongside Moitree. This framework includes a new scheduling algorithm which minimizes the total completion time of a collaborative computation that executes over a set of mobile/avatar pairs. Using the CASINO API, the programmers can mark their classes and functions as ”offloadable”. The framework collects profiling information (network, CPU, battery, etc.) from participating users’ mobile devices and avatars, and then schedules ”offloadable” tasks in mobiles and avatars in a way that reduces the total completion time. The scheduling problem is proven to be NP-Hard and there is no polynomial time optimization algorithm for it. The proposed algorithm can generate a schedule in polynomial time using a topological sorting and greedy technique

Applying multimodal sensing to human location estimation

Mobile devices like smartphones and smartwatches are beginning to "stick" to the human body. Given that these devices are equipped with a variety of sensors, they are becoming a natural platform to understand various aspects of human behavior. This dissertation will focus on just one dimension of human behavior, namely "location". We will begin by discussing our research on localizing humans in indoor environments, a problem that requires precise tracking of human footsteps. We investigated the benefits of leveraging smartphone sensors (accelerometers, gyroscopes, magnetometers, etc.) into the indoor localization framework, which breaks away from pure radio frequency based localization (e.g., cellular, WiFi). Our research leveraged inherent properties of indoor environments to perform localization. We also designed additional solutions, where computer vision was integrated with sensor fusion to offer highly precise localization. We will close this thesis with micro-scale tracking of the human wrist and demonstrate how motion data processing is indeed a "double-edged sword", offering unprecedented utility on one hand while breaching privacy on the other