Radar and RGB-depth sensors for fall detection: a review

This paper reviews recent works in the literature on the use of systems based on radar and RGB-Depth (RGB-D) sensors for fall detection, and discusses outstanding research challenges and trends related to this research field. Systems to detect reliably fall events and promptly alert carers and first responders have gained significant interest in the past few years in order to address the societal issue of an increasing number of elderly people living alone, with the associated risk of them falling and the consequences in terms of health treatments, reduced well-being, and costs. The interest in radar and RGB-D sensors is related to their capability to enable contactless and non-intrusive monitoring, which is an advantage for practical deployment and users’ acceptance and compliance, compared with other sensor technologies, such as video-cameras, or wearables. Furthermore, the possibility of combining and fusing information from The heterogeneous types of sensors is expected to improve the overall performance of practical fall detection systems. Researchers from different fields can benefit from multidisciplinary knowledge and awareness of the latest developments in radar and RGB-D sensors that this paper is discussing

Stacking-based visualization of trajectory attribute data

Visualizing trajectory attribute data is challenging because it involves showing the trajectories in their spatio-temporal context as well as the attribute values associated with the individual points of trajectories. Previous work on trajectory visualization addresses selected aspects of this problem, but not all of them. We present a novel approach to visualizing trajectory attribute data. Our solution covers space, time, and attribute values. Based on an analysis of relevant visualization tasks, we designed the visualization solution around the principle of stacking trajectory bands. The core of our approach is a hybrid 2D/3D display. A 2D map serves as a reference for the spatial context, and the trajectories are visualized as stacked 3D trajectory bands along which attribute values are encoded by color. Time is integrated through appropriate ordering of bands and through a dynamic query mechanism that feeds temporally aggregated information to a circular time display. An additional 2D time graph shows temporal information in full detail by stacking 2D trajectory bands. Our solution is equipped with analytical and interactive mechanisms for selecting and ordering of trajectories, and adjusting the color mapping, as well as coordinated highlighting and dedicated 3D navigation. We demonstrate the usefulness of our novel visualization by three examples related to radiation surveillance, traffic analysis, and maritime navigation. User feedback obtained in a small experiment indicates that our hybrid 2D/3D solution can be operated quite well

DRLViz: Understanding Decisions and Memory in Deep Reinforcement Learning

We present DRLViz, a visual analytics interface to interpret the internal memory of an agent (e.g. a robot) trained using deep reinforcement learning. This memory is composed of large temporal vectors updated when the agent moves in an environment and is not trivial to understand due to the number of dimensions, dependencies to past vectors, spatial/temporal correlations, and co-correlation between dimensions. It is often referred to as a black box as only inputs (images) and outputs (actions) are intelligible for humans. Using DRLViz, experts are assisted to interpret decisions using memory reduction interactions, and to investigate the role of parts of the memory when errors have been made (e.g. wrong direction). We report on DRLViz applied in the context of video games simulators (ViZDoom) for a navigation scenario with item gathering tasks. We also report on experts evaluation using DRLViz, and applicability of DRLViz to other scenarios and navigation problems beyond simulation games, as well as its contribution to black box models interpretability and explainability in the field of visual analytics

Hierarchical Graphs as Organisational Principle and Spatial Model Applied to Pedestrian Indoor Navigation

In this thesis, hierarchical graphs are investigated from two different angles – as a general modelling principle for (geo)spatial networks and as a practical means to enhance navigation in buildings. The topics addressed are of interest from a multi-disciplinary point of view, ranging from Computer Science in general over Artificial Intelligence and Computational Geometry in particular to other fields such as Geographic Information Science. Some hierarchical graph models have been previously proposed by the research community, e.g. to cope with the massive size of road networks, or as a conceptual model for human wayfinding. However, there has not yet been a comprehensive, systematic approach for modelling spatial networks with hierarchical graphs. One particular problem is the gap between conceptual models and models which can be readily used in practice. Geospatial data is commonly modelled - if at all - only as a flat graph. Therefore, from a practical point of view, it is important to address the automatic construction of a graph hierarchy based on the predominant data models. The work presented deals with this problem: an automated method for construction is introduced and explained. A particular contribution of my thesis is the proposition to use hierarchical graphs as the basis for an extensible, flexible architecture for modelling various (geo)spatial networks. The proposed approach complements classical graph models very well in the sense that their expressiveness is extended: various graphs originating from different sources can be integrated into a comprehensive, multi-level model. This more sophisticated kind of architecture allows for extending navigation services beyond the borders of one single spatial network to a collection of heterogeneous networks, thus establishing a meta-navigation service. Another point of discussion is the impact of the hierarchy and distribution on graph algorithms. They have to be adapted to properly operate on multi-level hierarchies. By investigating indoor navigation problems in particular, the guiding principles are demonstrated for modelling networks at multiple levels of detail. Complex environments like large public buildings are ideally suited to demonstrate the versatile use of hierarchical graphs and thus to highlight the benefits of the hierarchical approach. Starting from a collection of floor plans, I have developed a systematic method for constructing a multi-level graph hierarchy. The nature of indoor environments, especially their inherent diversity, poses an additional challenge: among others, one must deal with complex, irregular, and/or three-dimensional features. The proposed method is also motivated by practical considerations, such as not only finding shortest/fastest paths across rooms and floors, but also by providing descriptions for these paths which are easily understood by people. Beyond this, two novel aspects of using a hierarchy are discussed: one as an informed heuristic exploiting the specific characteristics of indoor environments in order to enhance classical, general-purpose graph search techniques. At the same time, as a convenient by- product of this method, clusters such as sections and wings can be detected. The other reason is to better deal with irregular, complex-shaped regions in a way that instructions can also be provided for these spaces. Previous approaches have not considered this problem. In summary, the main results of this work are: • hierarchical graphs are introduced as a general spatial data infrastructure. In particular, this architecture allows us to integrate different spatial networks originating from different sources. A small but useful set of operations is proposed for integrating these networks. In order to work in a hierarchical model, classical graph algorithms are generalised. This finding also has implications on the possible integration of separate navigation services and systems; • a novel set of core data structures and algorithms have been devised for modelling indoor environments. They cater to the unique characteristics of these environments and can be specifically used to provide enhanced navigation in buildings. Tested on models of several real buildings from our university, some preliminary but promising results were gained from a prototypical implementation and its application on the models

Device-Free, Activity during Daily Life, Recognition Using a Low-Cost Lidar

Device-free or off-body sensing methods, such as Lidar, can be used for location-driven Activities during Daily Life (ADL) recognition without the need for a mobile host such as a human or robot to use on-body location sensors. Because if such an attachment fails, or is not operational (powered up), when such mobile hosts are device free, it still works. Hence, this paper proposes an innovative method for recognizing ADLs using a state-of-art seq2seq Recurrent Neural Network (RNN) model to classify centimeter level accurate location data from a low-cost, 360°rotating 2D Lidar device. We researched, developed, deployed and validated the system. The results indicate that it can provide a centimeter-level localization accuracy of 88% when recognizing 17 targeted location-related daily activities

Indoor Semantic Modelling for Routing: The Two-Level Routing Approach for Indoor Navigation

Humans perform many activities indoors and they show a growing need for indoor navigation, especially in unfamiliar buildings such as airports, museums and hospitals. Complexity of such buildings poses many challenges for building managers and visitors. Indoor navigation services play an important role in supporting these indoor activities. Indoor navigation covers extensive topics such as: 1) indoor positioning and localization; 2) indoor space representation for navigation model generation; 3) indoor routing computation; 4) human wayfinding behaviours; and 5) indoor guidance (e.g., textual directories). So far, a large number of studies of pedestrian indoor navigation have presented diverse navigation models and routing algorithms/methods. However, the major challenge is rarely referred to: how to represent the complex indoor environment for pedestrians and conduct routing according to the different roles and sizes of users. Such complex buildings contain irregular shapes, large open spaces, complicated obstacles and different types of passages. A navigation model can be very complicated if the indoors are accurately represented. Although most research demonstrates feasible indoor navigation models and related routing methods in regular buildings, the focus is still on a general navigation model for pedestrians who are simplified as circles. In fact, pedestrians represent different sizes, motion abilities and preferences (e.g., described in user profiles), which should be reflected in navigation models and be considered for indoor routing (e.g., relevant Spaces of Interest and Points of Interest). In order to address this challenge, this thesis proposes an innovative indoor modelling and routing approach – two-level routing. It specially targets the case of routing in complex buildings for distinct users. The conceptual (first) level uses general free indoor spaces: this is represented by the logical network whose nodes represent the spaces and edges stand for their connectivity; the detailed (second) level focuses on transition spaces such as openings and Spaces of Interest (SOI), and geometric networks are generated regarding these spaces. Nodes of a geometric network refers to locations of doors, windows and subspaces (SOIs) inside of the larger spaces; and the edges represent detailed paths among these geometric nodes. A combination of the two levels can represent complex buildings in specified spaces, which avoids maintaining a largescale complete network. User preferences on ordered SOIs are considered in routing on the logical network, and preferences on ordered Points of Interest (POI) are adopted in routing on geometric networks. In a geometric network, accessible obstacle-avoiding paths can be computed for users with different sizes. To facilitate automatic generation of the two types of network in any building, a new data model named Indoor Navigation Space Model (INSM) is proposed to store connectivity, semantics and geometry of indoor spaces for buildings. Abundant semantics of building components are designed in INSM based on navigational functionalities, such as VerticalUnit(VU) and HorizontalConnector(HC) as vertical and horizontal passages for pedestrians. The INSM supports different subdivision ways of a building in which indoor spaces can be assigned proper semantics. A logical and geometric network can be automatically derived from INSM, and they can be used individually or together for indoor routing. Thus, different routing options are designed. Paths can be provided by using either the logical network when some users are satisfied with a rough description of the path (e.g., the name of spaces), or a geometric path is automatically computed for a user who needs only a detailed path which shows how obstacles can be avoided. The two-level routing approach integrates both logical and geometric networks to obtain paths, when a user provides her/his preferences on SOIs and POIs. For example, routing results for the logical network can exclude unrelated spaces and then derive geometric paths more efficiently. In this thesis, two options are proposed for routing just on the logical network, three options are proposed for routing just on the geometric networks, and seven options for two-level routing. On the logical network, six routing criteria are proposed and three human wayfinding strategies are adopted to simulate human indoor behaviours. According to a specific criterion, space semantics of logical nodes is utilized to assign different weights to logical nodes and edges. Therefore, routing on the logical network can be accomplished by applying the Dijkstra algorithm. If multiple criteria are adopted, an order of criteria is applied for routing according to a specific user. In this way, logical paths can be computed as a sequence of indoor spaces with clear semantics. On geometric networks, this thesis proposes a new routing method to provide detailed paths avoiding indoor obstacles with respect to pedestrian sizes. This method allows geometric networks to be derived for individual users with different sizes for any specified spaces. To demonstrate the use of the two types of network, this thesis tests routing on one level (the logical or the geometric network). Four case studies about the logical network are presented in both simple and complex buildings. In the simple building, no multiple paths lie between spaces A and B, but in the complex buildings, multiple logical paths exist and the candidate paths can be reduced by applying these routing criteria in an order for a user. The relationships of these criteria to user profiles are assumed in this thesis. The proposed geometric routing regarding user sizes is tested with three case studies: 1) routing for pedestrians with two distinct sizes in one space; 2) routing for pedestrians with changed sizes in one space; and 3) a larger geometric network formed by the ones in a given sequence of spaces. The first case shows that a small increase of user size can largely change the accessible path; the second case shows different path segments for distinct sizes can be combined into one geometric path; the third case demonstrates a geometric network can be created ’on the fly’ for any specified spaces of a building. Therefore, the generation and routing of geometric networks are very flexible and fit to given users. To demonstrate the proposed two-level routing approach, this thesis designs five cases. The five cases are distinguished according to the method of model creation (pre-computed or ’on-the-fly’) and model storage (on the client or server). Two of them are realized in this thesis: 1) Case 1 just in the client pre-computes the logical network and derives geometric networks ’on the fly’; 2) Case 2 just in the client pre-computes and stores the logical and geometric networks for certain user sizes. Case 1 is implemented in a desktop application for building managers, and Case 2 is realized as a mobile mock-up for mobile users without an internet connection. As this thesis shows, two-level routing is powerful enough to effectively provide indicative logical paths and/or comprehensive geometric paths, according to different user requirements on path details. In the desktop application, three of the proposed routing options for two-level routing are tested for the simple OTB building and the complex Schiphol Airport building. These use cases demonstrate that the two-level routing approach includes the following merits: It supports routing in different abstraction forms of a building. The INSM model can describe different subdivision results of a building, and it allows two types of routing network to be derived – pure logical and geometric ones. The logical network contains the topology and semantics of indoor spaces, and the geometric network provides accurate geometry for paths. A consistent navigation model is formed with the two networks, i.e., the conceptual and detailed levels. On the conceptual level, it supports routing on a logical network and assists the derivation of a conceptual path (i.e., logical path) for a user in terms of space sequence. Routing criteria are designed based on the INSM semantics of spaces, which can generate logical paths similar to human wayfinding results such as minimizing VerticalUnit or HorizontalConnector. On the detailed level, it considers the size of users and results in obstacle-avoiding paths. By using this approach, geometric networks can be generated to avoid obstacles for the given users and accessible paths are flexibly provided for user demands. This approach can process changes of user size more efficiently, in contrast to routing on a complete geometric network. It supports routing on both the logical and the geometric networks, which can generate geometric paths based on user-specific logical paths, or re-compute logical paths when geometric paths are inaccessible. This computation method is very useful for complex buildings. The two-level routing approach can flexibly provide logical and geometric paths according to user preferences and sizes, and can adjust the generated paths in limited time. Based on the two-level routing approach, this thesis also provides a vision on possible cooperation with other methods. A potential direction is to design more routing options according to other indoor scenarios and user preferences. Extensions of the two-level routing approach, such as other types of semantics, multi-level networks and dynamic obstacles, will make it possible to deal with other routing cases. Last but not least, it is also promising to explore its relationships with indoor guidance, different building subdivisions and outdoor navigation. &nbsp

Indoor Semantic Modelling for Routing:

Humans perform many activities indoors and they show a growing need for indoor navigation, especially in unfamiliar buildings such as airports, museums and hospitals. Complexity of such buildings poses many challenges for building managers and visitors. Indoor navigation services play an important role in supporting these indoor activities. Indoor navigation covers extensive topics such as: 1) indoor positioning and localization; 2) indoor space representation for navigation model generation; 3) indoor routing computation; 4) human wayfinding behaviours; and 5) indoor guidance (e.g., textual directories). So far, a large number of studies of pedestrian indoor navigation have presented diverse navigation models and routing algorithms/methods. However, the major challenge is rarely referred to: how to represent the complex indoor environment for pedestrians and conduct routing according to the different roles and sizes of users. Such complex buildings contain irregular shapes, large open spaces, complicated obstacles and different types of passages. A navigation model can be very complicated if the indoors are accurately represented. Although most research demonstrates feasible indoor navigation models and related routing methods in regular buildings, the focus is still on a general navigation model for pedestrians who are simplified as circles. In fact, pedestrians represent different sizes, motion abilities and preferences (e.g., described in user profiles), which should be reflected in navigation models and be considered for indoor routing (e.g., relevant Spaces of Interest and Points of Interest). In order to address this challenge, this thesis proposes an innovative indoor modelling and routing approach – two-level routing. It specially targets the case of routing in complex buildings for distinct users. The conceptual (first) level uses general free indoor spaces: this is represented by the logical network whose nodes represent the spaces and edges stand for their connectivity; the detailed (second) level focuses on transition spaces such as openings and Spaces of Interest (SOI), and geometric networks are generated regarding these spaces. Nodes of a geometric network refers to locations of doors, windows and subspaces (SOIs) inside of the larger spaces; and the edges represent detailed paths among these geometric nodes. A combination of the two levels can represent complex buildings in specified spaces, which avoids maintaining a largescale complete network. User preferences on ordered SOIs are considered in routing on the logical network, and preferences on ordered Points of Interest (POI) are adopted in routing on geometric networks. In a geometric network, accessible obstacle-avoiding paths can be computed for users with different sizes. To facilitate automatic generation of the two types of network in any building, a new data model named Indoor Navigation Space Model (INSM) is proposed to store connectivity, semantics and geometry of indoor spaces for buildings. Abundant semantics of building components are designed in INSM based on navigational functionalities, such as VerticalUnit(VU) and HorizontalConnector(HC) as vertical and horizontal passages for pedestrians. The INSM supports different subdivision ways of a building in which indoor spaces can be assigned proper semantics. A logical and geometric network can be automatically derived from INSM, and they can be used individually or together for indoor routing. Thus, different routing options are designed. Paths can be provided by using either the logical network when some users are satisfied with a rough description of the path (e.g., the name of spaces), or a geometric path is automatically computed for a user who needs only a detailed path which shows how obstacles can be avoided. The two-level routing approach integrates both logical and geometric networks to obtain paths, when a user provides her/his preferences on SOIs and POIs. For example, routing results for the logical network can exclude unrelated spaces and then derive geometric paths more efficiently. In this thesis, two options are proposed for routing just on the logical network, three options are proposed for routing just on the geometric networks, and seven options for two-level routing. On the logical network, six routing criteria are proposed and three human wayfinding strategies are adopted to simulate human indoor behaviours. According to a specific criterion, space semantics of logical nodes is utilized to assign different weights to logical nodes and edges. Therefore, routing on the logical network can be accomplished by applying the Dijkstra algorithm. If multiple criteria are adopted, an order of criteria is applied for routing according to a specific user. In this way, logical paths can be computed as a sequence of indoor spaces with clear semantics. On geometric networks, this thesis proposes a new routing method to provide detailed paths avoiding indoor obstacles with respect to pedestrian sizes. This method allows geometric networks to be derived for individual users with different sizes for any specified spaces. To demonstrate the use of the two types of network, this thesis tests routing on one level (the logical or the geometric network). Four case studies about the logical network are presented in both simple and complex buildings. In the simple building, no multiple paths lie between spaces A and B, but in the complex buildings, multiple logical paths exist and the candidate paths can be reduced by applying these routing criteria in an order for a user. The relationships of these criteria to user profiles are assumed in this thesis. The proposed geometric routing regarding user sizes is tested with three case studies: 1) routing for pedestrians with two distinct sizes in one space; 2) routing for pedestrians with changed sizes in one space; and 3) a larger geometric network formed by the ones in a given sequence of spaces. The first case shows that a small increase of user size can largely change the accessible path; the second case shows different path segments for distinct sizes can be combined into one geometric path; the third case demonstrates a geometric network can be created ’on the fly’ for any specified spaces of a building. Therefore, the generation and routing of geometric networks are very flexible and fit to given users. To demonstrate the proposed two-level routing approach, this thesis designs five cases. The five cases are distinguished according to the method of model creation (pre-computed or ’on-the-fly’) and model storage (on the client or server). Two of them are realized in this thesis: 1) Case 1 just in the client pre-computes the logical network and derives geometric networks ’on the fly’; 2) Case 2 just in the client pre-computes and stores the logical and geometric networks for certain user sizes. Case 1 is implemented in a desktop application for building managers, and Case 2 is realized as a mobile mock-up for mobile users without an internet connection. As this thesis shows, two-level routing is powerful enough to effectively provide indicative logical paths and/or comprehensive geometric paths, according to different user requirements on path details. In the desktop application, three of the proposed routing options for two-level routing are tested for the simple OTB building and the complex Schiphol Airport building. These use cases demonstrate that the two-level routing approach includes the following merits: It supports routing in different abstraction forms of a building. The INSM model can describe different subdivision results of a building, and it allows two types of routing network to be derived – pure logical and geometric ones. The logical network contains the topology and semantics of indoor spaces, and the geometric network provides accurate geometry for paths. A consistent navigation model is formed with the two networks, i.e., the conceptual and detailed levels. On the conceptual level, it supports routing on a logical network and assists the derivation of a conceptual path (i.e., logical path) for a user in terms of space sequence. Routing criteria are designed based on the INSM semantics of spaces, which can generate logical paths similar to human wayfinding results such as minimizing VerticalUnit or HorizontalConnector. On the detailed level, it considers the size of users and results in obstacle-avoiding paths. By using this approach, geometric networks can be generated to avoid obstacles for the given users and accessible paths are flexibly provided for user demands. This approach can process changes of user size more efficiently, in contrast to routing on a complete geometric network. It supports routing on both the logical and the geometric networks, which can generate geometric paths based on user-specific logical paths, or re-compute logical paths when geometric paths are inaccessible. This computation method is very useful for complex buildings. The two-level routing approach can flexibly provide logical and geometric paths according to user preferences and sizes, and can adjust the generated paths in limited time. Based on the two-level routing approach, this thesis also provides a vision on possible cooperation with other methods. A potential direction is to design more routing options according to other indoor scenarios and user preferences. Extensions of the two-level routing approach, such as other types of semantics, multi-level networks and dynamic obstacles, will make it possible to deal with other routing cases. Last but not least, it is also promising to explore its relationships with indoor guidance, different building subdivisions and outdoor navigation