Discrete Multi-modal Hashing with Canonical Views for Robust Mobile Landmark Search

Mobile landmark search (MLS) recently receives increasing attention for its great practical values. However, it still remains unsolved due to two important challenges. One is high bandwidth consumption of query transmission, and the other is the huge visual variations of query images sent from mobile devices. In this paper, we propose a novel hashing scheme, named as canonical view based discrete multi-modal hashing (CV-DMH), to handle these problems via a novel three-stage learning procedure. First, a submodular function is designed to measure visual representativeness and redundancy of a view set. With it, canonical views, which capture key visual appearances of landmark with limited redundancy, are efficiently discovered with an iterative mining strategy. Second, multi-modal sparse coding is applied to transform visual features from multiple modalities into an intermediate representation. It can robustly and adaptively characterize visual contents of varied landmark images with certain canonical views. Finally, compact binary codes are learned on intermediate representation within a tailored discrete binary embedding model which preserves visual relations of images measured with canonical views and removes the involved noises. In this part, we develop a new augmented Lagrangian multiplier (ALM) based optimization method to directly solve the discrete binary codes. We can not only explicitly deal with the discrete constraint, but also consider the bit-uncorrelated constraint and balance constraint together. Experiments on real world landmark datasets demonstrate the superior performance of CV-DMH over several state-of-the-art methods

Facilitation of Learning Spatial Relations among Goal Locations does not Require Visual Exposure to the Configuration of Goal Locations

Human participants searched in a virtual-environment open-field search task for four hidden goal locations arranged in a diamond configuration located in a 5 x 5 matrix. Participants were randomly assigned to one of three groups: Pattern Only, Landmark + Pattern, or Cues + Pattern. All participants experienced a Training phase followed by a Testing phase. During Training, visual cues were coincident with goal locations for the Cues + Pattern group, and a single visual cue at a non-goal location maintained a consistent spatial relationship with the goal locations for the Landmark + Pattern group. All groups were then tested in the absence of visual cues. Presence of the visual cue(s) during Training facilitated acquisition of the task, but the Landmark + Pattern group and the Cues + Pattern group did not differ when their visual cues were removed during Testing and performed superior to the Pattern Only group. Results suggest learning based upon the spatial relations among locations may not be susceptible to cue-competition effects and facilitation of learning spatial relations by visual cues does not require visual exposure to the configuration of goal locations

Landmark Visualization on Mobile Maps – Effects on Visual Attention, Spatial Learning, and Cognitive Load during Map-Aided Real-World Navigation of Pedestrians

Even though they are day-to-day activities, humans find navigation and wayfinding to be cognitively challenging. To facilitate their everyday mobility, humans increasingly rely on ubiquitous mobile maps as navigation aids. However, the over-reliance on and habitual use of omnipresent navigation aids deteriorate humans' short-term ability to learn new information about their surroundings and induces a long-term decline in spatial skills. This deterioration in spatial learning is attributed to the fact that these aids capture users' attention and cause them to enter a passive navigation mode. Another factor that limits spatial learning during map-aided navigation is the lack of salient landmark information on mobile maps. Prior research has already demonstrated that wayfinders rely on landmarks—geographic features that stand out from their surroundings—to facilitate navigation and build a spatial representation of the environments they traverse. Landmarks serve as anchor points and help wayfinders to visually match the spatial information depicted on the mobile map with the information collected during the active exploration of the environment. Considering the acknowledged significance of landmarks for human wayfinding due to their visibility and saliency, this thesis investigates an open research question: how to graphically communicate landmarks on mobile map aids to cue wayfinders' allocation of attentional resources to these task-relevant environmental features. From a cartographic design perspective, landmarks can be depicted on mobile map aids on a graphical continuum ranging from abstract 2D text labels to realistic 3D buildings with high visual fidelity. Based on the importance of landmarks for human wayfinding and the rich cartographic body of research concerning their depiction on mobile maps, this thesis investigated how various landmark visualization styles affect the navigation process of two user groups (expert and general wayfinders) in different navigation use contexts (emergency and general navigation tasks). Specifically, I conducted two real-world map-aided navigation studies to assess the influence of various landmark visualization styles on wayfinders' navigation performance, spatial learning, allocation of visual attention, and cognitive load. In Study I, I investigated how depicting landmarks as abstract 2D building footprints or realistic 3D buildings on the mobile map affected expert wayfinders' navigation performance, visual attention, spatial learning, and cognitive load during an emergency navigation task. I asked expert navigators recruited from the Swiss Armed Forces to follow a predefined route using a mobile map depicting landmarks as either abstract 2D building footprints or realistic 3D buildings and to identify the depicted task-relevant landmarks in the environment. I recorded the experts' gaze behavior with a mobile eye-tracer and their cognitive load with EEG during the navigation task, and I captured their incidental spatial learning at the end of the task. The wayfinding experts' exhibited high navigation performance and low cognitive load during the map-aided navigation task regardless of the landmark visualization style. Their gaze behavior revealed that wayfinding experts navigating with realistic 3D landmarks focused more on the visualizations of landmarks on the mobile map than those who navigated with abstract 2D landmarks, while the latter focused more on the depicted route. Furthermore, when the experts focused for longer on the environment and the landmarks, their spatial learning improved regardless of the landmark visualization style. I also found that the spatial learning of experts with self-reported low spatial abilities improved when they navigated with landmarks depicted as realistic 3D buildings. In Study II, I investigated the influence of abstract and realistic 3D landmark visualization styles on wayfinders sampled from the general population. As in Study I, I investigated wayfinders' navigation performance, visual attention, spatial learning, and cognitive load. In contrast to Study I, the participants in Study II were exposed to both landmark visualization styles in a navigation context that mimics everyday navigation. Furthermore, the participants were informed that their spatial knowledge of the environment would be tested after navigation. As in Study I, the wayfinders in Study II exhibited high navigation performance and low cognitive load regardless of the landmark visualization style. Their visual attention revealed that wayfinders with low spatial abilities and wayfinders familiar with the study area fixated on the environment longer when they navigated with realistic 3D landmarks on the mobile map. Spatial learning improved when wayfinders with low spatial abilities were assisted by realistic 3D landmarks. Also, when wayfinders were assisted by realistic 3D landmarks and paid less attention to the map aid, their spatial learning improved. Taken together, the present real-world navigation studies provide ecologically valid results on the influence of various landmark visualization styles on wayfinders. In particular, the studies demonstrate how visualization style modulates wayfinders' visual attention and facilitates spatial learning across various user groups and navigation use contexts. Furthermore, the results of both studies highlight the importance of individual differences in spatial abilities as predictors of spatial learning during map-assisted navigation. Based on these findings, the present work provides design recommendations for future mobile maps that go beyond the traditional concept of "one fits all." Indeed, the studies support the cause for landmark depiction that directs individual wayfinders' visual attention to task-relevant landmarks to further enhance spatial learning. This would be especially helpful for users with low spatial skills. In doing so, future mobile maps could dynamically adapt the visualization style of landmarks according to wayfinders' spatial abilities for cued visual attention, thus meeting individuals' spatial learning needs

How does the design of landmarks on a mobile map influence wayfinding experts’ spatial learning during a real-world navigation task?

Humans increasingly rely on GPS-enabled mobile maps to navigate novel environments. However, this reliance can negatively affect spatial learning, which can be detrimental even for expert navigators such as search and rescue personnel. Landmark visualization has been shown to improve spatial learning in general populations by facilitating object identification between the map and the environment. How landmark visualization supports expert users’ spatial learning during map-assisted navigation is still an open research question. We thus conducted a real-world study with wayfinding experts in an unknown residential neighborhood. We aimed to assess how two different landmark visualization styles (abstract 2D vs. realistic 3D buildings) would affect experts’ spatial learning in a map-assisted navigation task during an emergency scenario. Using a between-subjects design, we asked Swiss military personnel to follow a given route using a mobile map, and to identify five task-relevant landmarks along the route. We recorded experts’ gaze behavior while navigating and examined their spatial learning after the navigation task. We found that experts’ spatial learning improved when they focused their visual attention on the environment, but the direction of attention between the map and the environment was not affected by the landmark visualization style. Further, there was no difference in spatial learning between the 2D and 3D groups. Contrary to previous research with general populations, this study suggests that the landmark visualization style does not enhance expert navigators’ navigation or spatial learning abilities, thus highlighting the need for population-specific mobile map design solutions

Learning to Find Eye Region Landmarks for Remote Gaze Estimation in Unconstrained Settings

Conventional feature-based and model-based gaze estimation methods have proven to perform well in settings with controlled illumination and specialized cameras. In unconstrained real-world settings, however, such methods are surpassed by recent appearance-based methods due to difficulties in modeling factors such as illumination changes and other visual artifacts. We present a novel learning-based method for eye region landmark localization that enables conventional methods to be competitive to latest appearance-based methods. Despite having been trained exclusively on synthetic data, our method exceeds the state of the art for iris localization and eye shape registration on real-world imagery. We then use the detected landmarks as input to iterative model-fitting and lightweight learning-based gaze estimation methods. Our approach outperforms existing model-fitting and appearance-based methods in the context of person-independent and personalized gaze estimation

Overcoming Spatial Deskilling Using Landmark-Based Navigation Assistance Systems

Abstract Background The repeated use of navigation assistance systems leads to decreased spatial orienting abilities. Previous studies demonstrated that augmentation of landmarks using auditory navigation instructions can improve incidental spatial learning when driving on a single route through an unfamiliar environment. Objective Based on these results, a series of experiments was conducted to further investigate both the impairment of spatial knowledge acquisition by standard navigation instructions and the positive impact of landmark augmentation in auditory navigation instructions on incidental spatial learning. Method The first Experiment replicated the previous setup in a driving simulator without additional visual route indicators. In a second experiment, spatial knowledge was tested after watching a video depicting assisted navigation along a real-world urban route. Finally, a third Experiment investigated incidental spatial knowledge acquisition when participants actively navigated through an unrestricted real-world,urban environment. Results All three experiments demonstrated better cued-recall performance for participants navigating with landmark-based auditory navigation instructions as compared to standard instructions. Notably, standard instructions were associated with reduced learning of landmarks at navigation relevant intersections as compared to landmarks alongside straight segments and the recognition of novel landmarks. Conclusion The results revealed a suppression of spatial learning by established navigation instructions, which were overcome by landmark-based navigation instructions. This emphasizes the positive impact of auditory landmark augmentation on incidental spatial learning and its generalizability to real-life settings. Application This research is paving the way for navigation assistants that, instead of impairing orienting abilities, incidentally foster spatial learning during every-day navigation. Précis This series of three experiments replicates the suppression of spatial learning by standard navigation instructions and the positive impact of landmark augmentation in auditory navigation instructions on incidental spatial learning during assisted navigation. Three experiments with growing degree of realism revealed the applicability and generalizability to real-life settings

Reinforcement learning of visually guided spatial goal directed movement

A range of visually guided, spatial goal directed tasks are investigated, using a computational neuroethology approach. Animats are embedded within a bounded, 2-D environment, and map a 1-D visual array, through a convolution network, to a topography preserving motor array that stochastically determines the direction of movement. Temporal difference reinforcement learning modifies the convolution network in response to a reinforcement signal received only at the goal location. Three forms of visual coding are compared: multiscale coding, where the visual array is convolved by Laplacian of Gaussian filters at a range of spatial scales before convolution to determine the motor array; rectified multiscale coding, where the multiscale array is split into positive and negative components; and intensity coding, where the unfiltered visual array is convolved to determine the motor array. After learning, animats are examined in terms of performance, behaviour and internal structure. When animats learn to approach a solitary circle, of randomly varying contrast, rectified multiscale coding animats learn to outperform multiscale and intensity coding animats in both independent and coarse scale noise conditions. Analysis of the learned internal structure shows that rectified multiscale filtering facilitates learning by enabling detection of the circle at scales least affected by noise. Cartwright and Collett (1983) showed that honeybees learn the angle subtended by a featureless landmark to guide movement to a food source at a fixed distance from the landmark, and furthermore, when tested with only the edges of the landmark, still search in the same location. In a simulation of this experiment, animats are reinforced for moving to where the angle subtended by a solitary circle falls within a certain range. Rectified multiscale filtering leads to better performing animats, with fewer hidden units, in both independent and coarse scale visual noise conditions, though for different reasons in each case. Only those animats with rectified multiscale filtering, that learn in the presence of coarse scale noise, show similar generalisation to the honeybees. Collett, Cartwright and Smith (1986) trained gerbils to search at locations relative to arrangemments of landmarks and tested their search patterns in modifications of the training arrangements. These experiments are simulated with landmark distance coded as either a 1-D intensity array, or a 2-D vector array, plus a simple compass sense. Vector coding animats significantly outperform those using intensity coding and do so with fewer hidden units. Furthermore, vector coding animats show a close match to gerbil behaviour in tests with modified landmark arrangements

The Role of Landscapes and Landmarks in Bee Navigation: A Review.

The ability of animals to explore landmarks in their environment is essential to their fitness. Landmarks are widely recognized to play a key role in navigation by providing information in multiple sensory modalities. However, what is a landmark? We propose that animals use a hierarchy of information based upon its utility and salience when an animal is in a given motivational state. Focusing on honeybees, we suggest that foragers choose landmarks based upon their relative uniqueness, conspicuousness, stability, and context. We also propose that it is useful to distinguish between landmarks that provide sensory input that changes ("near") or does not change ("far") as the receiver uses these landmarks to navigate. However, we recognize that this distinction occurs on a continuum and is not a clear-cut dichotomy. We review the rich literature on landmarks, focusing on recent studies that have illuminated our understanding of the kinds of information that bees use, how they use it, potential mechanisms, and future research directions

Familiarity-dependent computational modelling of indoor landmark selection for route communication: a ranking approach

Landmarks play key roles in human wayfinding and mobile navigation systems. Existing computational landmark selection models mainly focus on outdoor environments, and aim to identify suitable landmarks for guiding users who are unfamiliar with a particular environment, and fail to consider familiar users. This study proposes a familiarity-dependent computational method for selecting suitable landmarks for communicating with familiar and unfamiliar users in indoor environments. A series of salience measures are proposed to quantify the characteristics of each indoor landmark candidate, which are then combined in two LambdaMART-based learning-to-rank models for selecting landmarks for familiar and unfamiliar users, respectively. The evaluation with labelled landmark preference data by human participants shows that people’s familiarity with environments matters in the computational modelling of indoor landmark selection for guiding them. The proposed models outperform state-of-the-art models, and achieve hit rates of 0.737 and 0.786 for familiar and unfamiliar users, respectively. Furthermore, semantic relevance of a landmark candidate is the most important measure for the familiar model, while visual intensity is most informative for the unfamiliar model. This study enables the development of human-centered indoor navigation systems that provide familiarity-adaptive landmark-based navigation guidance