Acquisition and consolidation of hierarchical representations of space

Navigation – the ability to reach targets which are no visible from the current position - depends on the correct recall of the desired target and the environment between one's current position and this target. The content of these representations are subject to influences from different modalities, e.g. vision, and language. A place can be recognized through different cues, e.g. due to a salient object, but also because of the angles of the routes at an intersection, or a name. The location of places as well as the routes connecting them can be integrated and memorized in an allocentric, survey-like representation. Depending on the amount of detail, the granularity level of a representation can be coarser or finer; the different levels are organized hierarchically. Characteristics of a superordinate category, like a region, can affect the perception of its constituting elements, the places; an inheritance of qualities from region to place levels is possible. The formation of superordinate categories depends both on environmental factors as well as individual ones: what is recognized, what is remembered, and which predictions are drawn from this representation? In this dissertation I examine the acquisition of representations of space, in order to identify features that are well suited for being remembered and auxiliary for navigation. I have two research foci: First, I examine the impact of language by using different hierarchically structured naming schemes as place names. Wiener & Mallot (2003) found that characterizing places only with landmarks belonging to different semantic categories influenced route choice as well as the representations of space. I compare these findings to the impact of different naming schemes. I show that there are naming schemes that may influence behavior in a similar way as a landmark does, but that seeing something and reading its name is by far the same thing. The second part focuses on the content of representations established during navigation. With three different navigation experiments, I examine the content of the concepts of space that are acquired during navigation. What is remembered - the location of places, the routes, or the hierarchical structure of the experimental environment? Are there features that are more likely to be consolidated during sleep, e.g., the transfer of concrete knowledge about places and routes into an abstract, survey-like representation? I show that there are improvements in one wayfinding task correlated to sleep. In the other experiments, learning effects were found for both groups. I also address the question of suitable parameters for measuring survey knowledge

Theoretical Computational Models for the Cognitive Map

In den letzten Jahrzehnten hat die Forschung nach der Frage, wie Raum im Gehirn repräsentiert wird, ein weit verzweigtes Netzwerk von spezialisierten Zellen aufgedeckt. Es ist nun klar, dass Räumlichkeit auf irgendeine Art repräsentiert sein muss, aber die genaue Umsetzung wird nach wie vor debattiert. Folgerichtig liegt das übergeordnete Ziel meiner Dissertation darin, das Verständnis von der neuronalen Repräsentation, der Kognitiven Karte, mithilfe von theoretischer Computermodellierung (im Gegensatz zu datengetriebener Modellierung) zu erweitern. Die Arbeit setzt sich aus vier Publikationen zusammen, die das Problem aus verschiedenen, aber miteinander kompatiblen Richtungen angehen: In den ersten beiden Publikationen geht es um zielgerichtete Navigation durch topologische Graphen, in denen die erkundete Umgebung als Netzwerk aus loka len Positionen und sie verbindenden Handlungen dargestellt wird. Im Gegensatz zu Koordinaten-basierten metrischen Karten sind Graphenmodelle weniger gebunden und haben verschiedene Vorteile wie z.B. Algorithmen, die garantiert optimale Pfade finden. Im ersten Modell sind Orte durch Populationen von einfachen Bildfeatures im Graphen gespeichert. Für die Navigation werden dann mehrere Pfade gleichzeitig zwischen Start- und Zielpopulationen berechnet und die schlussendliche Route folgt dem Durchschnitt der Pfade. Diese Methode macht die Wegsuche robuster und umgeht das Problem, Orte entlang der Route wiedererkennen zu müssen. In der zweiten Publikation wird ein hierarchisches Graphenmodell vorgeschlagen, bei dem die Umgebung in mehrere Regionen unterteilt ist. Das Regionenwissen ist ebenfalls als übergeordnete Knoten im Graphen gespeichert. Diese Struktur führt bei der Wegsuche dazu, dass die berechneten Routen verzerrt sind, was mit dem Verhalten von menschlichen Probanden in Navigationsstudien übereinstimmt. In der dritten Publikation geht es auch um Regionen, der Fokus liegt aber auf der konkreten biologischen Umsetzung in Form von Place Cell und Grid Cell-Aktivität. Im Gegensatz zu einzigartigen Ortsknoten im Graphen zeigen Place Cells multiple Feuerfelder in verschiedenen Regionen oder Kontexten. Dieses Phänomen wird als Remapping bezeichnet und könnte der Mechanismus hinter Regionenwissen sein. Wir modellieren das Phänomen mithilfe eines Attraktor-Netzwerks aus Place- und Grid Cells: Immer, wenn sich der virtuelle Agent des Modells von einer Region in eine andere bewegt, verändert sich der Kontext und die Zellaktivität springt zu einem anderen Attraktor, was zu einem Remapping führt. Das Modell kann die Zellaktivität von Tieren in mehreren Experimentalumgebungen replizieren und ist daher eine plausible Erklärung für die Vorgänge im biologischen Gehirn. In der vierten Publikation geht es um den Vergleich von Graphen- und Kartenmodellen als fundamentale Struktur der kognitiven Karte. Im Speziellen geht es bei dieser Debatte um die Unterscheidung zwischen nicht-metrischen Graphen und metrischen euklidischen Karten; euklidische Karten sind zwar mächtiger als die Alternative, aber menschliche Probanden neigen dazu, Fehler zu machen, die stark von einer metrischen Vorhersage abweichen. Deshalb wird häufig argumentiert, dass nicht-metrische Modelle das Verhalten besser erklären können. Wir schlagen eine alternative metrische Erklärung für die nichtmetrischen Graphen vor, indem wir die Graphen im metrischen Raum einbetten. Die Methode wird in einer bestimmten nicht-euklidischen Beispielumgebung gezeigt, in der sie Versuchspersonenverhalten genauso gut vorhersagen kann, wie ein nichtmetrischer Graph. Wir argumentieren daher, dass unser Modell ein besseres Modell für Raumrepräsentation sein könnte. Zusätzlich zu den Einzelergebnissen diskutiere ich außerdem die Gemeinsamkeiten der Modelle und wie sie in den derzeitigen Stand der Forschung zur kognitiven Karte passen. Darüber hinaus erörtere ich, wie die Ergebnisse zu komplexeren Modellen vereint werden könnten, um unser Bild der Raumkognition zu erweitern.Decades of research into the neural representation of physical space have uncovered a complex and distributed network of specialized cells in the mammalian brain. It is now clear that space is represented in some form, but the realization remains debated. Accordingly, the overall aim of my thesis is to further the understanding of the neural representation of space, the cognitive map, with the aid of theoretical computational modeling (as opposed to data-driven modeling). It consists of four separate publications which approach the problem from different but complementing perspectives: The first two publications consider goal-directed navigation with topological graph models, which encode the environment as a state-action graph of local positions connected by simple movement instructions. Graph models are often less constrained than coordinate-based metric maps and offer a variety of computational advantages; for example, graph search algorithms may be used to derive optimal routes between arbitrary positions. In the first model, places are encoded by population codes of low-level image features. For goal-directed navigation, a set of simultaneous paths is obtained between the start and goal populations and the final trajectory follows the population average. This makes route following more robust and circumvents problems related to place recognition. The second model proposes a hierarchical place graph which subdivides the known environment into well-defined regions. The region knowledge is included in the graph as superordinate nodes. During wayfinding, these nodes distort the resulting paths in a way that matches region-related biases observed in human navigation experiments. The third publication also considers region coding but focuses on more concrete biological implementation in the form of place cell and grid cell activity. As opposed to unique nodes in a graph, place cells may express multiple firing fields in different contexts or regions. This phenomenon is known as “remapping” and may be fundamental to the encoding region knowledge. The dynamics are modeled in a joint attractor neural network of place and grid cells: Whenever a virtual agent moves into another region, the context changes and the model remaps the cell activity to an associated pattern from memory. The model is able to replicate experimental findings in a series of mazes and may therefore be an explanation for the observed activity in the biological brain. The fourth publication again returns to graph models, joining the debate on the fundamental structure of the cognitive map: The internal representation of space has often been argued to either take the form of a non-metric topological graph or a Euclidean metric map in which places are assigned specific coordinates. While the Euclidean map is more powerful, human navigation in experiments often strongly deviates from a (correct) metric prediction, which has been taken as an argument for the non-metric alternative. However, it may also be possible to find an alternative metric explanation to the non-metric graphs by embedding the latter into metric space. The method is shown with a specific non-Euclidean example environment where it can explain subject behavior equally well to the purely non-metric graph, and it is argued that it is therefore a better model for spatial knowledge. Beyond the individual results, the thesis discusses the commonalities of the models and how they compare to current research on the cognitive map. I also consider how the findings may be combined into more complex models to further the understanding of the cognitive neuroscience of space

From cognitive maps to spatial schemas

A schema refers to a structured body of prior knowledge that captures common patterns across related experiences. Schemas have been studied separately in the realms of episodic memory and spatial navigation across different species and have been grounded in theories of memory consolidation, but there has been little attempt to integrate our understanding across domains, particularly in humans. We propose that experiences during navigation with many similarly structured environments give rise to the formation of spatial schemas (for example, the expected layout of modern cities) that share properties with but are distinct from cognitive maps (for example, the memory of a modern city) and event schemas (such as expected events in a modern city) at both cognitive and neural levels. We describe earlier theoretical frameworks and empirical findings relevant to spatial schemas, along with more targeted investigations of spatial schemas in human and non-human animals. Consideration of architecture and urban analytics, including the influence of scale and regionalization, on different properties of spatial schemas may provide a powerful approach to advance our understanding of spatial schemas

Evidence from London taxi drivers of hierarchical route planning in a real-world environment

The ability to navigate a spatial environment strongly depends on how well individuals learn, represent and make use of their knowledge about the environment. In the past, research investigated these aspects separately and often in a virtual environment. The current work studied these three aspects of navigation in a real-real world setting to understand how humans navigate naturally in a complex, urban environment like London, UK. Of particular interest was to determine if there was evidence of hierarchical representations during route planning as found in previous behavioural, neuroscientific or computational studies. Most past studies have explored knowledge for simplistic environments or fragmented knowledge of real-world environments. By contrast, licensed London taxi drivers acquire a unique, almost perfect mental representation of the street network, the location of places and the traffic rules that apply to it. Here, the rare knowledge of these navigation experts was explored in three studies with novel approaches. First, to gain an understanding of the training process of unqualified taxi drivers, information from an interview with a teacher, training lessons and study material was collected, summarised and reported. A range of learning strategies was identified that was linked to theoretical, map-based learning and practical, in-situ experiences of London and pointed towards a segmented planning of routes through subgoal selection. Second, a potential mental segregation of London was studied with qualified taxi drivers through boundary drawings of specific London districts with a paper map to understand a potential hierarchical representation. Higher agreement was found for geographical structures and topically distinct districts surrounded by a linear, almost rectangular street network, whereas agreement was lowest for irregularly shaped districts with similarities to neighbouring areas. Finally, taxi drivers were asked to plan and then verbally recall each street they would take along routes between selected origin destination pairs. Audio recordings of these routes made it possible to relate the response times between individual streets to specific street network properties. The analysis using a linear mixed model indicated slower responses at upcoming turns and entering main roads, whereas boundary streets were recalled faster, as were finial streets when compared to initial street. No effects of Euclidean distance or detours were found. Observations from the training process indicate that a potential segregation of the environment, which might impact on later route planning, might be formed already through specific learning strategies. Faster response times for boundary streets support models in which planning is hierarchical. These findings extend past work on route planning in lab-based networks to real-world city street networks and highlight avenues for future research to explore and make use of real-world data

Neural basis of route-planning and goal-coding during flexible navigation

Animals and humans are remarkable in their ability to flexibly adapt to changes in their surroundings. Navigational flexibility may take many forms and in this thesis we investigate its neural and behavioral underpinnings using a variety of methods and tasks tailored to each specific research aim. These methods include functional resonance magnetic imaging (fMRI), freely moving virtual reality, desktop virtual reality, large-scale online testing, and computational modelling. First, we reanalysed previously collected rodent data in the lab to better under- stand behavioural bias that may occur during goal-directed navigation tasks. Based on finding some biases we designed a new approach of simulating results on maze configurations prior to data collection to select the ideal mazes for our task. In a parallel line of methods development, we designed a freely moving navigation task using large-scale wireless virtual reality in a 10x10 space. We compared human behaviour to that of a select number of reinforcement learning agents to investigate the feasibility of computational modelling approaches to freely moving behaviour. Second, we further developed our new approach of simulating results on maze configuration to design a novel spatial navigation task used in a parallel experiment in both rats and humans. We report the human findings using desktop virtual reality and fMRI. We identified a network of regions including hippocampal, caudate nu- cleus, and lateral orbitofrontal cortex involvement in learning hidden goal locations. We also identified a positive correlation between Euclidean goal distance and brain activity in the caudate nucleus during ongoing navigation. Third, we developed a large online testing paradigm to investigate the role of home environment on wayfinding ability. We extended previous reports that street network complexity is beneficial in improving wayfinding ability as measured using a previously reported virtual navigation game, Sea Hero Quest, as well as in a novel virtual navigation game, City Hero Quest. We also report results of a navigational strategies questionnaire that highlights differences of growing up inside and outside cities in the United States and how this relates to wayfinding ability. Fourth, we investigate route planning in a group of expert navigators, licensed London taxi drivers. We designed a novel mental route planning task, probing 120 different routes throughout the extensive street network of London. We find hip- pocampal and retrosplenial involvement in route planning. We also identify the frontopolar cortex as one of several brain regions parametrically modulated by plan- ning demand. Lastly, I summarize the findings from these studies and how they all come to provide different insights into our remarkable ability to flexibly adapt to naviga- tional challenges in our environment

An Epiduroscopy Simulator Based on a Serious Game for Spatial Cognitive Training (EpiduroSIM): User-Centered Design Approach

Background: Performing high-level surgeries with endoscopy is challenging, and hence, an efficient surgical training method or system is required. Serious game-based simulators can provide a trainee-centered educational environment unlike traditional teacher-centered education environments since serious games provide a high level of interaction (feedback that induces learning). Objective: This study aimed to propose an epiduroscopy simulator, EpiduroSIM, based on a serious game for spatial cognitive training. Methods: EpiduroSIM was designed based on a serious game. For spatial cognitive training, the virtual environment of EpiduroSIM was modeled based on a cognitive map. Results: EpiduroSIM was developed considering user accessibility to provide various functions. The experiment for the validation of EpiduroSIM focused on psychological fidelity and repetitive training effects. The experiments were conducted by dividing 16 specialists into 2 groups of 8 surgeons. The group was divided into beginner and expert based on their epiduroscopy experience. The psychological fidelity of EpiduroSIM was confirmed through the training results of the expert group rather than the beginner group. In addition, the repetitive training effect of EpiduroSIM was confirmed by improving the training results in the beginner group. Conclusions: EpiduroSIM may be useful for training beginner surgeons in epiduroscopy.ope

Personal Wayfinding Assistance

We are traveling many different routes every day. In familiar environments it is easy for us to find our ways. We know our way from bedroom to kitchen, from home to work, from parking place to office, and back home at the end of the working day. We have learned these routes in the past and are now able to find our destination without having to think about it. As soon as we want to find a place beyond the demarcations of our mental map, we need help. In some cases we ask our friends to explain us the way, in other cases we use a map to find out about the place. Mobile phones are increasingly equipped with wayfinding assistance. These devices are usually at hand because they are handy and small, which enables us to get wayfinding assistance everywhere where we need it. While the small size of mobile phones makes them handy, it is a disadvantage for displaying maps. Geographic information requires space to be visualized in order to be understandable. Typically, not all information displayed in maps is necessary. An example are walking ways in parks for car drivers, they are they are usually no relevant route options. By not displaying irrelevant information, it is possible to compress the map without losing important information. To reduce information purposefully, we need information about the user, the task at hand, and the environment it is embedded in. In this cumulative dissertation, I describe an approach that utilizes the prior knowledge of the user to adapt maps to the to the limited display options of mobile devices with small displays. I focus on central questions that occur during wayfinding and relate them to the knowledge of the user. This enables the generation of personal and context-specific wayfinding assistance in the form of maps which are optimized for small displays. To achieve personalized assistance, I present algorithmic methods to derive spatial user profiles from trajectory data. The individual profiles contain information about the places users regularly visit, as well as the traveled routes between them. By means of these profiles it is possible to generate personalized maps for partially familiar environments. Only the unfamiliar parts of the environment are presented in detail, the familiar parts are highly simplified. This bears great potential to minimize the maps, while at the same time preserving the understandability by including personally meaningful places as references. To ensure the understandability of personalized maps, we have to make sure that the names of the places are adapted to users. In this thesis, we study the naming of places and analyze the potential to automatically select and generate place names. However, personalized maps only work for environments the users are partially familiar with. If users need assistance for unfamiliar environments, they require complete information. In this thesis, I further present approaches to support uses in typical situations which can occur during wayfinding. I present solutions to communicate context information and survey knowledge along the route, as well as methods to support self-localization in case orientation is lost

Simulated cognitive topologies: automatically generating highly contextual maps for complex journeys

As people traverse complex journeys, they engage in a number of information interactions across spatial scales and levels of abstraction. Journey complexity is characterised by factors including the number of actions required, and by variation in the contextual basis of reasoning such as a transition between different modes of transport. The high-level task of an A to B journey decomposes into a sequence of lower-level navigational sub-tasks, with the representation of geographic entities that support navigation during, between and across sub-tasks, varying relative to the nature of the task and the character of the geography. For example, transitioning from or to a particular mode of transport has a direct bearing on the natural level of representational abstraction that supports the task, as well as on the overall extent of the task’s region of influence on the traveller’s focus. Modern mobile technologies send data to a device that can in theory be context-specific in terms of explicitly reflecting a traveller’s heterogeneous information requirements, however the extent to which context is explicitly reflected in the selection and display of navigational information remains limited in practice, with a rigid, predetermined scale-based hierarchy of cartographic views remaining the underlying representational paradigm. The core subject of the research is the context-dependent selection and display of navigational information, and while there are many and varied considerations in developing techniques to address selection and display, the central challenge can simply be articulated as how to determine the probability, given the traveller’s current context, that a feature should be in the current map view. Clearly this central challenge extends to all features in the spatial extent, and so from a practical perspective, research questions centre around the initial selection of a subset of features, and around determining an overall probability distribution over the subset given the significance of features within the hierarchically ordered sequence of tasks. In this thesis research is presented around the use of graph structures as a practical basis for modeling urban geography to support heterogenous selections across viewing scales, and ultimately for displaying highly context-specific cartographic views. Through an iterative, empirical research methodology, a formalised approach based on routing networks is presented, which serves as the basis for modeling, selection and display. Findings are presented from a series of 7 situated navigation studies that included research with an existing navigation application as well as experimental research stimuli. Hypotheses were validated and refined over the course of the studies, with a focus on journey-specific regions that form around the navigable network. Empirical data includes sketch maps, textual descriptions, video and device interactions over the course of complex navigation exercises. Study findings support the proposed graph architecture, including subgraph classes that approximate cognitive structures central to natural comprehension and reasoning. Empirical findings lead to the central argument of a model based on causal mechanisms, in which relations are formalised between task, selection and abstraction. A causal framework for automatically determining map content for a given journey context is presented, with the approach involving a conceptual shift from treating geographic features as spatially indexed records, to treating them as variables with a finite number of possible states. Causal nets serve as the practical basis of reasoning, with geographic features being represented by variables in these causal structures. The central challenge of finding the probability that a variable in a causal net is in a particular state is addressed through a causal model in which journey context serves as the evidence that propagates over the net. In this way, complex heterogeneous selections for interactive multi-scale information spaces are expressed as probability distributions determined through message propagation. The thesis concludes with a discussion around the implications of the approach for the presentation of navigational information, and it is shown how the framework can support context-specific selection and disambiguation of map content, demonstrated through the central use case of navigating complex urban journeys

Multi-scale Pedestrian Navigation and Movement in Urban Areas

Sustainable transport planning highlights the importance of walking to low-carbon and healthy urban transport systems. Studies have identified multiple ways in which vehicle traffic can negatively impact pedestrians and inhibit walking intentions. However, pedestrian-vehicle interactions are underrepresented in models of pedestrian mobility. This omission limits the ability of transport simulations to support pedestrian-centric street design. Pedestrian navigation decisions take place simultaneously at multiple spatial scales. Yet most models of pedestrian behaviour focus either on local physical interactions or optimisation of routes across a road network. This thesis presents a novel hierarchical pedestrian route choice framework that integrates dynamic, perceptual decisions at the street level with abstract, network based decisions at the neighbourhood level. The framework is based on Construal Level Theory which states that decision makers construe decisions based on their psychological distance from the object of the decision. The route choice framework is implemented in a spatial agent-based simulation in which pedestrian and vehicle agents complete trips in an urban environment. Global sensitivity analysis is used to explore the behaviour produced by the multi-scale pedestrian route choice model. Finally, simulation experiments are used to explore the impacts of restrictions to pedestrian movement. The results demonstrate the potential insights that can be gained by linking street scale movement and interactions with neighbourhood level mobility patterns