Olfaction-enhanced multimedia: Perspectives and challenges

This is the post-print version of the Article. The official published version can be accessed from the link below - Copyright @ 2011 Springer VerlagOlfaction—or smell—is one of the last challenges which multimedia and multimodal applications have to conquer. Enhancing such applications with olfactory stimuli has the potential to create a more complex—and richer—user multimedia experience, by heightening the sense of reality and diversifying user interaction modalities. Nonetheless, olfaction-enhanced multimedia still remains a challenging research area. More recently, however, there have been initial signs of olfactory-enhanced applications in multimedia, with olfaction being used towards a variety of goals, including notification alerts, enhancing the sense of reality in immersive applications, and branding, to name but a few. However, as the goal of a multimedia application is to inform and/or entertain users, achieving quality olfaction-enhanced multimedia applications from the users’ perspective is vital to the success and continuity of these applications. Accordingly, in this paper we have focused on investigating the user perceived experience of olfaction-enhanced multimedia applications, with the aim of discovering the quality evaluation factors that are important from a user’s perspective of these applications, and consequently ensure the continued advancement and success of olfaction-enhanced multimedia applications

Imposing structure on odor representations during learning in the prefrontal cortex

Animals have evolved sensory systems that afford innate and adaptive responses to stimuli in the environment. Innate behaviors are likely to be mediated by hardwired circuits that respond to invariant predictive cues over long periods of evolutionary time. However, most stimuli do not have innate value. Over the lifetime of an animal, learning provides a mechanism for animals to update the predictive value of cues through experience. Sensory systems must therefore generate neuronal representations that are able to acquire value through learning. A fundamental challenge in neuroscience is to understand how and where value is imposed in brain during learning. The olfactory system is an attractive sensory modality to study learning because the anatomical organization is concise in that there are relatively few synapses separating the sense organ from brain areas implicated in learning. Thus, the circuits for learned olfactory behaviors appear to be relatively shallow and therefore more experimentally accessible than other sensory systems. The goal of this thesis is to characterize the representation and function of neural circuits involved in olfactory associative learning. Odor perception is initiated by the binding of odors onto olfactory receptors expressed in the sensory epithelium. Each olfactory receptor neuron (ORN) expresses one of 1500 different receptor genes, the expression of which pushes the ORN to project with spatial specificity onto a defined loci within the olfactory bulb, the olfactory glomeruli. Therefore, each and every odor evokes a stereotyped map of glomerular activity in the bulb. The projection neurons of the olfactory bulb, mitral and tufted (M/T) cells, send axons to higher brain areas, including a significant input to the primary olfactory cortex, the piriform cortex. Axons from M/T cells project diffusely to the piriform without apparent spatial preference; as a consequence, the spatial order of the bulb is discarded in the piriform. In agreement with anatomical data, electrophysiological and optical imaging studies also demonstrate that individual odorants activate sparse subsets of neurons across the piriform without any spatial order. Moreover, individual piriform neurons exhibit discontinuous receptive fields that defy chemical or perceptual categorization. These observations suggests that piriform neurons receive random subsets of glomerular input. Therefore, odor representations in piriform are unlikely to be hardwired to drive specific behaviors. Rather, this model suggests that value must be imposed upon the piriform through learning. Indeed, the piriform has been shown to be both sufficient and necessary for aversive olfactory learning without affecting innate odor responses. However, how value is imposed on odor representations in the piriform and downstream associational areas remain largely unknown. We first developed a strategy to track neural activity in a population of neurons across multiple days in deep brain areas using 2-photon endoscopic imaging. This allowed us to assay changes in neural responses to odors during learning in piriform and in downstream associative areas. Using this technique, we first observe that piriform odor responses are unaffected by learning, so learning must therefore impose discernable changes in neural activity downstream of piriform. Piriform projects to multiple downstream areas that are implicated in appetitive associative learning, such as the orbitofrontal cortex (OFC). Imaging of neural activity in the OFC reveal that OFC neurons acquire strong responses to conditioned odors (CS+) during learning. Moreover, multiple and distinct CS+ odors activatethe same population of OFC neurons, and these responses are gated by context and internal state. Together, our imaging data shows that an external and sensory representation in the piriform is transformed into an internal and cognitive representation of value in the OFC. Moreover, we found that optogenetic silencing of the OFC impaired the ability of mice to acquire learned associations. Therefore, the robust representation of expected value of the odor cues is necessary for the formation of appetitive associations. We made an important observation: once the task has been learned with a set of odors, the OFC representation decays after learning has plateaued and remains silent even when mice encounter novel odors they haven’t previously experienced. Moreover, silencing the OFC when it was not actively engaged during the subsequent learning of new odors had no effect on learning. These sets of imaging and silencing experiments reveal that the OFC is only important during initial learning; once task structure has been acquired, it is no longer needed. Task performance after initial task acquisition must therefore be accommodated by other brain regions that can store the learned association for long durations. We therefore searched for other brain regions that held learned associations long-term. In the medial prefrontal cortex (mPFC), we observe that the learned representation persists throughout the entire course of training. Unlike the OFC, not only does this representation encode the positive expected value of CS+ odors, it also encodes the negative expected value of CS- odors in a non-overlapping ensemble of neurons. We further show through optogenetic silencing that this representation is necessary for task performance after the task structure has already been acquired. Therefore, while the OFC representation is required for initial task acquisition, the mPFC representation is required for subsequent appetitive learning and performance. Why would a learned representation vanish in the OFC and betransfered elsewhere? We hypothesize that the brain may allocate a portion of its real estate to be a cognitive playground where experimentation and hypothesis testing takes place. Once this area solves a task, it may unload what it has learned to storage units located elsewhere to free up space to learn new tasks. We further imaged another associative area, the basolateral amygdala (BLA), and found a representation of positive value that appears to be generated from a Hebbian learning mechanism. However, the silencing of this representation during learning had no effect. This suggests that while multiple and distributed brain areas encode cues that predict the reward, not all may be necessary for the learning process or for task performance. In summary, we have described a series of experiments that map the representation and function of different associational areas that underlie learning. The data and the techniques employed have the potential to significantly advance the understanding of learned behavior

Active Olfactomotor Responses in Head-Fixed Mice

Autism spectrum disorder (ASD or autism) is a neurodevelopmental condition characterized by deficits in verbal and non-verbal communication skills, narrowed interests, and repetitive behaviors. Altered sensory behaviors, such as abnormal eye tracking, temperature insensitivity, and excessive sniffing, which we will refer to as “olfactomotor” behaviors, have been identified as a common symptom in individuals with autism. Olfactomotor responses, such as sniffing, are respiratory, orofacial, and locomotive movements that allow an organism to sample and react to odors (Esquivelzeta Rabell et al., 2017; Findley et al., 2021; Johnson et al., 2003a; Jones & Urban, 2018; Kurnikova et al., 2019; Wesson et al., 2008). Neurotypical individuals modulate their sniffing behavior when presented with aversive odors, but those with ASD do not despite identifying the odors as unpleasant, suggesting an altered unconscious motor response (Rozenkrantz et al. 2015). To investigate the neural mechanisms underlying olfactomotor sampling, we investigated respiratory and orofacial responses to odor using wildtype mice. Wildtype mice were exposed to 2-phenylethanol (attractive odor), 2-methylbutyric acid (aversive odor), alpha-pinene (neutral odor), or clean air over the course of a behavioral session. We recorded respiration with an intranasal thermistor and track orofacial movements using DeepLabCut. Our preliminary results in wildtype mice (n=3) suggest that mice alter their sniffing and nose movement in response to odor stimuli. This work will shed light on active olfaction and establish the framework for testing autism model mice in the future

Active touch sensing in pinnipeds

Active touch sensing in humans is characterised by making purposive movements with their fingertips. These movements are task-specific to maximise the relevant information gathered from an object. In whisker-touch sensing, previous research has suggested that whisker movements are purposive, but no one has ever examined task-specific whisker movements in any animal. Pinnipeds are whisker specialists, with long, mobile, sensitive whiskers and diverse whisker morphologies. The aim of this PhD is to investigate active touch sensing in Pinnipeds (seals, sea lions and walrus), by: i) describing whisker morphology; ii) comparing and quantifying whisker movements; and iii) characterising task-dependency of whisker movements during texture, size and luminance discrimination tasks. Pinnipeds with long, numerous whiskers, such as California sea lions (Zalophus californianus) and Stellar sea lions (Eumetopias jubatus) have larger infraorbital foramen (IOF) sizes and therefore, more sensitive whiskers. The IOF being a small hole in the skull, allowing the infraorbital nerve (ION) to pass through, which supplies sensation to the whiskers. Comparing whisker movements in Harbor seals (Phoca vitulina), California sea lions and Pacific walrus (Odobenidae rosmarus), showed these species all protracted their whiskers forwards and oriented their head towards a moving fish stimulus. However, California sea lions moved their whiskers more than the other species, and independently of the head. Due to the movement capabilities and sensitivity of whiskers in California sea lions, this species was used to investigate whether whiskers can be moved in a task-specific way. Results suggested that California sea lions make task-specific movements, by feeling around the edge of different-sized shapes, and focussing and spreading their whiskers on the centre of different-textured shapes. Therefore, California sea lion whiskers are controlled like a true active touch sensory system, similar to human fingertips. I suggest that active touch sensing is likely to efficiently guide foraging and prey capture in dark, murky waters in these animals. Moreover, the complexity of California sea lion whisker movements and their subsequent behaviours makes them a good candidate from which to further investigate animal decision-making, perception and cognition

Seven Years after the Manifesto: Literature Review and Research Directions for Technologies in Animal Computer Interaction

As technologies diversify and become embedded in everyday lives, the technologies we expose to animals, and the new technologies being developed for animals within the field of Animal Computer Interaction (ACI) are increasing. As we approach seven years since the ACI manifesto, which grounded the field within Human Computer Interaction and Computer Science, this thematic literature review looks at the technologies developed for (non-human) animals. Technologies that are analysed include tangible and physical, haptic and wearable, olfactory, screen technology and tracking systems. The conversation explores what exactly ACI is whilst questioning what it means to be animal by considering the impact and loop between machine and animal interactivity. The findings of this review are expected to form the first grounding foundation of ACI technologies informing future research in animal computing as well as suggesting future areas for exploratio

Thermal and wind devices for multisensory human-computer interaction: an overview

In order to create immersive experiences in virtual worlds, we need to explore different human senses (sight, hearing, smell, taste, and touch). Many different devices have been developed by both industry and academia towards this aim. In this paper, we focus our attention on the researched area of thermal and wind devices to deliver the sensations of heat and cold against people’s skin and their application to human-computer interaction (HCI). First, we present a review of devices and their features that were identified as relevant. Then, we highlight the users’ experience with thermal and wind devices, highlighting limitations either found or inferred by the authors and studies selected for this survey. Accordingly, from the current literature, we can infer that, in wind and temperature-based haptic systems (i) users experience wind effects produced by fans that move air molecules at room temperature, and (ii) there is no integration of thermal components to devices intended for the production of both cold or hot airflows. Subsequently, an analysis of why thermal wind devices have not been devised yet is undertaken, highlighting the challenges of creating such devices.Espírito Santo Research and Innovation Foundation (FAPES, Brazil) - Finance Code 2021-GL60J), the Coordination for the Improvement of Higher Education Personnel (CAPES, Brazil) - Finance Code 88881.187844/2018-01 and 88887.570688/2020-00 and by the National Council for Scientific and Technological (CNPq, Brazil) - Finance Code 307718/2020-4. The work was also funded by the European Union’s Horizon 2020 Research and Innovation programme under Grant Agreement no. 688503. E. B. Saleme additionally acknowledges aid from the Federal Institute of Espírito Santo

Designing an interactive olfactory robot for and with dogs.

This thesis follows the development of Scent Bot, a smell-based enrichment and training device ecosystem for dogs. The device is designed for dogs to use independently. The sponsor of the project, Nose Academy Oy, gave the design brief. Through the choices made while developing the device, a dog-centric design approach emerges, which is discussed at length. Challenges such as those of linguistics and cognition that arise when designing for another species are mitigated through an iterative, multispecies participatory design process. In addition to differences in comprehension, differences in physiology, and ways of experiencing the world are key elements taken into consideration while designing. The interactions of the dog with the device are based on methods coming from ethology and animal training. The interactions were then tested with dogs and revised based on the test results in an iterative looping manner. The design method used in this thesis forwards the conversation around the involvement of animals in the design process while designing for animal-computer interactions. Such design methods can also be used to understand what participatory design can mean where user groups cannot give direct verbal feedback to the designers such as young children and others who are differently abled. The product finds use both in research related to canine olfaction and commercial applications. If launched now, Scent Bot will be the first commercially available automated olfaction-based interactive enrichment device for dogs in the world

Challenges and advanced concepts for the assessment of learning and memory function in mice

The mechanisms underlying the formation and retrieval of memories are still an active area of research and discussion. Manifold models have been proposed and refined over the years, with most assuming a dichotomy between memory processes involving non-conscious and conscious mechanisms. Despite our incomplete understanding of the underlying mechanisms, tests of memory and learning count among the most performed behavioral experiments. Here, we will discuss available protocols for testing learning and memory using the example of the most prevalent animal species in research, the laboratory mouse. A wide range of protocols has been developed in mice to test, e.g., object recognition, spatial learning, procedural memory, sequential problem solving, operant- and fear conditioning, and social recognition. Those assays are carried out with individual subjects in apparatuses such as arenas and mazes, which allow for a high degree of standardization across laboratories and straightforward data interpretation but are not without caveats and limitations. In animal research, there is growing concern about the translatability of study results and animal welfare, leading to novel approaches beyond established protocols. Here, we present some of the more recent developments and more advanced concepts in learning and memory testing, such as multi-step sequential lockboxes, assays involving groups of animals, as well as home cage-based assays supported by automated tracking solutions; and weight their potential and limitations against those of established paradigms. Shifting the focus of learning tests from the classical experimental chamber to settings which are more natural for rodents comes with a new set of challenges for behavioral researchers, but also offers the opportunity to understand memory formation and retrieval in a more conclusive way than has been attainable with conventional test protocols. We predict and embrace an increase in studies relying on methods involving a higher degree of automatization, more naturalistic- and home cage-based experimental setting as well as more integrated learning tasks in the future. We are confident these trends are suited to alleviate the burden on animal subjects and improve study designs in memory research