Neuroadaptive modelling for generating images matching perceptual categories

Brain-computer interfaces enable active communication and execution of a pre-defined set of commands, such as typing a letter or moving a cursor. However, they have thus far not been able to infer more complex intentions or adapt more complex output based on brain signals. Here, we present neuroadaptive generative modelling, which uses a participant's brain signals as feedback to adapt a boundless generative model and generate new information matching the participant's intentions. We report an experiment validating the paradigm in generating images of human faces. In the experiment, participants were asked to specifically focus on perceptual categories, such as old or young people, while being presented with computer-generated, photorealistic faces with varying visual features. Their EEG signals associated with the images were then used as a feedback signal to update a model of the user's intentions, from which new images were generated using a generative adversarial network. A double-blind follow-up with the participant evaluating the output shows that neuroadaptive modelling can be utilised to produce images matching the perceptual category features. The approach demonstrates brain-based creative augmentation between computers and humans for producing new information matching the human operator's perceptual categories.Peer reviewe

Inferring relevance from eye movements with wrong models

Statistical inference forms the backbone of modern science. It is often viewed as giving an objective validation for hypotheses or models. Perhaps for this reason the theory of statistical inference is often derived with the assumption that the "truth" is within the model family. However, in many real-world applications the applied statistical models are incorrect. A more appropriate probabilistic model may be computationally too complex, or the problem to be modelled may be so new that there is little prior information to be incorporated. However, in statistical theory the theoretical and practical implications of the incorrectness of the model family are to a large extent unexplored. This thesis focusses on conditional statistical inference, that is, modeling of classes of future observations given observed data, under the assumption that the model is incorrect. Conditional inference or prediction is one of the main application areas of statistical models which is still lacking a conclusive theoretical justification of Bayesian inference. The main result of the thesis is an axiomatic derivation where, given an incorrect model and assuming that the utility is conditional likelihood, a discriminative posterior yields a distribution on model parameters which best agrees with the utility. The devised discriminative posterior outperforms the classical Bayesian joint likelihood-based approach in conditional inference. Additionally, a theoretically justified expectation maximization-type algorithm is presented for obtaining conditional maximum likelihood point estimates for conditional inference tasks. The convergence of the algorithm is shown to be more stable than in earlier partly heuristic variants. The practical application field of the thesis is inference of relevance from eye movement signals in an information retrieval setup. It is shown that relevance can be predicted to some extent, and that this information can be exploited in a new kind of task, proactive information retrieval. Besides making it possible to design new kinds of engineering applications, statistical modeling of eye tracking data can also be applied in basic psychological research to make hypotheses of cognitive processes affecting eye movements, which is the second application area of the thesis

Self consciousness, self identity, and self knowledge

In this thesis I attempt to give an account of the experiential self: the self of inner experience. I put forward the view that the empirical self is the only self there is, and attribute attempts to identify a Transcendental Self, a Pure Ego, a Mind, or a Soul, to a misunderstanding of the nature of the empirical, experiential self. The postulation of a real, or metaphysical, self is put down to the fact that philosophers have entertained altogether too simple a view of the nature of the empirical self. Their major assumption has been that if we have experience of an empirical self, a real self must be postulated which does the experiencing of the empirical self. On this basis the real self cannot itself become an object of experience without this leading to an infinite regress. My contention is, on the contrary, that the empirical self can itself perform all the functions allotted to the metaphysical self. On this view, the notion of a metaphysical self is redundant, and we are saved the embarrassment of defending the existence of a peculiarly elusive metaphysical entity.The position I have just outlined could quite easily be mistaken for one leading up to a theory like Strawson's in which a self is identified with a man; or Shoemaker's in which a self is described as a non -spiritual substance. In fact it will be seen to differ radically from these two, essentially coincident, approaches. Strawson and Shoemaker, together with most recent philosophers dealing with the problem of self- identity, see the problem exclusively in terms of the identity of other persons; not in the identity of a person for himself. Even when the question of one's own identity arises, this is still taken by them to be a problem of the identity of other persons. Thus on their view if I have to establish my own identity, I must use precisely the same criteria as other people would use if they had to establish my identity. The problem I am concerned with, however, is one about which Strawson is silent: namely, the problem of explaining in what consists our own identity for ourselves. Chapter one opens with a discussion of the significance of the fact that we have first -hand knowledge of what selves are. The Pure Ego Theory and The Serial Theory are rejected for failing to take advantage of the unique position this places us in, to describe what selves are.Since I dispute the widely held Humean view of the empirical self, and since I wish to take cognizance of the fact that we ourselves are the selves to be identified, I undertake an analysis of consciousness itself. Most of chapter one is devoted to that task. The reality of consciousness is defended against arguments denying its existence, and an exclusively empirical analysis of consciousness is offered. This is to protect my programme from the objection that a metaphysical interpretation of the self is only avoided at the cost of introducing a metaphysical interpretation of consciousness. To this end I defend the proposition that consciousness is nothing over and above its several manifestations, and I deny in particular that consciousness consists of acts of awareness. Then again adherence to the notion of consciousness is defended against the reductionist argument that reference to it should be dispensed with, in favour of piecemeal references to its manifestations. It is maintained that it is erroneous to conceive of consciousness as an aggregate of its manifestations.As a preliminary to investigating the constitution of consciousness two opposing points of view are considered in the form put forward by four Nineteenth Century philosophers. The views of Hamilton and Ferrier who contend that consciousness exhibits a duality between subject and object are contrasted with those of Hodgson and James who deny that consciousness has any such "inner duplicity." This short survey leads to the conclusion that an independent analysis of consciousness is needed, in view of the fact that opposing positions are defended by appeals to self -evidence that cancel each other out. What is required, and what I undertake, is to give an analysis of consciousness that is based on empirical facts and eschews any appeal to intuition or self -evidence.Attention is made the subject of study in chapter two, because it supplies immediate evidence of the existence of the bifurcation of elements within consciousness. This effect of attention is investigated in the surmise that it might turn out to be the empirical basis to the alleged duality between subject and object. I attempt to show that attention is operative in all normal forms of consciousness: even when we believe ourselves to be totally inattentive. It transpires that attention reveals the structure of consciousness. It polarizes consciousness into that part of it that is receiving attention and a remaining part that is not. The part receiving attention I refer to as the object of attention, and the part from which attention is withdrawn I refer to as residual consciousness. These two aspects of consciousness are dealt with in chapters two and three, respectively. Evidence from the psychologists William James, and his French contemporary, T.H. Ribot, is adduced to support the analysis of the relation between consciousness and attention.A central distinction to my enterprise is the distinction I draw between what I call "interrogative attention" on the one hand, and "non- interrogative attention' on the other. I argue that these two forms of attention amount to the different ways in which consciousness may be structured. In the case of interrogative attention, in which attention is bestowed without a problem - solving intelligence being brought to bear on the object of attention, the two poles of consciousness -- residual consciousness and object of attention - -are merely severed from each other. In the case of interrogative attention, in which attention is given for the sake of finding something out, the two poles of consciousness are in a systematic relationship with each other. The investigation suggests that the different forms of attention are just the different ways in which consciousness may be structured. There is therefore no suggestion that attention may assume the role of an occult cause. Although the structure of consciousness will vary according to the form of attention, its bi- polarity is a formal feature common to all such structures. No matter what the content, the form of consciousness remains unchanged. There is nothing static about the content of residual consciousness on one side, nor about the object of attention on the other. Both, it is maintained, change ceaselessly through the continuous interchange of elements from one pole of consciousness to the other. An important observation is made to the effect that the changing content of the two poles is not noticed equally at both poles. The nature of the structure of consciousness is revealed to be one in which the changes occurring to the object of attention attract notice, while the changes taking place in the content of residual consciousness pass largely unnoticed. This is shown to be necessarily the case, and the feature of consciousness in question I describe as the directionality of consciousness.Chapter four is the crucial chapter in the work. The two preceding chapters are designed to prepare the ground for the denouement which it presents, and much of the significance of the earlier discussion can only be fully appreciated in its light. In it the view is advanced that the bi -polar structure of consciousness accounts for the duality between subject and object which Hamilton had asserted to be a self- evident feature of consciousness. The essence of the theory is that residual consciousness (which may be said to be, metaphorically speaking, the negative pole of consciousness), is the empirical self for which we have been searching. It is shown to have all the "marks" philosophers have taken to be characteristics of the self. The empirical self is, on this theory, located within consciousness, and for this reason its knowability is guaranteed. At the same time the empirical self is not identified with the content of consciousness, and in this way the difficulty that the self is continuously passing away, which besets the Serial Theory of the Self, is overcome. From this perspective it can be appreciated that the Humean identification of the self with a series of perceptions is the very reverse of the truth. It is tantamount to the identification of the self with what is essentially not -self: viz. the objects of attention (i.e., the wrong pole of consciousness: the positive instead of the negative pole, so to speak).I contrast this theory with its closest rival --the theory that identifies the self with the mass of somatic feeling - -and demonstrate its manifest superiority to the latter theory. Objections to the plausibility of the proposed identification of the self with residual consciousness are considered and answered. It is shown to share the merit, which is particularly claimed by the somatic feeling theory, of enabling us to maintain that even an isolated perception can be said to be the perception of a self, quite independently of its relation to other perceptions. In this way it overcomes a major stumbling block to theories of the empirical self which in general permit of no such claim. At the same tine this feature of the theory destroys the basis of one of the strongest arguments in favour of a Pure Ego Theory: namely the supposai that only on a Pure Ego Theory could it be maintained that one isolated perception could be the perception of a self. Finally the implications of the theory on the question of self - consciousness are drawn out, and the limits to complete self - objectification are explained.Perhaps the single greatest difficulty for a theory that locates the self within experience, is to account for self- identity through time: in other words, the problem of the continuity of the self. A theory identifying the self with residual consciousness must seem particularly vulnerable in this respect, because of the fact that residual consciousness has an ever - changing content. This problem is taken up in chapter five. The Serial Theory is subjected to close examination, because it is offered as a solution to this problem. It is shown to be unsatisfactory, because of its dependence on the Humean view of consciousness, which, as I have already indicated, I dispute. I try to show that my analysis of consciousness escapes the atomistic implications of the Serial Theory, and demands instead that consciousness be seen as a continuum.The argument put forward to establish the fact that consciousness is continuous, is dependent on a subsidiary thesis that runs parallel to the main one throughout the work. It is the contention that states of consciousness come into being in conjunction with the bodily activities involving the use of our sense- organs. In the present context this thesis is brought in to explain the continuity of consciousness, in the following manner. I show that states of consciousness are sustained by bodily activities, and reason that since bodily activities are continuous while they last, the states of consciousness they sustain must themselves be continuous for as long as the bodily activity continues. The inference is then drawn that since residual consciousness is a part of consciousness, the continuousness of consciousness must be reflected in residual consciousness as well. Thus, on the basis of the identification of the self with residual consciousness, the continuity of the self is assured. Memory is then brought in to explain our awareness of our self- continuity.The thesis concludes with a sketch of some of the wider implications of my findings. It is argued that the relation between bodily activity and consciousness demands that the self be conceived of as embodied. And finally it is suggested that the theory offers the prospect of a reconciliation between Realism and Idealism; if not in the main, at least in respect of the issue of the active versus the passive self in perceptual experience

Multi-sensor human action recognition with particular application to tennis event-based indexing

The ability to automatically classify human actions and activities using vi- sual sensors or by analysing body worn sensor data has been an active re- search area for many years. Only recently with advancements in both fields and the ubiquitous nature of low cost sensors in our everyday lives has auto- matic human action recognition become a reality. While traditional sports coaching systems rely on manual indexing of events from a single modality, such as visual or inertial sensors, this thesis investigates the possibility of cap- turing and automatically indexing events from multimodal sensor streams. In this work, we detail a novel approach to infer human actions by fusing multimodal sensors to improve recognition accuracy. State of the art visual action recognition approaches are also investigated. Firstly we apply these action recognition detectors to basic human actions in a non-sporting con- text. We then perform action recognition to infer tennis events in a tennis court instrumented with cameras and inertial sensing infrastructure. The system proposed in this thesis can use either visual or inertial sensors to au- tomatically recognise the main tennis events during play. A complete event retrieval system is also presented to allow coaches to build advanced queries, which existing sports coaching solutions cannot facilitate, without an inordi- nate amount of manual indexing. The event retrieval interface is evaluated against a leading commercial sports coaching tool in terms of both usability and efficiency

Pattern Formation and Organization of Epithelial Tissues

Developmental biology is a study of how elaborate patterns, shapes, and functions emerge as an organism grows and develops its body plan. From the physics point of view this is very much a self-organization process. The genetic blueprint contained in the DNA does not explicitly encode shapes and patterns an animal ought to make as it develops from an embryo. Instead, the DNA encodes various proteins which, among other roles, specify how different cells function and interact with each other. Epithelial tissues, from which many organs are sculpted, serve as experimentally- and analytically-tractable systems to study patterning mechanisms in animal development. Despite extensive studies in the past decade, the mechanisms that shape epithelial tissues into functioning organs remain incompletely understood. This thesis summarizes various studies we have done on epithelial organization and patterning, both in abstract theory and in close contact with experiments. A novel mechanism to establish cellular left-right asymmetry based on planar polarity instabilities is discussed. Tissue chirality is often assumed to originate from handedness of biological molecules. Here we propose an alternative where it results from spontaneous symmetry breaking of planar polarity mechanisms. We show that planar cell polarity (PCP), a class of well-studied mechanisms that allows epithelia to spontaneously break rotational symmetry, is also generically capable of spontaneously breaking reflection symmetry. Our results provide a clear interpretation of many mutant phenotypes, especially those that result in incomplete inversion. To bridge theory and experiments, we develop quantitative methods to analyze fluorescence microscopy images. Included in this thesis are algorithms to selectively project intensities from a surface in z-stack images, analysis of cells forming short chain fragments, analysis of thick fluorescent bands using steerable ridge detector, and analysis of cell recoil in laser ablation experiments. These techniques, though developed in the context of zebrafish retina mosaic, are general and can be adapted to other systems. Finally we explore correlated noise in morphogenesis of fly pupa notum. Here we report unexpected correlation of noise in cell movements between left and right halves of developing notum, suggesting that feedback or other mechanisms might be present to counteract stochastic noise and maintain left-right symmetry.PHDPhysicsUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/138800/1/hjeremy_1.pd

Multisensory self-motion processing in humans

Humans obtain and process sensory information from various modalities to ensure successful navigation through the environment. While visual, vestibular, and auditory self-motion perception have been extensively investigated, studies on tac-tile self-motion perception are comparably rare. In my thesis, I have investigated tactile self-motion perception and its interaction with the visual modality. In one of two behavioral studies, I analyzed the influence of a tactile heading stimulus intro-duced as a distractor on visual heading perception. In the second behavioral study, I analyzed visuo-tactile perception of self-motion direction (heading). In both studies, visual self-motion was simulated as forward motion over a 2D ground plane. Tactile self-motion was simulated by airflow towards the subjects’ forehead, mimicking the experience of travel wind, e.g., during a bike ride. In the analysis of the subjects’ perceptual reports, I focused on possible visuo-tactile interactions and applied dif-ferent models to describe the integration of visuo-tactile heading stimuli. Lastly, in a functional magnetic resonance imaging study (fMRI), I investigated neural correlates of visual and tactile perception of traveled distance (path integration) and its modu-lation by prediction and cognitive task demands. In my first behavioral study, subjects indicated perceived heading from uni-modal visual (optic flow), unimodal tactile (tactile flow) or from a combination of stimuli from both modalities, simulating either congruent or incongruent heading (bimodal condition). In the bimodal condition, the subjects’ task was to indicate visually perceived heading. Hence, here tactile stimuli were behaviorally irrelevant. In bimodal trials, I found a significant interaction of stimuli from both modalities. Visually perceived heading was biased towards tactile heading direction for an offset of up to 10° between both heading directions. The relative weighting of stimuli from both modalities in the visuo-tactile in-teraction were examined in my second behavioral study. Subjects indicated per-ceived heading from unimodal visual, unimodal tactile and bimodal trials. Here, in bimodal trials, stimuli form both modalities were presented as behaviorally rele-vant. By varying eye- relative to head position during stimulus presentation, possi-ble influences of different reference frames of the visual and tactile modality were investigated. In different sensory modalities, incoming information is encoded rela-tive to the reference system of the receiving sensory organ (e.g., relative to the reti-na in vision or relative to the skin in somatosensation). In unimodal tactile trials, heading perception was shifted towards eye-position. In bimodal trials, varying head- and eye-position had no significant effect on perceived heading: subjects indicated perceived heading based on both, the vis-ual and tactile stimulus, independently of the behavioral relevance of the tactile stimulus. In sum, results of both studies suggest that the tactile modality plays a greater role in self-motion perception than previously thought. Besides the perception of travel direction (heading), information about trav-eled speed and duration are integrated to achieve a measure of the distance trav-eled (path integration). One previous behavioral study has shown that tactile flow can be used for the reproduction of travel distance (Churan et al., 2017). However, studies on neural correlates of tactile distance encoding in humans are lacking en-tirely. In my third study, subjects solved two path integration tasks from unimodal visual and unimodal tactile self-motion stimuli. Brain activity was measured by means of functional magnetic resonance imaging (fMRI). Both tasks varied in the engagement of cognitive task demands. In the first task, subjects replicated (Active trial) a previously observed traveled distance (Passive trial) (= Reproduction task). In the second task, subjects traveled a self-chosen distance (Active trial) which was then recorded and played back to them (Passive trial) (= Self task). The predictive coding theory postulates an internal model which creates predictions about sensory outcomes-based mismatches between predictions and sensory input which enables the system to sharpen future predictions (Teufel et al., 2018). Recent studies sug-gested a synergistical interaction between prediction and cognitive demands, there-by reversing the attenuating effect of prediction. In my study, this hypothesis was tested by manipulating cognitive demands between both tasks. For both tasks, Ac-tive trials compared to Passive trials showed BOLD enhancement of early sensory cortices and suppression of higher order areas (e.g., the intraparietal lobule (IPL)). For both modalities, enhancement of early sensory areas might facilitate task solv-ing processes at hand, thereby reversing the hypothesized attenuating effect of pre-diction. Suppression of the IPL indicates this area as an amodal comparator of pre-dictions and incoming self-motion signals. In conclusion, I was able to show that tactile self-motion information, i.e., tactile flow, provides significant information for the processing of two key features of self-motion perception: Heading and path integration. Neural correlates of tactile path-integration were investigated by means of fMRI, showing similarities between visual and tactile path integration on early processing stages as well as shared neu-ral substrates in higher order areas located in the IPL. Future studies should further investigate the perception of different self-motion parameters in the tactile modali-ty to extend the understanding of this less researched – but important – modality

Rumination and selective attention: An investigation of the impaired disengagement hypothesis

The primary aim of this thesis was to investigate the relationship between rumination and selective attention, in particular, whether the tendency to ruminate is associated with impaired attentional disengagement from negative information. It is well-established that the tendency to ruminate in response to negative mood is a key vulnerability factor in the development of depression (Nolen-Hoekseman, Wisco, & Lyubomirsky, 2008; Watkins, 2008), but attempts to understand the underlying processes contributing to heightened ruminative disposition have been relatively limited. Recently, a number of researchers have suggested that rumination may be characterised by biased attentional processing of negative information, particularly that individuals with high levels of ruminative disposition may have difficulty disengaging their attention from negative information (e.g., Koster, De Lissnyder, Derakshan, & De Raedt, 2011). Studies One and Two each investigated the relationship between individual differences in ruminative disposition and selective attention for negative information, using a modified dot-probe task designed by Grafton, Watkins, and MacLeod (2012) to enable the discrete assessment of biases in attentional engagement and disengagement. Study One found that heightened levels of dispositional ruminative brooding, as assessed by both the Ruminative Responses Scale (RRS; Nolen-Hoeksema & Morrow, 1991) and an in-vivo assessment of ruminative disposition, were associated with impaired attentional disengagement from negative relative to positive information. Similarly, Study Two also found that heightened levels of ruminative disposition were associated with impaired attentional disengagement from negative information, particularly for depression relevant stimuli presented for 1000ms. Study Three sought to extend these findings using an eye-tracking assessment of selective attention to measure the spontaneous allocation of attention between stimuli. However, ruminative disposition was not significantly associated with any index of attentional bias during the eye-tracking assessment, neither with biased attentional disengagement, nor with biased attentional engagement or maintenance of attention. Study Four then sought to replicate findings from Study Two using a selected sample of individuals with high and low levels of ruminative disposition. Participants in the high rumination group demonstrated greater attentional bias for depression relevant negative stimuli presented for 1000ms in comparison to those in the low rumination group. However, this between group difference reflected a general attentional preference for negative relative to positive stimuli (i.e., composite of attentional engagement and disengagement bias), but no specific difference in attentional disengagement bias or attentional engagement bias was observed. Finally, Study Five took a first step towards investigate the causal relationship between rumination and selective attention by investigating the causal effect of rumination on attentional bias. Although there no main effect of induced rumination on attentional bias was observed, the effect of induced rumination on attentional bias was found to be moderated by ruminative disposition. However, contrary to hypotheses, individuals with low levels of ruminative disposition demonstrated an attentional bias for valence-incongruent stimuli, which shifted to a bias for valence-congruent stimuli as ruminative disposition increased. Overall, there was support across the studies for the primary hypothesis that heightened ruminative disposition is associated with impaired attentional disengagement from negative information. However, the findings do not suggest that ruminative disposition is exclusively associated with attentional disengagement bias, but instead indicate that facilitated attentional engagement may also be involved under some circumstances

Primary and secondary processing of a waking subliminal stimulus in REM and non-REM sleep - empirical investigation of a psychoanalytic concept

The present study deals with one of the fundamental psychoanalytic assumptions: the existence of the primary and secondary process. It can be hypothesized that these two different ways of mental functioning differentiate not only conscious and unconscious processes but also REM- and non-REM-sleep. In REM-sleep, when most bizarre dreams occur, the primary process is supposed to be the dominant mode of thinking while non-REM-sleep, when dreams are mainly characterized as thought-like, is supposed to be organized along secondary process lines. So far, only one single study tried to test this hypothesis applying the method of subliminal stimulation on the sleep-dream cycle (Shevrin & Fisher, 1967). In this experimental study we successfully replicated these early findings. Moreover, additional stimuli were used and neurophysiological measurements recorded. Our findings demonstrate that REM- and non-REM-sleep can indeed be differentiated on the basis of their prevailing kind of mental functioning matching the Freudian idea of primary and secondary process thinking