Intelligent Robotic Sonographer: Mutual Information-based Disentangled Reward Learning from Few Demonstrations

Ultrasound (US) imaging is widely used for biometric measurement and diagnosis of internal organs due to the advantages of being real-time and radiation-free. However, due to high inter-operator variability, resulting images highly depend on operators' experience. In this work, an intelligent robotic sonographer is proposed to autonomously "explore" target anatomies and navigate a US probe to a relevant 2D plane by learning from expert. The underlying high-level physiological knowledge from experts is inferred by a neural reward function, using a ranked pairwise image comparisons approach in a self-supervised fashion. This process can be referred to as understanding the "language of sonography". Considering the generalization capability to overcome inter-patient variations, mutual information is estimated by a network to explicitly extract the task-related and domain features in latent space. Besides, a Gaussian distribution-based filter is developed to automatically evaluate and take the quality of the expert's demonstrations into account. The robotic localization is carried out in coarse-to-fine mode based on the predicted reward associated to B-mode images. To demonstrate the performance of the proposed approach, representative experiments for the "line" target and "point" target are performed on vascular phantom and two ex-vivo animal organ phantoms (chicken heart and lamb kidney), respectively. The results demonstrated that the proposed advanced framework can robustly work on different kinds of known and unseen phantoms

Prodigious polyphyly in Pleuroceridae (Gastropoda: Cerithioidea)

Phylogenomic studies with hundreds or thousands of loci are rare for most invertebrate groups, including freshwater gastropods. This can prevent understanding of phylogeny, which hinders many areas of research. Pleuroceridae is a family of freshwater snails that is highly imperiled and plays an essential role in the ecology of many freshwater systems of the eastern United States. However, the evolutionary history of the family is not understood, and the systematics of the family has not been revised in a modern framework. Pleurocerids display a variety of egg-deposition behaviors and shell shapes, making the family an ideal system for studying evolution of invertebrate life history and morphology. However, past mitochondrial-based phylogenetic analyses have failed to produce meaningful phylogenetic hypotheses, preventing conclusions about pleurocerid systematics and evolution. Here, we generated a novel anchored hybrid enrichment probe set with phylogenetic utility for Pleuroceridae. We sampled pleurocerids from across their range to test the probe set and generated a backbone phylogeny. Our analyses uncovered striking levels of polyphyly among currently accepted genera. Numerous species were also polyphyletic, indicative of unrecognized diversity. Phylogenetic patterns also revealed considerable convergence of shell morphologies. In contrast, anatomical and life history features appeared to be much less homoplastic. Despite generic paraphyly, high support for most major clades and phylogenetic cohesiveness of non-shell characters indicate utility of the AHE probe set for studying pleurocerid evolution

Interregional compensatory mechanisms of motor functioning in progressing preclinical neurodegeneration.

Understanding brain reserve in preclinical stages of neurodegenerative disorders allows determination of which brain regions contribute to normal functioning despite accelerated neuronal loss. Besides the recruitment of additional regions, a reorganisation and shift of relevance between normally engaged regions are a suggested key mechanism. Thus, network analysis methods seem critical for investigation of changes in directed causal interactions between such candidate brain regions. To identify core compensatory regions, fifteen preclinical patients carrying the genetic mutation leading to Huntington's disease and twelve controls underwent fMRI scanning. They accomplished an auditory paced finger sequence tapping task, which challenged cognitive as well as executive aspects of motor functioning by varying speed and complexity of movements. To investigate causal interactions among brain regions a single Dynamic Causal Model (DCM) was constructed and fitted to the data from each subject. The DCM parameters were analysed using statistical methods to assess group differences in connectivity, and the relationship between connectivity patterns and predicted years to clinical onset was assessed in gene carriers. In preclinical patients, we found indications for neural reserve mechanisms predominantly driven by bilateral dorsal premotor cortex, which increasingly activated superior parietal cortices the closer individuals were to estimated clinical onset. This compensatory mechanism was restricted to complex movements characterised by high cognitive demand. Additionally, we identified task-induced connectivity changes in both groups of subjects towards pre- and caudal supplementary motor areas, which were linked to either faster or more complex task conditions. Interestingly, coupling of dorsal premotor cortex and supplementary motor area was more negative in controls compared to gene mutation carriers. Furthermore, changes in the connectivity pattern of gene carriers allowed prediction of the years to estimated disease onset in individuals. Our study characterises the connectivity pattern of core cortical regions maintaining motor function in relation to varying task demand. We identified connections of bilateral dorsal premotor cortex as critical for compensation as well as task-dependent recruitment of pre- and caudal supplementary motor area. The latter finding nicely mirrors a previously published general linear model-based analysis of the same data. Such knowledge about disease specific inter-regional effective connectivity may help identify foci for interventions based on transcranial magnetic stimulation designed to stimulate functioning and also to predict their impact on other regions in motor-associated networks

Specialized motor-driven dusp1 expression in the song systems of multiple lineages of vocal learning birds.

Mechanisms for the evolution of convergent behavioral traits are largely unknown. Vocal learning is one such trait that evolved multiple times and is necessary in humans for the acquisition of spoken language. Among birds, vocal learning is evolved in songbirds, parrots, and hummingbirds. Each time similar forebrain song nuclei specialized for vocal learning and production have evolved. This finding led to the hypothesis that the behavioral and neuroanatomical convergences for vocal learning could be associated with molecular convergence. We previously found that the neural activity-induced gene dual specificity phosphatase 1 (dusp1) was up-regulated in non-vocal circuits, specifically in sensory-input neurons of the thalamus and telencephalon; however, dusp1 was not up-regulated in higher order sensory neurons or motor circuits. Here we show that song motor nuclei are an exception to this pattern. The song nuclei of species from all known vocal learning avian lineages showed motor-driven up-regulation of dusp1 expression induced by singing. There was no detectable motor-driven dusp1 expression throughout the rest of the forebrain after non-vocal motor performance. This pattern contrasts with expression of the commonly studied activity-induced gene egr1, which shows motor-driven expression in song nuclei induced by singing, but also motor-driven expression in adjacent brain regions after non-vocal motor behaviors. In the vocal non-learning avian species, we found no detectable vocalizing-driven dusp1 expression in the forebrain. These findings suggest that independent evolutions of neural systems for vocal learning were accompanied by selection for specialized motor-driven expression of the dusp1 gene in those circuits. This specialized expression of dusp1 could potentially lead to differential regulation of dusp1-modulated molecular cascades in vocal learning circuits

Structural and functional connectivity between the lateral posterior-pulvinar complex and primary visual cortex in the ferret

The role of higher-order thalamic structures in sensory processing remains poorly understood. Here, we used the ferret (Mustela putorius furo) as a novel model species for the study of the lateral posterior-pulvinar complex (LP/pulvinar) and its structural and functional connectivity with area 17 (primary visual cortex, V1). We found reciprocal anatomical connections between the lateral part of the Lateral Posterior Nucleus of the LP/pulvinar (LPl) and V1. In order to investigate the role of this feedback loop between LPl and V1 in shaping network activity, we determined the functional interactions between LPl and supragranular, granular, and infragranular layers of V1 by recording multiunit activity (MUA) and local field potential (LFP). Coherence was strongest between LPl and supragranular V1 with the most distinct peaks in the delta and alpha frequency bands. Inter-area interaction measured by spike-phase coupling identified the delta frequency band dominated by infragranular V1 and multiple frequency bands that were most pronounced in supragranular V1. This inter-area coupling was differentially modulated by full-field synthetic and naturalistic visual stimulation. We also found that visual responses in LPl were distinct from the ones in V1 in terms of their reliability. Together, our data support a model of multiple communication channels between the LPl and layers of V1 that are enabled by oscillations in different frequency bands. This demonstration of anatomical and functional connectivity between LPl and V1 in ferrets provides a roadmap for studying the interaction dynamics during behavior and a template for identifying activity dynamics of other thalamic feedback loops

Robotic Ultrasound Imaging: State-of-the-Art and Future Perspectives

Ultrasound (US) is one of the most widely used modalities for clinical intervention and diagnosis due to the merits of providing non-invasive, radiation-free, and real-time images. However, free-hand US examinations are highly operator-dependent. Robotic US System (RUSS) aims at overcoming this shortcoming by offering reproducibility, while also aiming at improving dexterity, and intelligent anatomy and disease-aware imaging. In addition to enhancing diagnostic outcomes, RUSS also holds the potential to provide medical interventions for populations suffering from the shortage of experienced sonographers. In this paper, we categorize RUSS as teleoperated or autonomous. Regarding teleoperated RUSS, we summarize their technical developments, and clinical evaluations, respectively. This survey then focuses on the review of recent work on autonomous robotic US imaging. We demonstrate that machine learning and artificial intelligence present the key techniques, which enable intelligent patient and process-specific, motion and deformation-aware robotic image acquisition. We also show that the research on artificial intelligence for autonomous RUSS has directed the research community toward understanding and modeling expert sonographers' semantic reasoning and action. Here, we call this process, the recovery of the "language of sonography". This side result of research on autonomous robotic US acquisitions could be considered as valuable and essential as the progress made in the robotic US examination itself. This article will provide both engineers and clinicians with a comprehensive understanding of RUSS by surveying underlying techniques.Comment: Accepted by Medical Image Analysi

Illuminating tissue organization by imaging the spatial transcriptome

Our bodies consist of a large collection of cells that each have their own function in the organ that they reside in. The cells are grouped by functionality in cell types that arise during development as the result of the gene regulatory network encoded in the genome. With the development of novel single cell technologies, we are starting to understand just how diverse our cells are. In the brain for instance there are at least 3,000 distinguishable types. However, we have little understanding of how all these cell types are spatially organized in the tissue, because conventional labeling and microscopy techniques are incapable of resolving such high complexity in a single experiment. In this thesis I present the development of two methods that can resolve the cellular complexity and spatial organization of mouse and (developmental) human brain samples. These methods are built upon the concept of cyclic RNA labeling with single molecule Fluorescent in situ Hybridization (smFISH) to detect hundreds of gene targets in tissue samples. The resulting RNA localizations can then be used to study spatial gene expression and to identify the cell type of each cell in the sample. The cellular identity and position can then be used to study spatial relationships between cells to understand the tissue architecture. To place the development of these two methods into context, I will first review the field of spatially resolved transcriptomics. I will discuss the methods that are based on microscopy and spatially tagged RNA sequencing, where I will compare their strengths and weaknesses. Then I will present the two projects: Paper I presents the development of a cyclic smFISH protocol called osmFISH that leverages the high detection efficiency of smFISH to measure the gene expression of 33 cell type marker genes in the mouse somatosensory cortex at single cell resolution. We developed the labeling technology, instrumentation and analysis software to enable the study of cellular organization at multiple length scales. Even though osmFISH and related microscopy-based methods generate high quality data they are limited by the spatial throughput so that only small tissue areas can be processed. In paper II I present another method called EEL FISH that uses electrophoresis to transfer the RNA from a 3D tissue section onto a flat surface. The collapsing of one dimension substantially reduces the time needed to image, while retaining the information, so that the complex spatial gene expression profiles of entire mouse brain sections, sub-structures of the human brain and human developmental tissues can be studied. Lastly, I will discuss these results and look at the future of the field of spatially resolved transcriptomics