A haptic-enabled multimodal interface for the planning of hip arthroplasty

Multimodal environments help fuse a diverse range of sensory modalities, which is particularly important when integrating the complex data involved in surgical preoperative planning. The authors apply a multimodal interface for preoperative planning of hip arthroplasty with a user interface that integrates immersive stereo displays and haptic modalities. This article overviews this multimodal application framework and discusses the benefits of incorporating the haptic modality in this area

Compressive Sensing for Dynamic XRF Scanning

X-Ray Fluorescence (XRF) scanning is a widespread technique of high importance and impact since it provides chemical composition maps crucial for several scientific investigations. There are continuous requirements for larger, faster and highly resolved acquisitions in order to study complex structures. Among the scientific applications that benefit from it, some of them, such as wide scale brain imaging, are prohibitively difficult due to time constraints. However, typically the overall XRF imaging performance is improving through technological progress on XRF detectors and X-ray sources. This paper suggests an additional approach where XRF scanning is performed in a sparse way by skipping specific points or by varying dynamically acquisition time or other scan settings in a conditional manner. This paves the way for Compressive Sensing in XRF scans where data are acquired in a reduced manner allowing for challenging experiments, currently not feasible with the traditional scanning strategies. A series of different compressive sensing strategies for dynamic scans are presented here. A proof of principle experiment was performed at the TwinMic beamline of Elettra synchrotron. The outcome demonstrates the potential of Compressive Sensing for dynamic scans, suggesting its use in challenging scientific experiments while proposing a technical solution for beamline acquisition software.Comment: 16 pages, 7 figures, 1 tabl

M2LADS: A System for Generating MultiModal Learning Analytics Dashboards

In this article, we present a Web-based System called M2LADS, which supports the integration and visualization of multimodal data recorded in learning sessions in a MOOC in the form of Web-based Dashboards. Based on the edBB platform, the multimodal data gathered contains biometric and behavioral signals including electroencephalogram data to measure learners' cognitive attention, heart rate for affective measures, visual attention from the video recordings. Additionally, learners' static background data and their learning performance measures are tracked using LOGCE and MOOC tracking logs respectively, and both are included in the Web-based System. M2LADS provides opportunities to capture learners' holistic experience during their interactions with the MOOC, which can in turn be used to improve their learning outcomes through feedback visualizations and interventions, as well as to enhance learning analytics models and improve the open content of the MOOC

Multimodal imaging of human brain activity: rational, biophysical aspects and modes of integration

Until relatively recently the vast majority of imaging and electrophysiological studies of human brain activity have relied on single-modality measurements usually correlated with readily observable or experimentally modified behavioural or brain state patterns. Multi-modal imaging is the concept of bringing together observations or measurements from different instruments. We discuss the aims of multi-modal imaging and the ways in which it can be accomplished using representative applications. Given the importance of haemodynamic and electrophysiological signals in current multi-modal imaging applications, we also review some of the basic physiology relevant to understanding their relationship

A CMOS-Based Lab-on-Chip Array for Combined Magnetic Manipulation and Opto-Chemical Sensing

Accepted versio

Deciphering Organoids: High-Dimensional Analysis of Biomimetic Cultures

Organoids are self-organising stem cell-derived ex vivo cultures widely adopted as biomimetic models of healthy and diseased tissues. Traditional low-dimensional experimental methods such as microscopy and bulk molecular analysis have generated remarkable biological insights from organoids. However, as complex heterocellular systems, organoids are especially well-positioned to take advantage of emerging high-dimensional technologies. In particular, single-cell methods offer considerable opportunities to analyse organoids at unprecedented scale and depth, enabling comprehensive characterisation of cellular processes and spatial organisation underpinning organoid heterogeneity. This review evaluates state-of-the-art analytical methods applied to organoids, discusses the latest advances in single-cell technologies, and speculates on the integration of these two rapidly developing fields

Analysis of cellular heterogeneity in breast cancer by single cell sequencing

Breast cancer is a complex heterogenous 3D ecosystem. The heterogenous composition of breast cancer determines disease progression and treatment responses. Triple receptor negative breast cancer (TNBC) is a distinct subtype with poor clinical outcomes. Deconvolution of spatially-regulated transcriptomic and microenvironmental drivers unique to TNBC offers the potential to reveal new therapeutic vulnerabilities. Single cell RNA sequencing (scRNA-seq) and spatial transcriptomic technologies were applied to three treatment naive patient-derived breast cancer samples. New spatial transcriptomic and scRNA-seq experimental pipelines were established. The new technologies were successfully applied to clinical grade biopsy samples. Cellular heterogeneity within the epithelial and non-epithelial compartment was identified across the three samples. The heterogeneity identified is consistent with the published literature. Knowledge in the theoretical underpinnings for scRNA-seq analysis along with the skills required for data analysis in a small patient cohort were acquired during the DPhil. The application of algebraic topology, manifold learning and graph theory in evaluating and interpreting scRNA-seq has been studied. The computational tools available for integrating spatial transcriptomics and scRNA-seq data were critically appraised. Future perspectives on approachesfor multimodal integration were explored

Fast multi-core based multimodal registration of 2D cross-sections and 3D datasets

<p>Abstract</p> <p>Background</p> <p>Solving bioinformatics tasks often requires extensive computational power. Recent trends in processor architecture combine multiple cores into a single chip to improve overall performance. The Cell Broadband Engine (CBE), a heterogeneous multi-core processor, provides power-efficient and cost-effective high-performance computing. One application area is image analysis and visualisation, in particular registration of 2D cross-sections into 3D image datasets. Such techniques can be used to put different image modalities into spatial correspondence, for example, 2D images of histological cuts into morphological 3D frameworks.</p> <p>Results</p> <p>We evaluate the CBE-driven PlayStation 3 as a high performance, cost-effective computing platform by adapting a multimodal alignment procedure to several characteristic hardware properties. The optimisations are based on partitioning, vectorisation, branch reducing and loop unrolling techniques with special attention to 32-bit multiplies and limited local storage on the computing units. We show how a typical image analysis and visualisation problem, the multimodal registration of 2D cross-sections and 3D datasets, benefits from the multi-core based implementation of the alignment algorithm. We discuss several CBE-based optimisation methods and compare our results to standard solutions. More information and the source code are available from <url>http://cbe.ipk-gatersleben.de</url>.</p> <p>Conclusions</p> <p>The results demonstrate that the CBE processor in a PlayStation 3 accelerates computational intensive multimodal registration, which is of great importance in biological/medical image processing. The PlayStation 3 as a low cost CBE-based platform offers an efficient option to conventional hardware to solve computational problems in image processing and bioinformatics.</p

Structured data abstractions and interpretable latent representations for single-cell multimodal genomics

Single-cell multimodal genomics involves simultaneous measurement of multiple types of molecular data, such as gene expression, epigenetic marks and protein abundance, in individual cells. This allows for a comprehensive and nuanced understanding of the molecular basis of cellular identity and function. The large volume of data generated by single-cell multimodal genomics experiments requires specialised methods and tools for handling, storing, and analysing it. This work provides contributions on multiple levels. First, it introduces a single-cell multimodal data standard — MuData — designed to facilitate the handling, storage and exchange of multimodal data. MuData provides interfaces that enable transparent access to multimodal annotations as well as data from individual modalities. This data structure has formed the foundation for the multimodal integration framework, which enables complex and composable workflows that can be naturally integrated with existing omics-specific analysis approaches. Joint analysis of multimodal data can be performed using integration methods. In order to enable integration of single-cell data, an improved multi-omics factor analysis model (MOFA+) has been designed and implemented building on the canonical dimensionality reduction approach for multi-omics integration. Inferring later factors that explain variation across multiple modalities of the data, MOFA+ enables the modelling of latent factors with cell group-specific patterns of activity. MOFA+ model has been implemented as part of the respective multi-omics integration framework, and its utility has been extended by software solutions that facilitate interactive model exploration and interpretation. The newly improved model for multi-omics integration of single cells has been applied to the study of gene expression signatures upon targeted gene activation. In a dataset featuring targeted activation of candidate regulators of zygotic genome activation (ZGA) — a crucial transcriptional event in early embryonic development, — modelling expression of both coding and non-coding loci with MOFA+ allowed to rank genes by their potency to activate a ZGA-like transcriptional response. With identification of Patz1, Dppa2 and Smarca5 as potent inducers of ZGA-like transcription in mouse embryonic stem cells, these findings have contributed to the understanding of molecular mechanisms behind ZGA and laid the foundation for future research of ZGA in vivo. In summary, this work’s contributions include the development of data handling and integration methods as well as new biological insights that arose from applying these methods to studying gene expression regulation in early development. This highlights how single-cell multimodal genomics can aid to generate valuable insights into complex biological systems