Intraoperative Organ Motion Models with an Ensemble of Conditional Generative Adversarial Networks

In this paper, we describe how a patient-specific, ultrasound-probe-induced prostate motion model can be directly generated from a single preoperative MR image. Our motion model allows for sampling from the conditional distribution of dense displacement fields, is encoded by a generative neural network conditioned on a medical image, and accepts random noise as additional input. The generative network is trained by a minimax optimisation with a second discriminative neural network, tasked to distinguish generated samples from training motion data. In this work, we propose that 1) jointly optimising a third conditioning neural network that pre-processes the input image, can effectively extract patient-specific features for conditioning; and 2) combining multiple generative models trained separately with heuristically pre-disjointed training data sets can adequately mitigate the problem of mode collapse. Trained with diagnostic T2-weighted MR images from 143 real patients and 73,216 3D dense displacement fields from finite element simulations of intraoperative prostate motion due to transrectal ultrasound probe pressure, the proposed models produced physically-plausible patient-specific motion of prostate glands. The ability to capture biomechanically simulated motion was evaluated using two errors representing generalisability and specificity of the model. The median values, calculated from a 10-fold cross-validation, were 2.8+/-0.3 mm and 1.7+/-0.1 mm, respectively. We conclude that the introduced approach demonstrates the feasibility of applying state-of-the-art machine learning algorithms to generate organ motion models from patient images, and shows significant promise for future research.Comment: Accepted to MICCAI 201

The ethics of digital ethnography in a team project

This article draws on researcher vignettes to explore ethical decisions made in the process of collecting and analysing mobile messaging data as part of a team ethnographic project exploring multilingualism in superdiverse UK cities. The research involves observing key participants at work as well as recording them at home and collecting their digitally-mediated interactions. The nature of ethnographic research raises ethical issues which highlight the impossibility of divorcing ethics from project decision-making. We therefore take on board a re-conceptualisation of research ethics not as an external set of guidelines but as the core of research, driving decision-making at all steps of the process. The researcher vignettes on which we draw in exploring this process facilitate a reflective approach and enable us to identify and address ethical issues in our research. In this article, we focus on the potential impact that digital communications technologies can have on the kinds of relationships that are possible between researchers and research participants, and on the roles they can carry out within the project. In doing so, we explore the part that digitally-mediated communications play in the co-construction of social distance and closeness in research relationships. Our discussions around these issues highlight the need for an awareness not only of how our participants’ media ideologies shape their use and perceptions of digital technologies, but also how our own assumptions inform our handling of the digital data

Adversarial Deformation Regularization for Training Image Registration Neural Networks

We describe an adversarial learning approach to constrain convolutional neural network training for image registration, replacing heuristic smoothness measures of displacement fields often used in these tasks. Using minimally-invasive prostate cancer intervention as an example application, we demonstrate the feasibility of utilizing biomechanical simulations to regularize a weakly-supervised anatomical-label-driven registration network for aligning pre-procedural magnetic resonance (MR) and 3D intra-procedural transrectal ultrasound (TRUS) images. A discriminator network is optimized to distinguish the registration-predicted displacement fields from the motion data simulated by finite element analysis. During training, the registration network simultaneously aims to maximize similarity between anatomical labels that drives image alignment and to minimize an adversarial generator loss that measures divergence between the predicted- and simulated deformation. The end-to-end trained network enables efficient and fully-automated registration that only requires an MR and TRUS image pair as input, without anatomical labels or simulated data during inference. 108 pairs of labelled MR and TRUS images from 76 prostate cancer patients and 71,500 nonlinear finite-element simulations from 143 different patients were used for this study. We show that, with only gland segmentation as training labels, the proposed method can help predict physically plausible deformation without any other smoothness penalty. Based on cross-validation experiments using 834 pairs of independent validation landmarks, the proposed adversarial-regularized registration achieved a target registration error of 6.3 mm that is significantly lower than those from several other regularization methods.Comment: Accepted to MICCAI 201

Label-driven weakly-supervised learning for multimodal deformable image registration

Spatially aligning medical images from different modalities remains a challenging task, especially for intraoperative applications that require fast and robust algorithms. We propose a weakly-supervised, label-driven formulation for learning 3D voxel correspondence from higher-level label correspondence, thereby bypassing classical intensity-based image similarity measures. During training, a convolutional neural network is optimised by outputting a dense displacement field (DDF) that warps a set of available anatomical labels from the moving image to match their corresponding counterparts in the fixed image. These label pairs, including solid organs, ducts, vessels, point landmarks and other ad hoc structures, are only required at training time and can be spatially aligned by minimising a cross-entropy function of the warped moving label and the fixed label. During inference, the trained network takes a new image pair to predict an optimal DDF, resulting in a fully-automatic, label-free, real-time and deformable registration. For interventional applications where large global transformation prevails, we also propose a neural network architecture to jointly optimise the global- and local displacements. Experiment results are presented based on cross-validating registrations of 111 pairs of T2-weighted magnetic resonance images and 3D transrectal ultrasound images from prostate cancer patients with a total of over 4000 anatomical labels, yielding a median target registration error of 4.2 mm on landmark centroids and a median Dice of 0.88 on prostate glands.Comment: Accepted to ISBI 201

Monolithic integration of broadband optical isolators for polarization-diverse silicon photonics

Integrated optical isolators have been a longstanding challenge for photonic integrated circuits (PIC). An ideal integrated optical isolator for PIC should be made by a monolithic process, have a small footprint, exhibit broadband and polarization-diverse operation, and be compatible with multiple materials platforms. Despite significant progress, the optical isolators reported so far do not meet all these requirements. In this article we present monolithically integrated broadband magneto-optical isolators on silicon and silicon nitride (SiN) platforms operating for both TE and TM modes with record high performances, fulfilling all the essential characteristics for PIC applications. In particular, we demonstrate fully-TE broadband isolators by depositing high quality magneto-optical garnet thin films on the sidewalls of Si and SiN waveguides, a critical result for applications in TE-polarized on-chip lasers and amplifiers. This work demonstrates monolithic integration of high performance optical isolators on chip for polarization-diverse silicon photonic systems, enabling new pathways to impart nonreciprocal photonic functionality to a variety of integrated photonic devices

Ranks of permutative matrices

A new type of matrix, termed permutative, is defined and motivated herein. The focus is upon identifying circumstances under which square permutative matrices are rank deficient. Two distinct ways, along with variants upon them are given. These are a special kind of grouping of rows and a type of partition in which the blocks are again permutative. Other, results are given, along with some questions and conjectures

Designing research prototype for the elderly: a case study

This paper describes a research study regarding intergenerational story sharing of the elderly living in the nursing home, including four iterations, applyinga Research-through-Design approach. It started from an exploration prototype named Interactive Gallery(1st iteration), and its findings helped to narrow down our research area and define our research question.To answer it, the prototype named Slots-story (2nd iteration) and Slots-memento (3rd iteration) were designed and implemented, which focused on life story and memento story of the elderly respectively. While the 4th iteration aimed at facilitating intergenerational story sharing and sustainably. The above research iterations offer an example of how research prototypes supports to focus research area, and answer the research question in stages. We finally conclude witha discussion of insights on designing prototype for the non-tech-savvy elderly

Monolithic on-chip nonreciprocal photonics based on magneto-optical thin films

Monolithic integration of nonreciprocal optical devices on semiconductor substrates has been a long-sought goal of the photonics community. One promising route to achieve this goal is to deposit high quality magneto-optical (MO) oxide thin films directly on a semiconductor substrate. In this article, we will review our ongoing progress in material development and device engineering towards enabling a monolithically integrated, high-performance magneto-optical nonreciprocal photonics platform. In particular, we will discuss our recent work which has led to a new pulsed laser deposition (PLD) technique of Ce or Bi substituted yttrium iron garnet (YIG) thin films with reduced thermal budget, simplified growth protocols and improved magneto-optical characteristics. These materials were incorporated in monolithic resonator and interferometer based isolator devices to demonstrate on-chip optical isolation with improved device figure of merit. Challenges and opportunities for monolithic magneto-optical devices will be discussed in the context of our latest material and device performance metrics