On human motion prediction using recurrent neural networks

Human motion modelling is a classical problem at the intersection of graphics and computer vision, with applications spanning human-computer interaction, motion synthesis, and motion prediction for virtual and augmented reality. Following the success of deep learning methods in several computer vision tasks, recent work has focused on using deep recurrent neural networks (RNNs) to model human motion, with the goal of learning time-dependent representations that perform tasks such as short-term motion prediction and long-term human motion synthesis. We examine recent work, with a focus on the evaluation methodologies commonly used in the literature, and show that, surprisingly, state-of-the-art performance can be achieved by a simple baseline that does not attempt to model motion at all. We investigate this result, and analyze recent RNN methods by looking at the architectures, loss functions, and training procedures used in state-of-the-art approaches. We propose three changes to the standard RNN models typically used for human motion, which result in a simple and scalable RNN architecture that obtains state-of-the-art performance on human motion prediction.Comment: Accepted at CVPR 1

Physical Primitive Decomposition

Objects are made of parts, each with distinct geometry, physics, functionality, and affordances. Developing such a distributed, physical, interpretable representation of objects will facilitate intelligent agents to better explore and interact with the world. In this paper, we study physical primitive decomposition---understanding an object through its components, each with physical and geometric attributes. As annotated data for object parts and physics are rare, we propose a novel formulation that learns physical primitives by explaining both an object's appearance and its behaviors in physical events. Our model performs well on block towers and tools in both synthetic and real scenarios; we also demonstrate that visual and physical observations often provide complementary signals. We further present ablation and behavioral studies to better understand our model and contrast it with human performance.Comment: ECCV 2018. Project page: http://ppd.csail.mit.edu

Virtual clinical trials in medical imaging: a review

The accelerating complexity and variety of medical imaging devices and methods have outpaced the ability to evaluate and optimize their design and clinical use. This is a significant and increasing challenge for both scientific investigations and clinical applications. Evaluations would ideally be done using clinical imaging trials. These experiments, however, are often not practical due to ethical limitations, expense, time requirements, or lack of ground truth. Virtual clinical trials (VCTs) (also known as in silico imaging trials or virtual imaging trials) offer an alternative means to efficiently evaluate medical imaging technologies virtually. They do so by simulating the patients, imaging systems, and interpreters. The field of VCTs has been constantly advanced over the past decades in multiple areas. We summarize the major developments and current status of the field of VCTs in medical imaging. We review the core components of a VCT: computational phantoms, simulators of different imaging modalities, and interpretation models. We also highlight some of the applications of VCTs across various imaging modalities

DEVELOPMENT AND IMPLEMENTATION OF A HOMOGENEOUS AND A HETEROGENEOUS ANTHROPOMORPHIC END TO END QUALITY ASSURANCE AUDIT SYSTEM PHANTOM FOR MAGNETIC RESONANCE GUIDED RADIOTHERAPY MODALITIES RANGING FROM 0.35 T TO 1.50 T

Introduction: Magnetic resonance (MR) guided radiation therapy (MRgRT) is an emerging field that integrates an MR imager with either a linear accelerator or three radioactive cobalt-60 sources. Before institutions participate in multi-institutional NCI-sponsored clinical trials, they are required to perform a credentialing test provided by IROC-Houston. During the credentialing test, end-to-end phantoms are used to evaluate the institution’s ability to perform consistent and accurate radiation treatments. IROC-Houston’s conventional anthropomorphic phantoms are not visible in MR, thus they are insufficient for MRgRT systems. The purpose of this work was to create an anthropomorphic thorax and a head and neck (H&N) phantom for MRgRT systems with magnetic fields ranging from 0.35T to 1.5T. Methods: Over 80 synthetic materials were examined as potential materials used to construct the MRgRT thorax and H&N phantoms. Materials were characterized by: 1) measuring Hounsfield units, 2) visualizing in MR and CT imagers and 3) evaluating their dosimetric characteristics. Once materials were selected for the MRgRT phantoms, radiochromic film and double-loaded TLDs were then characterized in a 1.5T and a 0.35T MR environment. Reproducibility measurements on double-loaded TLDs were performed by using an acrylic block and irradiating it in 0T/1.5T and 0T/0.35T configurations on the Unity system and the MRIdian Cobalt 60 system, respectively. Geometrical thorax and H&N phantom slabs were designed to mimic similar interface conditions seen in anthropomorphic phantoms, but were simplified to reduce manufacturing time. The geometrical phantoms were designed with a rectangular tumor centrally located around surrounding tissue. These two phantoms were used to characterize radiochromic EBT3 film and TLDs by comparing beam profiles and point dose measurements irradiated with and without magnetic fields, respectively. GEANT4 Monte Carlo simulations validated the detectors in both Unity 0T/1.5T and MRIdian 0T/0.35T configurations. Two MRgRT anthropomorphic (H&N and thorax) phantoms were designed, manufactured and evaluated. A reproducibility and feasibility study was conducted to evaluate the phantom’s performance on MRgRT systems. Results: This study found four materials which were tissue equivalent and visible on both MR and CT. Additionally, this study showed negligible difference in dose response between TLDs and radiochromic film when irradiated in 0.35T and 1.5T magnetic field environments. Two anthropomorphic phantoms were constructed and evaluated. The anthropomorphic thorax and H&N phantoms passed IROC-Houston’s 7%/5mm and 7%/4mm gamma passing criteria, respectively. Conclusions: An anthropomorphic thorax and an H&N phantom were tissue equivalent, compatible with MR and CT workflows and could be used as end-to-end QA tools for MRgRT systems with magnetic fields ranging from 0.35T to 1.5T