Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation

This paper proposes a new hybrid architecture that consists of a deep Convolutional Network and a Markov Random Field. We show how this architecture is successfully applied to the challenging problem of articulated human pose estimation in monocular images. The architecture can exploit structural domain constraints such as geometric relationships between body joint locations. We show that joint training of these two model paradigms improves performance and allows us to significantly outperform existing state-of-the-art techniques

Geometric modeling of non-rigid 3D shapes : theory and application to object recognition.

One of the major goals of computer vision is the development of flexible and efficient methods for shape representation. This is true, especially for non-rigid 3D shapes where a great variety of shapes are produced as a result of deformations of a non-rigid object. Modeling these non-rigid shapes is a very challenging problem. Being able to analyze the properties of such shapes and describe their behavior is the key issue in research. Also, considering photometric features can play an important role in many shape analysis applications, such as shape matching and correspondence because it contains rich information about the visual appearance of real objects. This new information (contained in photometric features) and its important applications add another, new dimension to the problem\u27s difficulty. Two main approaches have been adopted in the literature for shape modeling for the matching and retrieval problem, local and global approaches. Local matching is performed between sparse points or regions of the shape, while the global shape approaches similarity is measured among entire models. These methods have an underlying assumption that shapes are rigidly transformed. And Most descriptors proposed so far are confined to shape, that is, they analyze only geometric and/or topological properties of 3D models. A shape descriptor or model should be isometry invariant, scale invariant, be able to capture the fine details of the shape, computationally efficient, and have many other good properties. A shape descriptor or model is needed. This shape descriptor should be: able to deal with the non-rigid shape deformation, able to handle the scale variation problem with less sensitivity to noise, able to match shapes related to the same class even if these shapes have missing parts, and able to encode both the photometric, and geometric information in one descriptor. This dissertation will address the problem of 3D non-rigid shape representation and textured 3D non-rigid shapes based on local features. Two approaches will be proposed for non-rigid shape matching and retrieval based on Heat Kernel (HK), and Scale-Invariant Heat Kernel (SI-HK) and one approach for modeling textured 3D non-rigid shapes based on scale-invariant Weighted Heat Kernel Signature (WHKS). For the first approach, the Laplace-Beltrami eigenfunctions is used to detect a small number of critical points on the shape surface. Then a shape descriptor is formed based on the heat kernels at the detected critical points for different scales. Sparse representation is used to reduce the dimensionality of the calculated descriptor. The proposed descriptor is used for classification via the Collaborative Representation-based Classification with a Regularized Least Square (CRC-RLS) algorithm. The experimental results have shown that the proposed descriptor can achieve state-of-the-art results on two benchmark data sets. For the second approach, an improved method to introduce scale-invariance has been also proposed to avoid noise-sensitive operations in the original transformation method. Then a new 3D shape descriptor is formed based on the histograms of the scale-invariant HK for a number of critical points on the shape at different time scales. A Collaborative Classification (CC) scheme is then employed for object classification. The experimental results have shown that the proposed descriptor can achieve high performance on the two benchmark data sets. An important observation from the experiments is that the proposed approach is more able to handle data under several distortion scenarios (noise, shot-noise, scale, and under missing parts) than the well-known approaches. For modeling textured 3D non-rigid shapes, this dissertation introduces, for the first time, a mathematical framework for the diffusion geometry on textured shapes. This dissertation presents an approach for shape matching and retrieval based on a weighted heat kernel signature. It shows how to include photometric information as a weight over the shape manifold, and it also propose a novel formulation for heat diffusion over weighted manifolds. Then this dissertation presents a new discretization method for the weighted heat kernel induced by the linear FEM weights. Finally, the weighted heat kernel signature is used as a shape descriptor. The proposed descriptor encodes both the photometric, and geometric information based on the solution of one equation. Finally, this dissertation proposes an approach for 3D face recognition based on the front contours of heat propagation over the face surface. The front contours are extracted automatically as heat is propagating starting from a detected set of landmarks. The propagation contours are used to successfully discriminate the various faces. The proposed approach is evaluated on the largest publicly available database of 3D facial images and successfully compared to the state-of-the-art approaches in the literature. This work can be extended to the problem of dense correspondence between non-rigid shapes. The proposed approaches with the properties of the Laplace-Beltrami eigenfunction can be utilized for 3D mesh segmentation. Another possible application of the proposed approach is the view point selection for 3D objects by selecting the most informative views that collectively provide the most descriptive presentation of the surface

Efficient Object Localization Using Convolutional Networks

Recent state-of-the-art performance on human-body pose estimation has been achieved with Deep Convolutional Networks (ConvNets). Traditional ConvNet architectures include pooling and sub-sampling layers which reduce computational requirements, introduce invariance and prevent over-training. These benefits of pooling come at the cost of reduced localization accuracy. We introduce a novel architecture which includes an efficient `position refinement' model that is trained to estimate the joint offset location within a small region of the image. This refinement model is jointly trained in cascade with a state-of-the-art ConvNet model to achieve improved accuracy in human joint location estimation. We show that the variance of our detector approaches the variance of human annotations on the FLIC dataset and outperforms all existing approaches on the MPII-human-pose dataset.Comment: 8 pages with 1 page of citation

Atmospheric Circulation of Terrestrial Exoplanets

The investigation of planets around other stars began with the study of gas giants, but is now extending to the discovery and characterization of super-Earths and terrestrial planets. Motivated by this observational tide, we survey the basic dynamical principles governing the atmospheric circulation of terrestrial exoplanets, and discuss the interaction of their circulation with the hydrological cycle and global-scale climate feedbacks. Terrestrial exoplanets occupy a wide range of physical and dynamical conditions, only a small fraction of which have yet been explored in detail. Our approach is to lay out the fundamental dynamical principles governing the atmospheric circulation on terrestrial planets--broadly defined--and show how they can provide a foundation for understanding the atmospheric behavior of these worlds. We first survey basic atmospheric dynamics, including the role of geostrophy, baroclinic instabilities, and jets in the strongly rotating regime (the "extratropics") and the role of the Hadley circulation, wave adjustment of the thermal structure, and the tendency toward equatorial superrotation in the slowly rotating regime (the "tropics"). We then survey key elements of the hydrological cycle, including the factors that control precipitation, humidity, and cloudiness. Next, we summarize key mechanisms by which the circulation affects the global-mean climate, and hence planetary habitability. In particular, we discuss the runaway greenhouse, transitions to snowball states, atmospheric collapse, and the links between atmospheric circulation and CO2 weathering rates. We finish by summarizing the key questions and challenges for this emerging field in the future.Comment: Invited review, in press for the Arizona Space Science Series book "Comparative Climatology of Terrestrial Planets" (S. Mackwell, M. Bullock, and J. Harder, editors). 56 pages, 26 figure

Multi-view Convolutional Neural Networks for 3D Shape Recognition

A longstanding question in computer vision concerns the representation of 3D shapes for recognition: should 3D shapes be represented with descriptors operating on their native 3D formats, such as voxel grid or polygon mesh, or can they be effectively represented with view-based descriptors? We address this question in the context of learning to recognize 3D shapes from a collection of their rendered views on 2D images. We first present a standard CNN architecture trained to recognize the shapes' rendered views independently of each other, and show that a 3D shape can be recognized even from a single view at an accuracy far higher than using state-of-the-art 3D shape descriptors. Recognition rates further increase when multiple views of the shapes are provided. In addition, we present a novel CNN architecture that combines information from multiple views of a 3D shape into a single and compact shape descriptor offering even better recognition performance. The same architecture can be applied to accurately recognize human hand-drawn sketches of shapes. We conclude that a collection of 2D views can be highly informative for 3D shape recognition and is amenable to emerging CNN architectures and their derivatives.Comment: v1: Initial version. v2: An updated ModelNet40 training/test split is used; results with low-rank Mahalanobis metric learning are added. v3 (ICCV 2015): A second camera setup without the upright orientation assumption is added; some accuracy and mAP numbers are changed slightly because a small issue in mesh rendering related to specularities is fixe

Nonrigid reconstruction of 3D breast surfaces with a low-cost RGBD camera for surgical planning and aesthetic evaluation

Accounting for 26% of all new cancer cases worldwide, breast cancer remains the most common form of cancer in women. Although early breast cancer has a favourable long-term prognosis, roughly a third of patients suffer from a suboptimal aesthetic outcome despite breast conserving cancer treatment. Clinical-quality 3D modelling of the breast surface therefore assumes an increasingly important role in advancing treatment planning, prediction and evaluation of breast cosmesis. Yet, existing 3D torso scanners are expensive and either infrastructure-heavy or subject to motion artefacts. In this paper we employ a single consumer-grade RGBD camera with an ICP-based registration approach to jointly align all points from a sequence of depth images non-rigidly. Subtle body deformation due to postural sway and respiration is successfully mitigated leading to a higher geometric accuracy through regularised locally affine transformations. We present results from 6 clinical cases where our method compares well with the gold standard and outperforms a previous approach. We show that our method produces better reconstructions qualitatively by visual assessment and quantitatively by consistently obtaining lower landmark error scores and yielding more accurate breast volume estimates

The 1999 Center for Simulation of Dynamic Response in Materials Annual Technical Report

Introduction: This annual report describes research accomplishments for FY 99 of the Center for Simulation of Dynamic Response of Materials. The Center is constructing a virtual shock physics facility in which the full three dimensional response of a variety of target materials can be computed for a wide range of compressive, ten- sional, and shear loadings, including those produced by detonation of energetic materials. The goals are to facilitate computation of a variety of experiments in which strong shock and detonation waves are made to impinge on targets consisting of various combinations of materials, compute the subsequent dy- namic response of the target materials, and validate these computations against experimental data

Numerical Investigation of Laser-Induced Ignition Phenomena

This thesis investigates various aspects of laser-ignition. Laser ignition is a form of combustion initiation by means of a focused laser pulse in a combustible mixture. The ignition process consists of a chain of processes with varying degrees of importance to the prediction of a successful flame propagation. Some of these processes include plasma formation, induced shock wave, emergence of a flame kernel, and successful transition to a self-sustained flame or flame quenching. This thesis will explore various aspects of this process using computational fluid dynamics and model analysis with the aim of identifying the controlling processes and simplified ways of capturing successful or failed ignition based on the injected laser energy, focusing optics and combustible gas compositions. The problem is motivated by practical considerations. Combustion systems are still the main energy conversion technologies and it appears that they will continue to be dominant in the near future. To address environmental pollution and sustainability concerns, clean and efficient systems are being explored. One of the key challenges encountered is the problem of assuring dependable ignition in these emerging technologies. Laser ignition is considered to be a promising technology which would guarantee smooth functioning of advanced clean and efficient engines. Benefits include its non-intrusive nature and the easy control of the spark location, timing, and energy deposition. For laser ignition systems to be useful, a good understanding of the process is needed. Understanding the degree to which each of the associated processes contributes to the development of a flame can lead to cost-effective models of ignition. This would align with current trends in computer aided engineering where simulations with physics-based models drastically reduce product development cycles. A perceived weakness in the laser ignition literature is the lack of simulations that compare models of different complexity in predicting the ensuing chemically reacting flows. The proposed research will focus on the laser ignition of methane and biogas from the perspective of numerical simulations. Experimental results will be used as validation targets for these simulations. The flow field and thermochemical features controlling the emergence of flame kernels will be determined. Explanations of possible quenching of the flame kernel will be sought. The problems addressed include numerical simulations of the laser-induced shock wave propagation, the transition of the laser-spark to a self-sustained flame with the help of chemical reactions, and the quenching of lean biogas flames. The shock wave study is found to accord with the blast wave theory, wherein the outward propagation can be predicted based on absorbed energy. Plasma kinetics is found to be unnecessary for the shock wave propagation. Using a compact or more detailed chemical scheme enables the prediction or the emergence of the flame. For prediction of the observed flame quenching behavior, however, the detailed scheme is necessary since the compact chemical scheme fails to capture the quenching event. Characteristic flow features are observed and explained in a manner that accords with experimental observations of global ignition features

Anisotropic diffusion of surface normals for feature preserving surface reconstruction

Journal ArticleFor 3D surface reconstruction problems with noisy and incomplete range data measured from complex scenes with arbitrary topologies, a low-level representation, such as level set surfaces, is used. Such surface reconstruction is typically accomplished by minimizing a weighted sum of data-model discrepancy and model smoothness terms. This paper introduces a new onlinear model smoothness term for surface reconstruction based on variations of the surface normals. A direct solution requires solving a fourth-order partial differential equation (PDE), which is very difficult with conventional numerical techniques. Our solution is based on processing the normals separately from the surface, which allows us to separate the problem into two second-order PDEs. The proposed method can smooth complex, noisy surfaces, while preserving sharp, geometric features, and it is a natural generalization of edge-preserving methods in image processing, such as anisotropic diffusion