25 research outputs found
Articulation-aware Canonical Surface Mapping
We tackle the tasks of: 1) predicting a Canonical Surface Mapping (CSM) that
indicates the mapping from 2D pixels to corresponding points on a canonical
template shape, and 2) inferring the articulation and pose of the template
corresponding to the input image. While previous approaches rely on keypoint
supervision for learning, we present an approach that can learn without such
annotations. Our key insight is that these tasks are geometrically related, and
we can obtain supervisory signal via enforcing consistency among the
predictions. We present results across a diverse set of animal object
categories, showing that our method can learn articulation and CSM prediction
from image collections using only foreground mask labels for training. We
empirically show that allowing articulation helps learn more accurate CSM
prediction, and that enforcing the consistency with predicted CSM is similarly
critical for learning meaningful articulation.Comment: To appear at CVPR 2020, project page
https://nileshkulkarni.github.io/acsm
A machine-learning data set prepared from the NASA solar dynamics observatory mission
In this paper, we present a curated data set from the NASA Solar Dynamics Observatory (SDO) mission in a format suitable for machine-learning research. Beginning from level 1 scientific products we have processed various instrumental corrections, down-sampled to manageable spatial and temporal resolutions, and synchronized observations spatially and temporally. We illustrate the use of this data set with two example applications: forecasting future extreme ultraviolet (EUV) Variability Experiment (EVE) irradiance from present EVE irradiance and translating Helioseismic and Magnetic Imager observations into Atmospheric Imaging Assembly observations. For each application, we provide metrics and baselines for future model comparison. We anticipate this curated data set will facilitate machine-learning research in heliophysics and the physical sciences generally, increasing the scientific return of the SDO mission. This work is a direct result of the 2018 NASA Frontier Development Laboratory Program. Please see the Appendix for access to the data set, totaling 6.5TBs
Large-Scale Spatial Cross-Calibration of Hinode/SOT-SP and SDO/HMI
We investigate the cross-calibration of the Hinode/SOT-SP and SDO/HMI
instrument meta-data, specifically the correspondence of the scaling and
pointing information. Accurate calibration of these datasets gives the
correspondence needed by inter-instrument studies and learning-based
magnetogram systems, and is required for physically-meaningful photospheric
magnetic field vectors. We approach the problem by robustly fitting geometric
models on correspondences between images from each instrument's pipeline. This
technique is common in computer vision, but several critical details are
required when using scanning slit spectrograph data like Hinode/SOT-SP. We
apply this technique to data spanning a decade of the Hinode mission. Our
results suggest corrections to the published Level 2 Hinode/SOT-SP data. First,
an analysis on approximately 2,700 scans suggests that the reported pixel size
in Hinode/SOT-SP Level 2 data is incorrect by around 1%. Second, analysis of
over 12,000 scans show that the pointing information is often incorrect by
dozens of arcseconds with a strong bias. Regression of these corrections
indicates that thermal effects have caused secular and cyclic drift in
Hinode/SOT-SP pointing data over its mission. We offer two solutions. First,
direct co-alignment with SDO/HMI data via our procedure can improve alignments
for many Hinode/SOT-SP scans. Second, since the pointing errors are
predictable, simple post-hoc corrections can substantially improve the
pointing. We conclude by illustrating the impact of this updated calibration on
derived physical data products needed for research and interpretation. Among
other things, our results suggest that the pointing errors induce a hemispheric
bias in estimates of radial current density.Comment: Under revisions at ApJ
Understanding 3D Object Interaction from a Single Image
Humans can easily understand a single image as depicting multiple potential
objects permitting interaction. We use this skill to plan our interactions with
the world and accelerate understanding new objects without engaging in
interaction. In this paper, we would like to endow machines with the similar
ability, so that intelligent agents can better explore the 3D scene or
manipulate objects. Our approach is a transformer-based model that predicts the
3D location, physical properties and affordance of objects. To power this
model, we collect a dataset with Internet videos, egocentric videos and indoor
images to train and validate our approach. Our model yields strong performance
on our data, and generalizes well to robotics data