167 research outputs found
HoloLens 2 Sensor Streaming
We present a HoloLens 2 server application for streaming device data via TCP
in real time. The server can stream data from the four grayscale cameras, depth
sensor, IMU, front RGB camera, microphone, head tracking, eye tracking, and
hand tracking. Each sent data frame has a timestamp and, optionally, the
instantaneous pose of the device in 3D space. The server allows downloading
device calibration data, such as camera intrinsics, and can be integrated into
Unity projects as a plugin, with support for basic upstream capabilities. To
achieve real time video streaming at full frame rate, we leverage the video
encoding capabilities of the HoloLens 2. Finally, we present a Python library
for receiving and decoding the data, which includes utilities that facilitate
passing the data to other libraries. The source code, Python demos, and
precompiled binaries are available at https://github.com/jdibenes/hl2ss.Comment: Technical repor
DDM-NET: End-to-end learning of keypoint feature Detection, Description and Matching for 3D localization
In this paper, we propose an end-to-end framework that jointly learns
keypoint detection, descriptor representation and cross-frame matching for the
task of image-based 3D localization. Prior art has tackled each of these
components individually, purportedly aiming to alleviate difficulties in
effectively train a holistic network. We design a self-supervised image warping
correspondence loss for both feature detection and matching, a
weakly-supervised epipolar constraints loss on relative camera pose learning,
and a directional matching scheme that detects key-point features in a source
image and performs coarse-to-fine correspondence search on the target image. We
leverage this framework to enforce cycle consistency in our matching module. In
addition, we propose a new loss to robustly handle both definite inlier/outlier
matches and less-certain matches. The integration of these learning mechanisms
enables end-to-end training of a single network performing all three
localization components. Bench-marking our approach on public data-sets,
exemplifies how such an end-to-end framework is able to yield more accurate
localization that out-performs both traditional methods as well as
state-of-the-art weakly supervised methods
NeurOCS: Neural NOCS Supervision for Monocular 3D Object Localization
Monocular 3D object localization in driving scenes is a crucial task, but
challenging due to its ill-posed nature. Estimating 3D coordinates for each
pixel on the object surface holds great potential as it provides dense 2D-3D
geometric constraints for the underlying PnP problem. However, high-quality
ground truth supervision is not available in driving scenes due to sparsity and
various artifacts of Lidar data, as well as the practical infeasibility of
collecting per-instance CAD models. In this work, we present NeurOCS, a
framework that uses instance masks and 3D boxes as input to learn 3D object
shapes by means of differentiable rendering, which further serves as
supervision for learning dense object coordinates. Our approach rests on
insights in learning a category-level shape prior directly from real driving
scenes, while properly handling single-view ambiguities. Furthermore, we study
and make critical design choices to learn object coordinates more effectively
from an object-centric view. Altogether, our framework leads to new
state-of-the-art in monocular 3D localization that ranks 1st on the
KITTI-Object benchmark among published monocular methods.Comment: Paper was accepted to CVPR 202
Efficient video collection association using geometry-aware Bag-of-Iconics representations
Abstract Recent years have witnessed the dramatic evolution in visual data volume and processing capabilities. For example, technical advances have enabled 3D modeling from large-scale crowdsourced photo collections. Compared to static image datasets, exploration and exploitation of Internet video collections are still largely unsolved. To address this challenge, we first propose to represent video contents using a histogram representation of iconic imagery attained from relevant visual datasets. We then develop a data-driven framework for a fully unsupervised extraction of such representations. Our novel Bag-of-Iconics (BoI) representation efficiently analyzes individual videos within a large-scale video collection. We demonstrate our proposed BoI representation with two novel applications: (1) finding video sequences connecting adjacent landmarks and aligning reconstructed 3D models and (2) retrieving geometrically relevant clips from video collections. Results on crowdsourced datasets illustrate the efficiency and effectiveness of our proposed Bag-of-Iconics representation
Warped K-Means: An algorithm to cluster sequentially-distributed data
[EN] Many devices generate large amounts of data that follow some sort of sequentiality, e.g.,
motion sensors, e-pens, eye trackers, etc. and often these data need to be compressed for
classification, storage, and/or retrieval tasks. Traditional clustering algorithms can be used
for this purpose, but unfortunately they do not cope with the sequential information
implicitly embedded in such data. Thus, we revisit the well-known K-means algorithm
and provide a general method to properly cluster sequentially-distributed data. We present
Warped K-Means (WKM), a multi-purpose partitional clustering procedure that minimizes
the sum of squared error criterion, while imposing a hard sequentiality constraint in the
classification step. We illustrate the properties of WKM in three applications, one being
the segmentation and classification of human activity. WKM outperformed five state-of-
the-art clustering techniques to simplify data trajectories, achieving a recognition accuracy
of near 97%, which is an improvement of around 66% over their peers. Moreover, such an
improvement came with a reduction in the computational cost of more than one order of
magnitude.This work has been partially supported by Casmacat (FP7-ICT-2011-7, Project 287576), tranScriptorium (FP7-ICT-2011-9, Project 600707), STraDA (MINECO, TIN2012-37475-0O2-01), and ALMPR (GVA, Prometeo/20091014) projects.Leiva Torres, LA.; Vidal, E. (2013). Warped K-Means: An algorithm to cluster sequentially-distributed data. Information Sciences. 237:196-210. https://doi.org/10.1016/j.ins.2013.02.042S19621023
The InfraRed Imaging Spectrograph (IRIS) for TMT: latest science cases and simulations
The Thirty Meter Telescope (TMT) first light instrument IRIS (Infrared
Imaging Spectrograph) will complete its preliminary design phase in 2016. The
IRIS instrument design includes a near-infrared (0.85 - 2.4 micron) integral
field spectrograph (IFS) and imager that are able to conduct simultaneous
diffraction-limited observations behind the advanced adaptive optics system
NFIRAOS. The IRIS science cases have continued to be developed and new science
studies have been investigated to aid in technical performance and design
requirements. In this development phase, the IRIS science team has paid
particular attention to the selection of filters, gratings, sensitivities of
the entire system, and science cases that will benefit from the parallel mode
of the IFS and imaging camera. We present new science cases for IRIS using the
latest end-to-end data simulator on the following topics: Solar System bodies,
the Galactic center, active galactic nuclei (AGN), and distant
gravitationally-lensed galaxies. We then briefly discuss the necessity of an
advanced data management system and data reduction pipeline.Comment: 15 pages, 7 figures, SPIE (2016) 9909-0
Solving the conundrum of intra-specific variation in metabolic rate: A multidisciplinary conceptual and methodological toolkit
Researchers from diverse disciplines, including organismal and cellular physiology, sports science, human nutrition, evolution and ecology, have sought to understand the causes and consequences of the surprising variation in metabolic rate found among and within individual animals of the same species. Research in this area has been hampered by differences in approach, terminology and methodology, and the context in which measurements are made. Recent advances provide important opportunities to identify and address the key questions in the field. By bringing together researchers from different areas of biology and biomedicine, we describe and evaluate these developments and the insights they could yield, highlighting the need for more standardisation across disciplines. We conclude with a list of important questions that can now be addressed by developing a common conceptual and methodological toolkit for studies on metabolic variation in animals
Geographical variations in the benefit of applying a prioritization system for cataract surgery in different regions of Spain
<p>Abstract</p> <p>Background</p> <p>In Spain, there are substantial variations in the utilization of health resources among regions. Because the need for surgery differs in patients with appropriate surgical indication, introducing a prioritization system might be beneficial. Our objective was to assess geographical variations in the impact of applying a prioritization system in patients on the waiting list for cataract surgery in different regions of Spain by using a discrete-event simulation model.</p> <p>Methods</p> <p>A discrete-event simulation model to evaluate demand and waiting time for cataract surgery was constructed. The model was reproduced and validated in five regions of Spain and was fed administrative data (population census, surgery rates, waiting list information) and data from research studies (incidence of cataract). The benefit of introducing a prioritization system was contrasted with the usual first-in, first-out (FIFO) discipline. The prioritization system included clinical, functional and social criteria. Priority scores ranged between 0 and 100, with greater values indicating higher priority. The measure of results was the waiting time weighted by the priority score of each patient who had passed through the waiting list. Benefit was calculated as the difference in time weighted by priority score between operating according to waiting time or to priority.</p> <p>Results</p> <p>The mean waiting time for patients undergoing surgery according to the FIFO discipline varied from 1.97 months (95% CI 1.85; 2.09) in the Basque Country to 10.02 months (95% CI 9.91; 10.12) in the Canary Islands. When the prioritization system was applied, the mean waiting time was reduced to a minimum of 0.73 months weighted by priority score (95% CI 0.68; 0.78) in the Basque Country and a maximum of 5.63 months (95% CI 5.57; 5.69) in the Canary Islands. The waiting time weighted by priority score saved by the prioritization system varied from 1.12 months (95% CI 1.07; 1.16) in Andalusia to 2.73 months (95% CI 2.67; 2.80) in Aragon.</p> <p>Conclusion</p> <p>The prioritization system reduced the impact of the variations found among the regions studied, thus improving equity. Prioritization allocates the available resources within each region more efficiently and reduces the waiting time of patients with greater need. Prioritization was more beneficial than allocating surgery by waiting time alone.</p
- …