1,271 research outputs found
PC-HMR: Pose Calibration for 3D Human Mesh Recovery from 2D Images/Videos
The end-to-end Human Mesh Recovery (HMR) approach has been successfully used
for 3D body reconstruction. However, most HMR-based frameworks reconstruct
human body by directly learning mesh parameters from images or videos, while
lacking explicit guidance of 3D human pose in visual data. As a result, the
generated mesh often exhibits incorrect pose for complex activities. To tackle
this problem, we propose to exploit 3D pose to calibrate human mesh.
Specifically, we develop two novel Pose Calibration frameworks, i.e., Serial
PC-HMR and Parallel PC-HMR. By coupling advanced 3D pose estimators and HMR in
a serial or parallel manner, these two frameworks can effectively correct human
mesh with guidance of a concise pose calibration module. Furthermore, since the
calibration module is designed via non-rigid pose transformation, our PC-HMR
frameworks can flexibly tackle bone length variations to alleviate misplacement
in the calibrated mesh. Finally, our frameworks are based on generic and
complementary integration of data-driven learning and geometrical modeling. Via
plug-and-play modules, they can be efficiently adapted for both
image/video-based human mesh recovery. Additionally, they have no requirement
of extra 3D pose annotations in the testing phase, which releases inference
difficulties in practice. We perform extensive experiments on the popular
bench-marks, i.e., Human3.6M, 3DPW and SURREAL, where our PC-HMR frameworks
achieve the SOTA results.Comment: 9 pages, 7 figures. AAAI202
Micro Fourier Transform Profilometry (FTP): 3D shape measurement at 10,000 frames per second
Recent advances in imaging sensors and digital light projection technology
have facilitated a rapid progress in 3D optical sensing, enabling 3D surfaces
of complex-shaped objects to be captured with improved resolution and accuracy.
However, due to the large number of projection patterns required for phase
recovery and disambiguation, the maximum fame rates of current 3D shape
measurement techniques are still limited to the range of hundreds of frames per
second (fps). Here, we demonstrate a new 3D dynamic imaging technique, Micro
Fourier Transform Profilometry (FTP), which can capture 3D surfaces of
transient events at up to 10,000 fps based on our newly developed high-speed
fringe projection system. Compared with existing techniques, FTP has the
prominent advantage of recovering an accurate, unambiguous, and dense 3D point
cloud with only two projected patterns. Furthermore, the phase information is
encoded within a single high-frequency fringe image, thereby allowing
motion-artifact-free reconstruction of transient events with temporal
resolution of 50 microseconds. To show FTP's broad utility, we use it to
reconstruct 3D videos of 4 transient scenes: vibrating cantilevers, rotating
fan blades, bullet fired from a toy gun, and balloon's explosion triggered by a
flying dart, which were previously difficult or even unable to be captured with
conventional approaches.Comment: This manuscript was originally submitted on 30th January 1
Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery
One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions
INFORMATION TECHNOLOGY FOR NEXT-GENERATION OF SURGICAL ENVIRONMENTS
Minimally invasive surgeries (MIS) are fundamentally constrained by image quality,access to the operative field, and the visualization environment on which thesurgeon relies for real-time information. Although invasive access benefits the patient,it also leads to more challenging procedures, which require better skills andtraining. Endoscopic surgeries rely heavily on 2D interfaces, introducing additionalchallenges due to the loss of depth perception, the lack of 3-Dimensional imaging,and the reduction of degrees of freedom.By using state-of-the-art technology within a distributed computational architecture,it is possible to incorporate multiple sensors, hybrid display devices, and3D visualization algorithms within a exible surgical environment. Such environmentscan assist the surgeon with valuable information that goes far beyond what iscurrently available. In this thesis, we will discuss how 3D visualization and reconstruction,stereo displays, high-resolution display devices, and tracking techniques arekey elements in the next-generation of surgical environments
Deformable and articulated 3D reconstruction from monocular video sequences
PhDThis thesis addresses the problem of deformable and articulated structure from motion from
monocular uncalibrated video sequences. Structure from motion is defined as the problem of
recovering information about the 3D structure of scenes imaged by a camera in a video sequence.
Our study aims at the challenging problem of non-rigid shapes (e.g. a beating heart or a smiling
face). Non-rigid structures appear constantly in our everyday life, think of a bicep curling, a
torso twisting or a smiling face. Our research seeks a general method to perform 3D shape
recovery purely from data, without having to rely on a pre-computed model or training data.
Open problems in the field are the difficulty of the non-linear estimation, the lack of a real-time
system, large amounts of missing data in real-world video sequences, measurement noise and
strong deformations. Solving these problems would take us far beyond the current state of the
art in non-rigid structure from motion. This dissertation presents our contributions in the field
of non-rigid structure from motion, detailing a novel algorithm that enforces the exact metric
structure of the problem at each step of the minimisation by projecting the motion matrices
onto the correct deformable or articulated metric motion manifolds respectively. An important
advantage of this new algorithm is its ability to handle missing data which becomes crucial
when dealing with real video sequences. We present a generic bilinear estimation framework,
which improves convergence and makes use of the manifold constraints. Finally, we demonstrate
a sequential, frame-by-frame estimation algorithm, which provides a 3D model and camera
parameters for each video frame, while simultaneously building a model of object deformation
Rotate and Hold and Scan (RAHAS): Structured Light Illumination for Use in Remote Areas
As a critical step after the discovery of material culture in the field, archaeologists have a need to document these findings with a slew of different physical measurements and photographs from varying perspectives. 3-D imaging is becoming increasingly popular as the primary documenting method to replace the plethora of tests and measurements, but for remote areas 3-D becomes more cumbersome due to physical and environmental constraints. The difficulty of using a 3-D imaging system in such environments is drastically lessened while using the RAHAS technique, since it acquires scans untethered to a computer. The goal of this thesis is to present the RAHAS Structured Light Illumination technique for 3-D image acquisition, and the performance of the RAHAS technique as a measurement tool for documentation of material culture on a field trip to the Rio Platano Biosphere in Honduras
- …