1,255 research outputs found
Towards markerless orthopaedic navigation with intuitive Optical See-through Head-mounted displays
The potential of image-guided orthopaedic navigation to improve surgical outcomes has been well-recognised during the last two decades. According to the tracked pose of target bone, the anatomical information and preoperative plans are updated and displayed to surgeons, so that they can follow the guidance to reach the goal with higher accuracy, efficiency and reproducibility. Despite their success, current orthopaedic navigation systems have two main limitations: for target tracking, artificial markers have to be drilled into the bone and calibrated manually to the bone, which introduces the risk of additional harm to patients and increases operating complexity; for guidance visualisation, surgeons have to shift their attention from the patient to an external 2D monitor, which is disruptive and can be mentally stressful.
Motivated by these limitations, this thesis explores the development of an intuitive, compact and reliable navigation system for orthopaedic surgery. To this end, conventional marker-based tracking is replaced by a novel markerless tracking algorithm, and the 2D display is replaced by a 3D holographic Optical see-through (OST) Head-mounted display (HMD) precisely calibrated to a user's perspective.
Our markerless tracking, facilitated by a commercial RGBD camera, is achieved through deep learning-based bone segmentation followed by real-time pose registration. For robust segmentation, a new network is designed and efficiently augmented by a synthetic dataset. Our segmentation network outperforms the state-of-the-art regarding occlusion-robustness, device-agnostic behaviour, and target generalisability. For reliable pose registration, a novel Bounded Iterative Closest Point (BICP) workflow is proposed. The improved markerless tracking can achieve a clinically acceptable error of 0.95 deg and 2.17 mm according to a phantom test.
OST displays allow ubiquitous enrichment of perceived real world with contextually blended virtual aids through semi-transparent glasses. They have been recognised as a suitable visual tool for surgical assistance, since they do not hinder the surgeon's natural eyesight and require no attention shift or perspective conversion. The OST calibration is crucial to ensure locational-coherent surgical guidance.
Current calibration methods are either human error-prone or hardly applicable to commercial devices. To this end, we propose an offline camera-based calibration method that is highly accurate yet easy to implement in commercial products, and an online alignment-based refinement that is user-centric and robust against user error. The proposed methods are proven to be superior to other similar State-of-
the-art (SOTA)s regarding calibration convenience and display accuracy.
Motivated by the ambition to develop the world's first markerless OST navigation system, we integrated the developed markerless tracking and calibration scheme into a complete navigation workflow designed for femur drilling tasks during knee replacement surgery. We verify the usability of our designed OST system with an experienced orthopaedic surgeon by a cadaver study. Our test validates the potential of the proposed markerless navigation system for surgical assistance, although further improvement is required for clinical acceptance.Open Acces
Markerless Human Motion Analysis
Measuring and understanding human motion is crucial in several domains,
ranging from neuroscience, to rehabilitation and sports biomechanics. Quantitative
information about human motion is fundamental to study how our
Central Nervous System controls and organizes movements to functionally
evaluate motor performance and deficits. In the last decades, the research in
this field has made considerable progress. State-of-the-art technologies that
provide useful and accurate quantitative measures rely on marker-based systems.
Unfortunately, markers are intrusive and their number and location must
be determined a priori. Also, marker-based systems require expensive laboratory
settings with several infrared cameras. This could modify the naturalness
of a subject\u2019s movements and induce discomfort. Last, but not less important,
they are computationally expensive in time and space. Recent advances on
markerless pose estimation based on computer vision and deep neural networks
are opening the possibility of adopting efficient video-based methods
for extracting movement information from RGB video data. In this contest,
this thesis presents original contributions to the following objectives: (i) the
implementation of a video-based markerless pipeline to quantitatively characterize
human motion; (ii) the assessment of its accuracy if compared with
a gold standard marker-based system; (iii) the application of the pipeline to
different domains in order to verify its versatility, with a special focus on the
characterization of the motion of preterm infants and on gait analysis. With
the proposed approach we highlight that, starting only from RGB videos and
leveraging computer vision and machine learning techniques, it is possible to
extract reliable information characterizing human motion comparable to that
obtained with gold standard marker-based systems
GANerated Hands for Real-time 3D Hand Tracking from Monocular RGB
We address the highly challenging problem of real-time 3D hand tracking based
on a monocular RGB-only sequence. Our tracking method combines a convolutional
neural network with a kinematic 3D hand model, such that it generalizes well to
unseen data, is robust to occlusions and varying camera viewpoints, and leads
to anatomically plausible as well as temporally smooth hand motions. For
training our CNN we propose a novel approach for the synthetic generation of
training data that is based on a geometrically consistent image-to-image
translation network. To be more specific, we use a neural network that
translates synthetic images to "real" images, such that the so-generated images
follow the same statistical distribution as real-world hand images. For
training this translation network we combine an adversarial loss and a
cycle-consistency loss with a geometric consistency loss in order to preserve
geometric properties (such as hand pose) during translation. We demonstrate
that our hand tracking system outperforms the current state-of-the-art on
challenging RGB-only footage
Markerless Motion Capture in the Crowd
This work uses crowdsourcing to obtain motion capture data from video
recordings. The data is obtained by information workers who click repeatedly to
indicate body configurations in the frames of a video, resulting in a model of
2D structure over time. We discuss techniques to optimize the tracking task and
strategies for maximizing accuracy and efficiency. We show visualizations of a
variety of motions captured with our pipeline then apply reconstruction
techniques to derive 3D structure.Comment: Presented at Collective Intelligence conference, 2012
(arXiv:1204.2991
A Primer on Motion Capture with Deep Learning: Principles, Pitfalls and Perspectives
Extracting behavioral measurements non-invasively from video is stymied by
the fact that it is a hard computational problem. Recent advances in deep
learning have tremendously advanced predicting posture from videos directly,
which quickly impacted neuroscience and biology more broadly. In this primer we
review the budding field of motion capture with deep learning. In particular,
we will discuss the principles of those novel algorithms, highlight their
potential as well as pitfalls for experimentalists, and provide a glimpse into
the future.Comment: Review, 21 pages, 8 figures and 5 boxe
- …