24 research outputs found
NOViSE: a virtual natural orifice transluminal endoscopic surgery simulator
Purpose: Natural Orifice Transluminal Endoscopic Surgery (NOTES) is a novel technique in minimally invasive surgery whereby a flexible endoscope is inserted via a natural orifice to gain access to the abdominal cavity, leaving no external scars. This innovative use of flexible endoscopy creates many new challenges and is associated with a steep learning curve for clinicians. Methods: We developed NOViSE - the first force-feedback enabled virtual reality simulator for NOTES training supporting a flexible endoscope. The haptic device is custom built and the behaviour of the virtual flexible endoscope is based on an established theoretical framework – the Cosserat Theory of Elastic Rods. Results: We present the application of NOViSE to the simulation of a hybrid trans-gastric cholecystectomy procedure. Preliminary results of face, content and construct validation have previously shown that NOViSE delivers the required level of realism for training of endoscopic manipulation skills specific to NOTES Conclusions: VR simulation of NOTES procedures can contribute to surgical training and improve the educational experience without putting patients at risk, raising ethical issues or requiring expensive animal or cadaver facilities. In the context of an experimental technique, NOViSE could potentially facilitate NOTES development and contribute to its wider use by keeping practitioners up to date with this novel surgical technique. NOViSE is a first prototype and the initial results indicate that it provides promising foundations for further development
Learning Quasi-Static 3D Models of Markerless Deformable Linear Objects for Bimanual Robotic Manipulation
The robotic manipulation of Deformable Linear Objects (DLOs) is a vital and
challenging task that is important in many practical applications. Classical
model-based approaches to this problem require an accurate model to capture how
robot motions affect the deformation of the DLO. Nowadays, data-driven models
offer the best tradeoff between quality and computation time. This paper
analyzes several learning-based 3D models of the DLO and proposes a new one
based on the Transformer architecture that achieves superior accuracy, even on
the DLOs of different lengths, thanks to the proposed scaling method. Moreover,
we introduce a data augmentation technique, which improves the prediction
performance of almost all considered DLO data-driven models. Thanks to this
technique, even a simple Multilayer Perceptron (MLP) achieves close to
state-of-the-art performance while being significantly faster to evaluate. In
the experiments, we compare the performance of the learning-based 3D models of
the DLO on several challenging datasets quantitatively and demonstrate their
applicability in the task of shaping a DLO.Comment: Under review for IEEE Robotics and Automation Letter
OstrichRL: A Musculoskeletal Ostrich Simulation to Study Bio-mechanical Locomotion
Muscle-actuated control is a research topic that spans multiple domains,
including biomechanics, neuroscience, reinforcement learning, robotics, and
graphics. This type of control is particularly challenging as bodies are often
overactuated and dynamics are delayed and non-linear. It is however a very well
tested and tuned actuation mechanism that has undergone millions of years of
evolution with interesting properties exploiting passive forces and efficient
energy storage of muscle-tendon units. To facilitate research on
muscle-actuated simulation, we release a 3D musculoskeletal simulation of an
ostrich based on the MuJoCo physics engine. The ostrich is one of the fastest
bipeds on earth and therefore makes an excellent model for studying
muscle-actuated bipedal locomotion. The model is based on CT scans and
dissections used to collect actual muscle data, such as insertion sites,
lengths, and pennation angles. Along with this model, we also provide a set of
reinforcement learning tasks, including reference motion tracking, running, and
neck control, used to infer muscle actuation patterns. The reference motion
data is based on motion capture clips of various behaviors that we preprocessed
and adapted to our model. This paper describes how the model was built and
iteratively improved using the tasks. We also evaluate the accuracy of the
muscle actuation patterns by comparing them to experimentally collected
electromyographic data from locomoting birds. The results demonstrate the need
for rich reward signals or regularization techniques to constrain muscle
excitations and produce realistic movements. Overall, we believe that this work
can provide a useful bridge between fields of research interested in muscle
actuation.Comment: https://github.com/vittorione94/ostrichr
Human 3D Avatar Modeling with Implicit Neural Representation: A Brief Survey
A human 3D avatar is one of the important elements in the metaverse, and the
modeling effect directly affects people's visual experience. However, the human
body has a complex topology and diverse details, so it is often expensive,
time-consuming, and laborious to build a satisfactory model. Recent studies
have proposed a novel method, implicit neural representation, which is a
continuous representation method and can describe objects with arbitrary
topology at arbitrary resolution. Researchers have applied implicit neural
representation to human 3D avatar modeling and obtained more excellent results
than traditional methods. This paper comprehensively reviews the application of
implicit neural representation in human body modeling. First, we introduce
three implicit representations of occupancy field, SDF, and NeRF, and make a
classification of the literature investigated in this paper. Then the
application of implicit modeling methods in the body, hand, and head are
compared and analyzed respectively. Finally, we point out the shortcomings of
current work and provide available suggestions for researchers.Comment: A Brief Surve
PerSival: Neural-network-based visualisation for pervasive continuum-mechanical simulations in musculoskeletal biomechanics
This paper presents a novel neural network architecture for the purpose of
pervasive visualisation of a 3D human upper limb musculoskeletal system model.
Bringing simulation capabilities to resource-poor systems like mobile devices
is of growing interest across many research fields, to widen applicability of
methods and results. Until recently, this goal was thought to be out of reach
for realistic continuum-mechanical simulations of musculoskeletal systems, due
to prohibitive computational cost. Within this work we use a sparse grid
surrogate to capture the surface deformation of the m.~biceps brachii in order
to train a deep learning model, used for real-time visualisation of the same
muscle. Both these surrogate models take 5 muscle activation levels as input
and output Cartesian coordinate vectors for each mesh node on the muscle's
surface. Thus, the neural network architecture features a significantly lower
input than output dimension. 5 muscle activation levels were sufficient to
achieve an average error of 0.97 +/- 0.16 mm, or 0.57 +/- 0.10 % for the 2809
mesh node positions of the biceps. The model achieved evaluation times of 9.88
ms per predicted deformation state on CPU only and 3.48 ms with GPU-support,
leading to theoretical frame rates of 101 fps and 287 fps respectively. Deep
learning surrogates thus provide a way to make continuum-mechanical simulations
accessible for visual real-time applications.Comment: 10 pages, 4 figures, 5 tables, to be submitted to Medical Image
Analysi
Remeshing Eulerian-on-Lagrangian Strands
Physics-based simulation of strands is an important and well-studied topic within the field of computer graphics. One particular case, in which a strand is bending and sliding around a sharp corner, has been a challenge to simulate due to the additional constraints that are involved. Many of the previous methods for simulating strands do not perform well with strands crossing and sliding with respect to each other. In this research, we have developed a formulation that is capable of simulating and remeshing bending and sliding strands that combine the traditional Lagrangian method of physics-based simulation with an Eulerian approach. We found that our program is able to support dynamic remeshing of an Eulerian-on-Lagrangian strand when it is bending around a sharp corner, which provides a more accurate simulation. This could be extended to more complex simulations, such as the simulation of Ayatori (string art or string figures). Additionally, there are potential engineering applications involving cables