SPRK: A Low-Cost Stewart Platform For Motion Study In Surgical Robotics

To simulate body organ motion due to breathing, heart beats, or peristaltic movements, we designed a low-cost, miniaturized SPRK (Stewart Platform Research Kit) to translate and rotate phantom tissue. This platform is 20cm x 20cm x 10cm to fit in the workspace of a da Vinci Research Kit (DVRK) surgical robot and costs $250, two orders of magnitude less than a commercial Stewart platform. The platform has a range of motion of +/- 1.27 cm in translation along x, y, and z directions and has motion modes for sinusoidal motion and breathing-inspired motion. Modular platform mounts were also designed for pattern cutting and debridement experiments. The platform's positional controller has a time-constant of 0.2 seconds and the root-mean-square error is 1.22 mm, 1.07 mm, and 0.20 mm in x, y, and z directions respectively. All the details, CAD models, and control software for the platform is available at github.com/BerkeleyAutomation/sprk

Sliding to predict: vision-based beating heart motion estimation by modeling temporal interactions

Purpose: Technical advancements have been part of modern medical solutions as they promote better surgical alternatives that serve to the benefit of patients. Particularly with cardiovascular surgeries, robotic surgical systems enable surgeons to perform delicate procedures on a beating heart, avoiding the complications of cardiac arrest. This advantage comes with the price of having to deal with a dynamic target which presents technical challenges for the surgical system. In this work, we propose a solution for cardiac motion estimation. Methods: Our estimation approach uses a variational framework that guarantees preservation of the complex anatomy of the heart. An advantage of our approach is that it takes into account different disturbances, such as specular reflections and occlusion events. This is achieved by performing a preprocessing step that eliminates the specular highlights and a predicting step, based on a conditional restricted Boltzmann machine, that recovers missing information caused by partial occlusions. Results: We carried out exhaustive experimentations on two datasets, one from a phantom and the other from an in vivo procedure. The results show that our visual approach reaches an average minima in the order of magnitude of 10-7 while preserving the heart’s anatomical structure and providing stable values for the Jacobian determinant ranging from 0.917 to 1.015. We also show that our specular elimination approach reaches an accuracy of 99% compared to a ground truth. In terms of prediction, our approach compared favorably against two well-known predictors, NARX and EKF, giving the lowest average RMSE of 0.071. Conclusion: Our approach avoids the risks of using mechanical stabilizers and can also be effective for acquiring the motion of organs other than the heart, such as the lung or other deformable objects.Peer ReviewedPostprint (published version

Motion Estimation and Reconstruction of a Heart Surface by Means of 2D-/3D-Membrane Models

In order to assist surgeons during minimally invasive interventions on the beating heart, it would be helpful to develop a robotic surgery system, which synchronizes the instruments with the heart surface, so that their positions do not change relative to the point of interest (POI). The synchronization of the robotic manipulators requires an estimation of the heart surface motion. In this paper, a modelbased motion estimation of the heart surface is presented. The motion of a partition of the heart surface is modelled by means of a thin or thick vibrating membrane in order to represent the epicardial surface or the connected epicard and myocard. The membrane motion is described by means of a system of coupled linear partial differential equations (PDEs), whose 3D-input function is assumed to be known. After spatial discretization of the PDE solution space by the Finite Spectral Element Method, a bank of lumped systems is obtained. A Kalman filter is used to estimate the state of the lumped systems by incorporating noisy measurements of the heart surface. Measurements can be the position or velocity of sonomicrometry-based sensors or of certain landmarks, which are tracked by optical sensors. With the model-based estimation it is possible to reconstruct the entire partition of the heart surface even at non-measurement points and thus at each POI

Design Space Exploration in Cyber-Physical Systems

Cyber physical systems (CPS) integrate a variety of engineering areas such as control, mechanical and computer engineering in a holistic design effort. While interdependencies between the different disciplines are key attributes of CPS design science, little is known about the impact of design decisions of the cyber part on the overall system qualities. To investigate these interdependencies, this paper proposes a simulation-based Design Space Exploration (DSE) framework that considers detailed cyber system parameters such as cache size, bus width, and voltage levels in addition to physical and control parameters of the CPS. We propose an exploration algorithm that surfs the parameter configurations in the cyber physical sub-systems, in order to approximate the Pareto-optimal design points with regards to the trade-os among the design objectives, such as energy consumption and control stability. We apply the proposed framework to a network control system for an inverted-pendulum application. The presented holistic evaluation of the identified Pareto-points reveals the presence of non-trivial trade-os, which are imposed by the control, physical, and detailed cyber parameters. For instance the identified energy and control optimal design points comprise configurations with a wide range of CPU speeds, sample times and cache configuration following non-trivial zig-zag patterns. The proposed framework could identify and manage those trade-os and, as a result, is an imperative rst step to automate the search for superior CSP configurations

3D Ultrasound-Guided Motion Compensation System for Beating Heart Mitral Valve Repair

Beating heart intracardiac procedures promise significant benefits for patients, however, the fast motion of the heart poses serious challenges to surgeons. We present a new 3D ultrasound-guided motion (3DUS) compensation system that synchronizes instrument motion with the heart. The system utilizes the fact that the motion of some intracardiac structures, including the mitral valve annulus, is largely constrained to translation along one axis. This allows the development of a real-time 3DUS tissue tracker which we integrate with a 1 degree-of-freedom actuated surgical instrument, real-time 3DUS instrument tracker, and predictive filter to devise a system with synchronization accuracy of 1.8 mm RMSE. User studies involving the deployment of surgical anchors in a simulated mitral annuloplasty procedure demonstrate that the system increases success rates by over 100%. Furthermore, it enables more careful anchor deployment by reducing forces to the tissue by 50% while allowing instruments to remain in contact with the tissue for longer periods.Engineering and Applied Science

Sliding to predict: vision-based beating heart motion estimation by modeling temporal interactions.

PURPOSE: Technical advancements have been part of modern medical solutions as they promote better surgical alternatives that serve to the benefit of patients. Particularly with cardiovascular surgeries, robotic surgical systems enable surgeons to perform delicate procedures on a beating heart, avoiding the complications of cardiac arrest. This advantage comes with the price of having to deal with a dynamic target which presents technical challenges for the surgical system. In this work, we propose a solution for cardiac motion estimation. METHODS: Our estimation approach uses a variational framework that guarantees preservation of the complex anatomy of the heart. An advantage of our approach is that it takes into account different disturbances, such as specular reflections and occlusion events. This is achieved by performing a preprocessing step that eliminates the specular highlights and a predicting step, based on a conditional restricted Boltzmann machine, that recovers missing information caused by partial occlusions. RESULTS: We carried out exhaustive experimentations on two datasets, one from a phantom and the other from an in vivo procedure. The results show that our visual approach reaches an average minima in the order of magnitude of [Formula: see text] while preserving the heart's anatomical structure and providing stable values for the Jacobian determinant ranging from 0.917 to 1.015. We also show that our specular elimination approach reaches an accuracy of 99% compared to a ground truth. In terms of prediction, our approach compared favorably against two well-known predictors, NARX and EKF, giving the lowest average RMSE of 0.071. CONCLUSION: Our approach avoids the risks of using mechanical stabilizers and can also be effective for acquiring the motion of organs other than the heart, such as the lung or other deformable objects

Simultaneous State and Parameter Estimation for Physics-Based Tracking of Heart Surface Motion

Most existing approaches for tracking of the beating heart motion assume known cardiac kinematics and material parameters. However, these assumptions are not realistic for application in beating heart surgery. In this paper, a novel probabilistic tracking approach based on a physical model of the heart surface is presented. In contrast to existing approaches, the physical information about heart kinematics and material properties is incorporated and considered in an estimation of the heart behavior. An additional advantage is that the time-dependencies and uncertainties of the heart parameters are efficiently handled by exploiting simultaneous state and parameter estimation. Furthermore, by decomposing the state into linear and nonlinear substructures, the computational complexity of the estimation problem is reduced. The experimental results demonstrate the high performance of the method proposed in this paper. The solution of the parameter identification problem allows a personalized physical model and opens up possibilities to apply the physics-based tracking of the heart surface motion in a clinical environment

Adaptive Model-Based Visual Stabilization of Image Sequences Using Feedback

Visual stabilization proposed in this paper compensates changes of the scene caused by motion and deformation of an observed object. This is of high importance in computer-assisted beating heart surgery, where the views of the beating heart should be stabilized. The proposed model-based method defines visual stabilization as a transformation of the current image sequence to a stabilized image sequence. This transformation incorporates physical model of the observed object and model of the measurement process. In contrast to standard approaches, the quality of the visual stabilization is continuously evaluated and improved in two aspects. On the one hand, discretization errors are reduced. On the other hand, the parameters of the underlying models are adjusted. The performance of the proposed method is evaluated in an experiment with a pressure-regulated artificial heart. Compared with standard methods, the model-based method provides higher accuracy, which is additionally improved by a feedback mechanism

Heart Surface Motion Estimation Framework for Robotic Surgery Employing Meshless Methods

A novel heart surface motion estimation frame- work for a robotic surgery on a stabilized beating heart is proposed. It includes an approach for the reconstruction and prediction of heart surface motion based on a novel physical model of the intervention area described by a distributed- parameter system. Instead of conventional element methods, a meshless method is used for a spatial and temporal decomposi- tion of this system. This leads to a finite-dimensional state-space form. Furthermore, the state of the resulting lumped-parameter system, which provides an approximation of the deflection and velocity of the heart surface, is dynamically estimated under consideration of uncertainties both occurring in the system and arising from noisy camera measurements. By using the estimation results, an accurate reconstruction of heart surface motion for the synchronisation of the surgical instruments is also achieved at occluded or non-measurement points