Development of a Novel Handheld Device for Active Compensation of Physiological Tremor

In microsurgery, the human hand imposes certain limitations in accurately positioning the tip of a device such as scalpel. Any errors in the motion of the hand make microsurgical procedures difficult and involuntary motions such as hand tremors can make some procedures significantly difficult to perform. This is particularly true in the case of vitreoretinal microsurgery. The most familiar source of involuntary motion is physiological tremor. Real-time compensation of tremor is, therefore, necessary to assist surgeons to precisely position and manipulate the tool-tip to accurately perform a microsurgery. In this thesis, a novel handheld device (AID) is described for compensation of physiological tremor in the hand. MEMS-based accelerometers and gyroscopes have been used for sensing the motion of the hand in six degrees of freedom (DOF). An augmented state complementary Kalman filter is used to calculate 2 DOF orientation. An adaptive filtering algorithm, band-limited Multiple Fourier linear combiner (BMFLC), is used to calculate the tremor component in the hand in real-time. Ionic Polymer Metallic Composites (IPMCs) have been used as actuators for deflecting the tool-tip to compensate for the tremor

Drift-Free Position Estimation of Periodic or Quasi-Periodic Motion Using Inertial Sensors

Position sensing with inertial sensors such as accelerometers and gyroscopes usually requires other aided sensors or prior knowledge of motion characteristics to remove position drift resulting from integration of acceleration or velocity so as to obtain accurate position estimation. A method based on analytical integration has previously been developed to obtain accurate position estimate of periodic or quasi-periodic motion from inertial sensors using prior knowledge of the motion but without using aided sensors. In this paper, a new method is proposed which employs linear filtering stage coupled with adaptive filtering stage to remove drift and attenuation. The prior knowledge of the motion the proposed method requires is only approximate band of frequencies of the motion. Existing adaptive filtering methods based on Fourier series such as weighted-frequency Fourier linear combiner (WFLC), and band-limited multiple Fourier linear combiner (BMFLC) are modified to combine with the proposed method. To validate and compare the performance of the proposed method with the method based on analytical integration, simulation study is performed using periodic signals as well as real physiological tremor data, and real-time experiments are conducted using an ADXL-203 accelerometer. Results demonstrate that the performance of the proposed method outperforms the existing analytical integration method

Integration of Local Positioning System & Strapdown Inertial Navigation System for Hand-Held Tool Tracking

This research concerns the development of a smart sensory system for tracking a hand-held moving device to millimeter accuracy, for slow or nearly static applications over extended periods of time. Since different operators in different applications may use the system, the proposed design should provide the accurate position, orientation, and velocity of the object without relying on the knowledge of its operation and environment, and based purely on the motion that the object experiences. This thesis proposes the design of the integration a low-cost Local Positioning System (LPS) and a low-cost StrapDown Inertial Navigation System (SDINS) with the association of the modified EKF to determine 3D position and 3D orientation of a hand-held tool within a required accuracy. A hybrid LPS/SDINS combines and complements the best features of two different navigation systems, providing a unique solution to track and localize a moving object more precisely. SDINS provides continuous estimates of all components of a motion, but SDINS loses its accuracy over time because of inertial sensors drift and inherent noise. LPS has the advantage that it can possibly get absolute position and velocity independent of operation time; however, it is not highly robust, is computationally quite expensive, and exhibits low measurement rate. This research consists of three major parts: developing a multi-camera vision system as a reliable and cost-effective LPS, developing a SDINS for a hand-held tool, and developing a Kalman filter for sensor fusion. Developing the multi-camera vision system includes mounting the cameras around the workspace, calibrating the cameras, capturing images, applying image processing algorithms and features extraction for every single frame from each camera, and estimating the 3D position from 2D images. In this research, the specific configuration for setting up the multi-camera vision system is proposed to reduce the loss of line of sight as much as possible. The number of cameras, the position of the cameras with respect to each other, and the position and the orientation of the cameras with respect to the center of the world coordinate system are the crucial characteristics in this configuration. The proposed multi-camera vision system is implemented by employing four CCD cameras which are fixed in the navigation frame and their lenses placed on semicircle. All cameras are connected to a PC through the frame grabber, which includes four parallel video channels and is able to capture images from four cameras simultaneously. As a result of this arrangement, a wide circular field of view is initiated with less loss of line-of-sight. However, the calibration is more difficult than a monocular or stereo vision system. The calibration of the multi-camera vision system includes the precise camera modeling, single camera calibration for each camera, stereo camera calibration for each two neighboring cameras, defining a unique world coordinate system, and finding the transformation from each camera frame to the world coordinate system. Aside from the calibration procedure, digital image processing is required to be applied into the images captured by all four cameras in order to localize the tool tip. In this research, the digital image processing includes image enhancement, edge detection, boundary detection, and morphologic operations. After detecting the tool tip in each image captured by each camera, triangulation procedure and optimization algorithm are applied in order to find its 3D position with respect to the known navigation frame. In the SDINS, inertial sensors are mounted rigidly and directly to the body of the tracking object and the inertial measurements are transformed computationally to the known navigation frame. Usually, three gyros and three accelerometers, or a three-axis gyro and a three-axis accelerometer are used for implementing SDINS. The inertial sensors are typically integrated in an inertial measurement unit (IMU). IMUs commonly suffer from bias drift, scale-factor error owing to non-linearity and temperature changes, and misalignment as a result of minor manufacturing defects. Since all these errors lead to SDINS drift in position and orientation, a precise calibration procedure is required to compensate for these errors. The precision of the SDINS depends not only on the accuracy of calibration parameters but also on the common motion-dependent errors. The common motion-dependent errors refer to the errors caused by vibration, coning motion, sculling, and rotational motion. Since inertial sensors provide the full range of heading changes, turn rates, and applied forces that the object is experiencing along its movement, accurate 3D kinematics equations are developed to compensate for the common motion-dependent errors. Therefore, finding the complete knowledge of the motion and orientation of the tool tip requires significant computational complexity and challenges relating to resolution of specific forces, attitude computation, gravity compensation, and corrections for common motion-dependent errors. The Kalman filter technique is a powerful method for improving the output estimation and reducing the effect of the sensor drift. In this research, the modified EKF is proposed to reduce the error of position estimation. The proposed multi-camera vision system data with cooperation of the modified EKF assists the SDINS to deal with the drift problem. This configuration guarantees the real-time position and orientation tracking of the instrument. As a result of the proposed Kalman filter, the effect of the gravitational force in the state-space model will be removed and the error which results from inaccurate gravitational force is eliminated. In addition, the resulting position is smooth and ripple-free. The experimental results of the hybrid vision/SDINS design show that the position error of the tool tip in all directions is about one millimeter RMS. If the sampling rate of the vision system decreases from 20 fps to 5 fps, the errors are still acceptable for many applications

Passive and active assistive writing devices in suppressing hand tremor

Patients with hand tremor disease frequently experience difficulties in performing their daily tasks, especially in handwriting activities. In order to prevent the ingestion of drugs and intervention of surgeries, a non-invasive solution was presented to improve their writing capabilities. In this study, there were two novel inventions of the hand-held device named as TREMORX and Active Assistive Writing Device (AAWD) with the approaches of passive and active elements respectively. For validation, the patient with tremor was assisted in using a normal pen and TREMORX to perform a handwriting task at the sitting and standing postures. For AAWD, the active suppressing element was the servo motor to control the hand tremor act on the writing tool tip and an accelerometer will measure the necessary parameters values for feedback control signal. The classic Proportional (P) controller and Proportional-Integral- Derivative (PID) were presented. The P controller was tuned with a meta-heuristic method by adjusting the parameters into several values to examine the response and robustness of the controller in suppressing the tremor. The evaluation was based on decreasing the coherence magnitude on the frequency response analysis. To optimise the performances, two types of Evolutionary Algorithms (EA) were employed which were Genetic Algorithm (GA) and Particle Swarm Optimisation (PSO). The optimisation techniques were integrated into the PID controller system to generate the optimum performances in controlling the tremor. For the simulation study, the parametric model representing the actual system of the AAWD was presented. The main objectives of this analysis were to determine the optimum value of PID parameters based on EA optimisation techniques. The determined parameters for both optimisations were then injected into the experimental environment to test and evaluate the performance of the controllers. The findings of the study exhibited that the PID controller for both EA optimisation provided excellent performances in suppressing the tremor signal act on the AAWD in comparison to the classic pure P controller. Based on the fitness evaluation, the GA optimisation significantly enhanced the PID controller performance compared to PSO optimisation. The handwriting performance using both TRREMORX and AAWD was recorded and from a visual justification, it showed that the quality of legibility was improved as compared with using normal handwriting devices. These outcomes provided an important contribution towards achieving novel methods in suppressing hand tremor by means of the invention of the handheld writing devices incorporated with intelligent control techniques

Classifying Human Leg Motions with Uniaxial Piezoelectric Gyroscopes

This paper provides a comparative study on the different techniques of classifying human leg motions that are performed using two low-cost uniaxial piezoelectric gyroscopes worn on the leg. A number of feature sets, extracted from the raw inertial sensor data in different ways, are used in the classification process. The classification techniques implemented and compared in this study are: Bayesian decision making (BDM), a rule-based algorithm (RBA) or decision tree, least-squares method (LSM), k-nearest neighbor algorithm (k-NN), dynamic time warping (DTW), support vector machines (SVM), and artificial neural networks (ANN). A performance comparison of these classification techniques is provided in terms of their correct differentiation rates, confusion matrices, computational cost, and training and storage requirements. Three different cross-validation techniques are employed to validate the classifiers. The results indicate that BDM, in general, results in the highest correct classification rate with relatively small computational cost

The Hand-Held Force Magnifier: Surgical Tools to Augment the Sense of Touch

Modern surgeons routinely perform procedures with noisy, sub-threshold, or obscured visual and haptic feedback,either due to the necessary approach, or because the systems on which they are operating are exceeding delicate. For example, in cataract extraction, ophthalmic surgeons must peel away thin membranes in order to access and replace the lens of the eye. Elsewhere, dissection is now commonly performed with energy-delivering tools – rather than sharp blades – and damage to deep structures is possible if tissue contact is not well controlled. Surgeons compensate for their lack of tactile sensibility by relying solely on visual feedback, observing tissue deformation and other visual cues through surgical microscopes or cameras. Using visual information alone can make a procedure more difficult, because cognitive mediation is required to convert visual feedback into motor action. We call this the “haptic problem” in surgery because the human sensorimotor loop is deprived of critical tactile afferent information, increasing the chance for intraoperative injury and requiring extensive training before clinicians reach independent proficiency. Tools that enhance the surgeon’s direct perception of tool-tissue forces can therefore potentially reduce the risk of iatrogenic complications and improve patient outcomes. Towards this end, we have developed and characterized a new robotic surgical tool, the Hand-Held Force Magnifier (HHFM), which amplifies forces at the tool tip so they may be readily perceived by the user, a paradigm we call “in-situ” force feedback. In this dissertation, we describe the development of successive generations of HHFM prototypes, and the evaluation of a proposed human-in-the-loop control framework using the methods of psychophysics. Using these techniques, we have verified that our tool can reduce sensory perception thresholds, augmenting the user’s abilities beyond what is normally possible. Further, we have created models of human motor control in surgically relevant tasks such as membrane puncture, which have shown to be sensitive to push-pull direction and handedness effects. Force augmentation has also demonstrated improvements to force control in isometric force generation tasks. Finally, in support of future psychophysics work, we have developed an inexpensive, high-bandwidth, single axis haptic renderer using a commercial audio speaker

Robocatch: Design and Making of a Hand-Held Spillage-Free Specimen Retrieval Robot for Laparoscopic Surgery

Specimen retrieval is an important step in laparoscopy, a minimally invasive surgical procedure performed to diagnose and treat a myriad of medical pathologies in fields ranging from gynecology to oncology. Specimen retrieval bags (SRBs) are used to facilitate this task, while minimizing contamination of neighboring tissues and port-sites in the abdominal cavity. This manual surgical procedure requires usage of multiple ports, creating a traffic of simultaneous operations of multiple instruments in a limited shared workspace. The skill-demanding nature of this procedure makes it time-consuming, leading to surgeons’ fatigue and operational inefficiency. This thesis presents the design and making of RoboCatch, a novel hand-held robot that aids a surgeon in performing spillage-free retrieval of operative specimens in laparoscopic surgery. The proposed design significantly modifies and extends conventional instruments that are currently used by surgeons for the retrieval task: The core instrumentation of RoboCatch comprises a webbed three-fingered grasper and atraumatic forceps that are concentrically situated in a folded configuration inside a trocar. The specimen retrieval task is achieved in six stages: 1) The trocar is introduced into the surgical site through an instrument port, 2) the three webbed fingers slide out of the tube and simultaneously unfold in an umbrella like-fashion, 3) the forceps slide toward, and grasp, the excised specimen, 4) the forceps retract the grasped specimen into the center of the surrounding grasper, 5) the grasper closes to achieve a secured containment of the specimen, and 6) the grasper, along with the contained specimen, is manually removed from the abdominal cavity. The resulting reduction in the number of active ports reduces obstruction of the port-site and increases the procedure’s efficiency. The design process was initiated by acquiring crucial parameters from surgeons and creating a design table, which informed the CAD modeling of the robot structure and selection of actuation units and fabrication material. The robot prototype was first examined in CAD simulation and then fabricated using an Objet30 Prime 3D printer. Physical validation experiments were conducted to verify the functionality of different mechanisms of the robot. Further, specimen retrieval experiments were conducted with porcine meat samples to test the feasibility of the proposed design. Experimental results revealed that the robot was capable of retrieving masses of specimen ranging from 1 gram to 50 grams. The making of RoboCatch represents a significant step toward advancing the frontiers of hand-held robots for performing specimen retrieval tasks in minimally invasive surgery

Physiological Tremor in Handgun Aiming and Shooting Tasks

When holding an outstretched limb or aiming at a target, humans produce small involuntary fluctuations that may hamper performance. Current strategies for minimizing the impact of tremulous oscillations predominantly include both extrinsic and intrinsic support. The aim of the current dissertation is to better understand the parameters of physiological tremor associated with handgun aiming with the end goal of improving shooting accuracy. Experiment 1 focused on handgun aiming and the influence of different arm posture adopted during aiming. Experiment 2 expanded upon the findings of experiment 1 by comparing tremor during finger pointing, handgun aiming, and handgun shooting. Experiment 3 attempted to confirm that both mechanical support and proprioceptive feedback play a role in both attenuation of tremor amplitude and handgun shooting accuracy. In experiment 1, thirty volunteers stood 6.4 meters from a target and aimed a weighted mock handgun for 10 seconds per trial. Two hand grips (bilateral, unilateral) and two arm positions (bent elbow, straight elbow) were assessed for acceleration in the anterior-posterior (AP), medial-lateral (ML), and vertical (VT) directions. Amplitude, regularity, and a frequency spectrum analysis of the acceleration signals were analyzed. Tremor amplitudes (VT, ML) were reduced using a bilateral grip and by bending the elbows. The irregularity of the tremor signal was increased by using two hands to support the handgun. Interestingly, irrespective of the posture adopted, ML accelerations were of greater amplitude than VT oscillations. AP oscillations were markedly smaller compared to VT and ML tremor, did not display consistent frequency peaks, and were not altered by the arm conditions. During experiment 2, twenty volunteers, in a counterbalanced order, pointed their finger, aimed a training handgun, or shot a training handgun, for 10 seconds at a bullseye target 6.4 meters away. Amplitude, regularity, and frequency spectrum analysis of the acceleration signals were computed. Aiming with the mass of a gun in the hand has primarily a damping effect on the amplitude of tremor in the distal segments as well as resulting in more regular movements. Overall, aiming with a gun and pointing with a finger were similar tasks except for the added mass of the handgun aiming condition. Shooting accuracy and handgun shooting experience were also assessed for correlations with acceleration amplitude and regularity. Both handgun shooting accuracy and experience revealed a stronger correlation with increased irregularity of the acceleration signal than decreased acceleration amplitude. A correlation was also run between shooting accuracy and handgun shooting experience. An increase in accuracy had a significant, moderate relationship with an increase in handgun shooting experience. Experiment 3 had twenty volunteers aim as well as shoot a training handgun at a bullseye target 6.4 meters away during two limb support conditions and two weight conditions for a total of four combinations. Amplitude, regularity, and frequency spectrum analysis of the acceleration signals were computed. Bilateral limb support again reduced tremor amplitude and increased the irregularity of the acceleration signal over unilateral conditions. Bilateral limb support also contributed to a significantly improved handgun shooting accuracy when compared to unilateral limb support conditions. By manipulating the weight of the handgun, the third study also indicated the addition of a second limb reduced acceleration amplitude through both mechanical support and proprioceptive feedback. The experiments demonstrate that finger pointing and handgun aiming share similar tremulous characteristics in all three directions (VT, ML, AP). These experiments also indicate that acceleration amplitude can be reduced while acceleration regularity and shooting accuracy are increased through the use of a bilateral limb support posture

Body sensor networks: smart monitoring solutions after reconstructive surgery

Advances in reconstructive surgery are providing treatment options in the face of major trauma and cancer. Body Sensor Networks (BSN) have the potential to offer smart solutions to a range of clinical challenges. The aim of this thesis was to review the current state of the art devices, then develop and apply bespoke technologies developed by the Hamlyn Centre BSN engineering team supported by the EPSRC ESPRIT programme to deliver post-operative monitoring options for patients undergoing reconstructive surgery. A wireless optical sensor was developed to provide a continuous monitoring solution for free tissue transplants (free flaps). By recording backscattered light from 2 different source wavelengths, we were able to estimate the oxygenation of the superficial microvasculature. In a custom-made upper limb pressure cuff model, forearm deoxygenation measured by our sensor and gold standard equipment showed strong correlations, with incremental reductions in response to increased cuff inflation durations. Such a device might allow early detection of flap failure, optimising the likelihood of flap salvage. An ear-worn activity recognition sensor was utilised to provide a platform capable of facilitating objective assessment of functional mobility. This work evolved from an initial feasibility study in a knee replacement cohort, to a larger clinical trial designed to establish a novel mobility score in patients recovering from open tibial fractures (OTF). The Hamlyn Mobility Score (HMS) assesses mobility over 3 activities of daily living: walking, stair climbing, and standing from a chair. Sensor-derived parameters including variation in both temporal and force aspects of gait were validated to measure differences in performance in line with fracture severity, which also matched questionnaire-based assessments. Monitoring the OTF cohort over 12 months with the HMS allowed functional recovery to be profiled in great detail. Further, a novel finding of continued improvements in walking quality after a plateau in walking quantity was demonstrated objectively. The methods described in this thesis provide an opportunity to revamp the recovery paradigm through continuous, objective patient monitoring along with self-directed, personalised rehabilitation strategies, which has the potential to improve both the quality and cost-effectiveness of reconstructive surgery services.Open Acces