Practical Processing Techniques for Magnetic 3D Motion Tracking

Tohoku University北村喜文課

MR Safe Robotic Manipulator for MRI-Guided Intracardiac Catheterization

This paper introduces a robotic manipulator to realize robot-assisted intracardiac catheterization in magnetic resonance imaging (MRI) environment. MRI can offer high-resolution images to visualize soft tissue features such as scars or edema. We hypothesize that robotic catheterization, combined with the enhanced monitoring of lesions creation using MRI intraoperatively, will significantly improve the procedural safety, accuracy, and effectiveness. This is designed particularly for cardiac electrophysiological (EP) intervention, which is an effective treatment of arrhythmia. We present the first MR Safe robot for intracardiac EP intervention. The robot actuation features small hysteresis, effective force transmission, and quick response, which has been experimentally verified for its capability to precisely telemanipulate a standard clinically used EP catheter. We also present timely techniques for real-time positional tracking in MRI and intraoperative image registration, which can be integrated with the presented manipulator to im prove the performance of teleoperated robotic catheterization

CRANE: A Redundant, Multi-Degree-of-Freedom Computed Tomography Robot for Heightened Needle Dexterity within a Medical Imaging Bore

Computed Tomography (CT) image guidance enables accurate and safe minimally invasive treatment of diseases, including cancer and chronic pain, with needle-like tools via a percutaneous approach. The physician incrementally inserts and adjusts the needle with intermediate images due to the accuracy limitation of free-hand adjustment and patient physiological motion. Scanning frequency is limited to minimize ionizing radiation exposure for the patient and physician. Robots can provide high positional accuracy and compensate for physiological motion with fewer scans. To accomplish this, the robots must operate within the confined imaging bore while retaining sufficient dexterity to insert and manipulate the needle. This paper presents CRANE: CT Robotic Arm and Needle Emplacer, a CT-compatible robot with a design focused on system dexterity that enables physicians to manipulate and insert needles within the scanner bore as naturally as they would be able to by hand. We define abstract and measurable clinically motivated metrics for in-bore dexterity applicable to general-purpose intra-bore image-guided needle placement robots, develop an automatic robot planning and control method for intra-bore needle manipulation and device setup, and demonstrate the redundant linkage design provides dexterity across various human morphology and meets the clinical requirements for target accuracy during an in-situ evaluation.Comment: 20 pages, 13 figures, Transactions on Robotic

A continuum robotic platform for endoscopic non-contact laser surgery: design, control, and preclinical evaluation

The application of laser technologies in surgical interventions has been accepted in the clinical domain due to their atraumatic properties. In addition to manual application of fibre-guided lasers with tissue contact, non-contact transoral laser microsurgery (TLM) of laryngeal tumours has been prevailed in ENT surgery. However, TLM requires many years of surgical training for tumour resection in order to preserve the function of adjacent organs and thus preserve the patient’s quality of life. The positioning of the microscopic laser applicator outside the patient can also impede a direct line-of-sight to the target area due to anatomical variability and limit the working space. Further clinical challenges include positioning the laser focus on the tissue surface, imaging, planning and performing laser ablation, and motion of the target area during surgery. This dissertation aims to address the limitations of TLM through robotic approaches and intraoperative assistance. Although a trend towards minimally invasive surgery is apparent, no highly integrated platform for endoscopic delivery of focused laser radiation is available to date. Likewise, there are no known devices that incorporate scene information from endoscopic imaging into ablation planning and execution. For focusing of the laser beam close to the target tissue, this work first presents miniaturised focusing optics that can be integrated into endoscopic systems. Experimental trials characterise the optical properties and the ablation performance. A robotic platform is realised for manipulation of the focusing optics. This is based on a variable-length continuum manipulator. The latter enables movements of the endoscopic end effector in five degrees of freedom with a mechatronic actuation unit. The kinematic modelling and control of the robot are integrated into a modular framework that is evaluated experimentally. The manipulation of focused laser radiation also requires precise adjustment of the focal position on the tissue. For this purpose, visual, haptic and visual-haptic assistance functions are presented. These support the operator during teleoperation to set an optimal working distance. Advantages of visual-haptic assistance are demonstrated in a user study. The system performance and usability of the overall robotic system are assessed in an additional user study. Analogous to a clinical scenario, the subjects follow predefined target patterns with a laser spot. The mean positioning accuracy of the spot is 0.5 mm. Finally, methods of image-guided robot control are introduced to automate laser ablation. Experiments confirm a positive effect of proposed automation concepts on non-contact laser surgery.Die Anwendung von Lasertechnologien in chirurgischen Interventionen hat sich aufgrund der atraumatischen Eigenschaften in der Klinik etabliert. Neben manueller Applikation von fasergeführten Lasern mit Gewebekontakt hat sich die kontaktfreie transorale Lasermikrochirurgie (TLM) von Tumoren des Larynx in der HNO-Chirurgie durchgesetzt. Die TLM erfordert zur Tumorresektion jedoch ein langjähriges chirurgisches Training, um die Funktion der angrenzenden Organe zu sichern und damit die Lebensqualität der Patienten zu erhalten. Die Positionierung des mikroskopis chen Laserapplikators außerhalb des Patienten kann zudem die direkte Sicht auf das Zielgebiet durch anatomische Variabilität erschweren und den Arbeitsraum einschränken. Weitere klinische Herausforderungen betreffen die Positionierung des Laserfokus auf der Gewebeoberfläche, die Bildgebung, die Planung und Ausführung der Laserablation sowie intraoperative Bewegungen des Zielgebietes. Die vorliegende Dissertation zielt darauf ab, die Limitierungen der TLM durch robotische Ansätze und intraoperative Assistenz zu adressieren. Obwohl ein Trend zur minimal invasiven Chirurgie besteht, sind bislang keine hochintegrierten Plattformen für die endoskopische Applikation fokussierter Laserstrahlung verfügbar. Ebenfalls sind keine Systeme bekannt, die Szeneninformationen aus der endoskopischen Bildgebung in die Ablationsplanung und -ausführung einbeziehen. Für eine situsnahe Fokussierung des Laserstrahls wird in dieser Arbeit zunächst eine miniaturisierte Fokussieroptik zur Integration in endoskopische Systeme vorgestellt. Experimentelle Versuche charakterisieren die optischen Eigenschaften und das Ablationsverhalten. Zur Manipulation der Fokussieroptik wird eine robotische Plattform realisiert. Diese basiert auf einem längenveränderlichen Kontinuumsmanipulator. Letzterer ermöglicht in Kombination mit einer mechatronischen Aktuierungseinheit Bewegungen des Endoskopkopfes in fünf Freiheitsgraden. Die kinematische Modellierung und Regelung des Systems werden in ein modulares Framework eingebunden und evaluiert. Die Manipulation fokussierter Laserstrahlung erfordert zudem eine präzise Anpassung der Fokuslage auf das Gewebe. Dafür werden visuelle, haptische und visuell haptische Assistenzfunktionen eingeführt. Diese unterstützen den Anwender bei Teleoperation zur Einstellung eines optimalen Arbeitsabstandes. In einer Anwenderstudie werden Vorteile der visuell-haptischen Assistenz nachgewiesen. Die Systemperformanz und Gebrauchstauglichkeit des robotischen Gesamtsystems werden in einer weiteren Anwenderstudie untersucht. Analog zu einem klinischen Einsatz verfolgen die Probanden mit einem Laserspot vorgegebene Sollpfade. Die mittlere Positioniergenauigkeit des Spots beträgt dabei 0,5 mm. Zur Automatisierung der Ablation werden abschließend Methoden der bildgestützten Regelung vorgestellt. Experimente bestätigen einen positiven Effekt der Automationskonzepte für die kontaktfreie Laserchirurgie

Teknik Hand Tracking Menggunakan Metode Inverse Kinematics Pada Pembuatan Animasi 3D

Tahap produksi animasi 3D semakin berkembang. Banyak teknik – teknik baru yang terapkan pada setiap proses produksinya. Belum banyak produksi animasi yang menggunakan Teknik baru, sehingga memungkinkan rumah produksi dapat melakukan Teknik tersebut sebagai salah satu cara meningkatkan kualitas produksi animasi di dalam negeri. Salah satu Teknik yang sering digunakan dalam memproduksi animasi 3D adalah motion tracking. Teknik hand tracking merupakan salah satu dari Teknik motion tracking yang lebih berfokus pada organ tangan. Motion Tracking merupakan sebuah proses dalam mendapatkan tracker dari marker sebuah footage yang telah dipasangkan marker. Marker yang dihasilkan dari proses motion tracking dapat disebut sebagai tracer. Tracer dapat dihubungkan dengan objek 3D yang telah lengkap modeling dan rigging. Sehingga dapat menghasilkan animasi dari pergerakan tracer yang terhunbung dengan bone modeling 3d. Menggunakan pendekatan metode inverse kinematics dilakukan untuk memperbaiki pada saat animasi yang dihasilkan lebih terkontrol pergerakannya dan dengan menambahkan DOF (Degree Of Freedom) pada setiap ruang gerak model yang telah terhubungkan tracer dengan bone (tulang) riggingnya menghindari gerakan animasi keluar dari jalurnya

Human to robot hand motion mapping methods: review and classification

In this article, the variety of approaches proposed in literature to address the problem of mapping human to robot hand motions are summarized and discussed. We particularly attempt to organize under macro-categories the great quantity of presented methods, that are often difficult to be seen from a general point of view due to different fields of application, specific use of algorithms, terminology and declared goals of the mappings. Firstly, a brief historical overview is reported, in order to provide a look on the emergence of the human to robot hand mapping problem as a both conceptual and analytical challenge that is still open nowadays. Thereafter, the survey mainly focuses on a classification of modern mapping methods under six categories: direct joint, direct Cartesian, taskoriented, dimensionality reduction based, pose recognition based and hybrid mappings. For each of these categories, the general view that associates the related reported studies is provided, and representative references are highlighted. Finally, a concluding discussion along with the authors’ point of view regarding future desirable trends are reported.This work was supported in part by the European Commission’s Horizon 2020 Framework Programme with the project REMODEL under Grant 870133 and in part by the Spanish Government under Grant PID2020-114819GB-I00.Peer ReviewedPostprint (published version

Capture and generalisation of close interaction with objects

Robust manipulation capture and retargeting has been a longstanding goal in both the fields of animation and robotics. In this thesis I describe a new approach to capture both the geometry and motion of interactions with objects, dealing with the problems of occlusion by the use of magnetic systems, and performing the reconstruction of the geometry by an RGB-D sensor alongside visual markers. This ‘interaction capture’ allows the scene to be described in terms of the spatial relationships between the character and the object using novel topological representations such as the Electric Parameters, which parametrise the outer space of an object using properties of the surface of the object. I describe the properties of these representations for motion generalisation and discuss how they can be applied to the problems of human-like motion generation and programming by demonstration. These generalised interactions are shown to be valid by demonstration of retargeting grasping and manipulation to robots with dissimilar kinematics and morphology using only local, gradient-based planning

A minimally invasive surgical system for 3D ultrasound guided robotic retrieval of foreign bodies from a beating heart

The result of various medical conditions and trauma, foreign bodies in the heart pose a serious health risk as they may interfere with cardiovascular function. Particles such as thrombi, bullet fragments, and shrapnel can become trapped in a person's heart after migrating through the venous system, or by direct penetration. The severity of disruption can range from benign to fatal, with associated symptoms including anxiety, fever, cardiac tamponade, hemorrhage, infection, embolism, arrhythmia, and valve dysfunction. Injuries of this nature are common in both civilian and military populations. For symptomatic cases, conventional treatment is removal of the foreign body through open surgery via a median sternotomy, the use of cardiopulmonary bypass, and a wide incision in the heart muscle; these methods incur pronounced perioperative risks and long recovery periods. In order to improve upon the standard of care, we propose an image guided robotic system and a corresponding minimally invasive surgical approach. The system employs a dexterous robotic capture device that can maneuver inside the heart through a small incision. Visualization and guidance within the otherwise occluded internal regions are provided by 3D transesophageal echocardiography (TEE), an emerging form of intraoperative medical imaging used in interventions such as mitral valve repair and device implantation. A robotic approach, as opposed to a manual procedure using rigid instruments, is motivated by the various challenges inherent in minimally invasive surgery, which arise from attempts to perform skilled surgical tasks through small incisions without direct vision. Challenges include reduced dexterity, constrained workspace, limited visualization, and difficult hand-eye coordination, which ultimately lead to poor manipulability. A dexterous robotic end effector with real-time image guidance can help overcome these challenges and potentially improve surgical performance. However promising, such a system and approach require that several technical hurdles be resolved. The foreign body must be automatically tracked as it travels about the dynamic environment of the heart. The erratically moving particle must then be captured using a dexterous robot that moves much more slowly in comparison. Furthermore, retrieval must be performed under 3D ultrasound guidance, amidst the uncertainties presented by both the turbulent flow and by the imaging modality itself. In addressing such barriers, this thesis explores the development of a prototype system capable of retrieving a foreign body from a beating heart, culminating in a set of demonstrative in vitro experiments

Musical combinatorics, tonnetz, and the CubeHarmonic

In this paper, we give an overview of some applications of combinatorics and permutations in music through the centuries. The concepts of permutation and tonnetz (spatial representation of voice leading and modulation) can be joined together in a physical device, the CubeHarmonic, a musical version of the Rubik’s cube. We finally describe a prototype of the CubeHarmonic that uses the magnetic tracking technology developed at the Tohoku University