A Framework for Tumor Localization in Robot-Assisted Minimally Invasive Surgery

Manual palpation of tissue is frequently used in open surgery, e.g., for localization of tumors and buried vessels and for tissue characterization. The overall objective of this work is to explore how tissue palpation can be performed in Robot-Assisted Minimally Invasive Surgery (RAMIS) using laparoscopic instruments conventionally used in RAMIS. This thesis presents a framework where a surgical tool is moved teleoperatively in a manner analogous to the repetitive pressing motion of a finger during manual palpation. We interpret the changes in parameters due to this motion such as the applied force and the resulting indentation depth to accurately determine the variation in tissue stiffness. This approach requires the sensorization of the laparoscopic tool for force sensing. In our work, we have used a da Vinci needle driver which has been sensorized in our lab at CSTAR for force sensing using Fiber Bragg Grating (FBG). A computer vision algorithm has been developed for 3D surgical tool-tip tracking using the da Vinci \u27s stereo endoscope. This enables us to measure changes in surface indentation resulting from pressing the needle driver on the tissue. The proposed palpation framework is based on the hypothesis that the indentation depth is inversely proportional to the tissue stiffness when a constant pressing force is applied. This was validated in a telemanipulated setup using the da Vinci surgical system with a phantom in which artificial tumors were embedded to represent areas of different stiffnesses. The region with high stiffness representing tumor and region with low stiffness representing healthy tissue showed an average indentation depth change of 5.19 mm and 10.09 mm respectively while maintaining a maximum force of 8N during robot-assisted palpation. These indentation depth variations were then distinguished using the k-means clustering algorithm to classify groups of low and high stiffnesses. The results were presented in a colour-coded map. The unique feature of this framework is its use of a conventional laparoscopic tool and minimal re-design of the existing da Vinci surgical setup. Additional work includes a vision-based algorithm for tracking the motion of the tissue surface such as that of the lung resulting from respiratory and cardiac motion. The extracted motion information was analyzed to characterize the lung tissue stiffness based on the lateral strain variations as the surface inflates and deflates

Flexible adaptation of iterative learning control with applications to synthetic bone graft manufacturing

Additive manufacturing processes are powerful tools; they are capable of fabricating structures without expensive structure specific tooling -- therefore structure designs can efficiently change from run-to-run -- and they can integrate multiple distinct materials into a single structure. This work investigates one such additive manufacturing process, micro-Robotic Deposition ($\mu$RD), and its utility in fabricating advanced architecture synthetic bone grafts. These bone grafts, also known as synthetic bone scaffolds, are highly porous three-dimensional structures that provide a matrix to support the natural process of bone remodeling. Ideally, the synthetic scaffold will stimulate complete bone healing in a skeletal defect site and also resorb with time so that only natural tissue remains. The objective of this research is to develop methods to integrate different regions with different porous microstructures into a single scaffold; there is evidence that scaffolds with designed regions of specific microstructures can be used to elicit a strong and directed bone ingrowth response that improves bone ingrowth rate and quality. The key contribution of this work is the development of a control algorithm that precisely places different build materials in specified locations, thereby the fabrication of advanced architecture scaffolds is feasible. Under previous control methods, designs were relegated to be composed of a single material. The control algorithm developed in this work is an adaptation of Iterative Learning Control (ILC), a control method that is typically best suited for mass manufacturing applications. This adaptation reorients the ILC framework such that it is more amenable to additive manufacturing systems, such as $\mu$RD. Control efficacy is demonstrated by the fabrication of advanced architecture scaffolds. Scaffolds with contoured forms, multiple domains with distinct porous microstructures, and hollow cavities are feasible when the developed controller is used in conjunction with a novel manufacturing workflow in which scaffolds are filled within patterned molds that support overhanging features. An additional application demonstrates controller performance on the robot positioning problem; this work has implications for additive manufacturing in general

Swarm-Based Techniques for Adaptive Navigation Primitives

Adaptive Navigation (AN) has, in the past, been successfully accomplished by using mobile multi-robot systems (MMS) in highly structured formations known as clusters. Such multi-robot adaptive navigation (MAN) allows for real-time reaction to sensor readings and navigation to a goal location not known a priori. This thesis successfully reproduces MAN cluster techniques via swarm control techniques, a less computationally expensive but less formalized control technique for MMS, which achieves robot control through a combination of primitive robot behaviors. While powerful for large numbers of robots, swarm robotics often relies on “emergent” swarm behaviors resulting from robot-level behaviors, rather than top-down specification of swarm behaviors. For adaptive navigation purposes, it was desired to be able to specify swarm-level behavior from a top down perspective rather than experimenting with emergent behaviors. To this end, a simulation environment was developed to allow rapid development and vetting of swarm behaviors while easily interfacing with an existing testbed for validation on hardware. An initial suite of robot primitive and composite behaviors was developed and vetted using this simulator, and the behaviors were validated using the existing testbed in Santa Clara University’s Robotics System Laboratory (RSL). Of particular importance were the adaptive navigation primitives of extrema finding and contour finding and following. These AN primitives were tested over a variety of experimental parameters, yielding design guidelines for top-down specification of swarm robotic adaptive navigation. These design guidelines are presented, and their usefulness is demonstrated for a Contour Finding and Following application using the RSL’s testbed. Finally, possible future work to expand the capability of swarm-based adaptive navigation techniques is discussed

A Framework for Collaborative Multi-task, Multi-robot Missions

Robotics is a transformative technology that will empower our civilization for a new scale of human endeavors. Massive scale is only possible through the collaboration of individual or groups of robots. Collaboration allows specialization, meaning a multirobot system may accommodate heterogeneous platforms including human partners. This work develops a unified control architecture for collaborative missions comprised of multiple, multi-robot tasks. Using kinematic equations and Jacobian matrices, the system states are transformed into alternative control spaces which are more useful for the designer or more convenient for the operator. The architecture allows multiple tasks to be combined, composing tightly coordinated missions. Using this approach, the designer is able to compensate for non-ideal behavior in the appropriate space using whatever control scheme they choose. This work presents a general design methodology, including analysis techniques for relevant control metrics like stability, responsiveness, and disturbance rejection, which were missing in prior work. Multiple tasks may be combined into a collaborative mission. The unified motion control architecture merges the control space components for each task into a concise federated system to facilitate analysis and implementation. The task coordination function defines task commands as functions of mission commands and state values to create explicit closed-loop collaboration. This work presents analysis techniques to understand the effects of cross-coupling tasks. This work analyzes system stability for the particular control architecture and identifies an explicit condition to ensure stable switching when reallocating robots. We are unaware of any other automated control architectures that address large-scale collaborative systems composed of task-oriented multi-robot coalitions where relative spatial control is critical to mission performance. This architecture and methodology have been validated in experiments and in simulations, repeating earlier work and exploring new scenarios and. It can perform large-scale, complex missions via a rigorous design methodology

COBE's search for structure in the Big Bang

The launch of Cosmic Background Explorer (COBE) and the definition of Earth Observing System (EOS) are two of the major events at NASA-Goddard. The three experiments contained in COBE (Differential Microwave Radiometer (DMR), Far Infrared Absolute Spectrophotometer (FIRAS), and Diffuse Infrared Background Experiment (DIRBE)) are very important in measuring the big bang. DMR measures the isotropy of the cosmic background (direction of the radiation). FIRAS looks at the spectrum over the whole sky, searching for deviations, and DIRBE operates in the infrared part of the spectrum gathering evidence of the earliest galaxy formation. By special techniques, the radiation coming from the solar system will be distinguished from that of extragalactic origin. Unique graphics will be used to represent the temperature of the emitting material. A cosmic event will be modeled of such importance that it will affect cosmological theory for generations to come. EOS will monitor changes in the Earth's geophysics during a whole solar color cycle

Cross-layer Optimized Wireless Video Surveillance

A wireless video surveillance system contains three major components, the video capture and preprocessing, the video compression and transmission over wireless sensor networks (WSNs), and the video analysis at the receiving end. The coordination of different components is important for improving the end-to-end video quality, especially under the communication resource constraint. Cross-layer control proves to be an efficient measure for optimal system configuration. In this dissertation, we address the problem of implementing cross-layer optimization in the wireless video surveillance system. The thesis work is based on three research projects. In the first project, a single PTU (pan-tilt-unit) camera is used for video object tracking. The problem studied is how to improve the quality of the received video by jointly considering the coding and transmission process. The cross-layer controller determines the optimal coding and transmission parameters, according to the dynamic channel condition and the transmission delay. Multiple error concealment strategies are developed utilizing the special property of the PTU camera motion. In the second project, the binocular PTU camera is adopted for video object tracking. The presented work studied the fast disparity estimation algorithm and the 3D video transcoding over the WSN for real-time applications. The disparity/depth information is estimated in a coarse-to-fine manner using both local and global methods. The transcoding is coordinated by the cross-layer controller based on the channel condition and the data rate constraint, in order to achieve the best view synthesis quality. The third project is applied for multi-camera motion capture in remote healthcare monitoring. The challenge is the resource allocation for multiple video sequences. The presented cross-layer design incorporates the delay sensitive, content-aware video coding and transmission, and the adaptive video coding and transmission to ensure the optimal and balanced quality for the multi-view videos. In these projects, interdisciplinary study is conducted to synergize the surveillance system under the cross-layer optimization framework. Experimental results demonstrate the efficiency of the proposed schemes. The challenges of cross-layer design in existing wireless video surveillance systems are also analyzed to enlighten the future work. Adviser: Song C

Cross-layer Optimized Wireless Video Surveillance

A wireless video surveillance system contains three major components, the video capture and preprocessing, the video compression and transmission over wireless sensor networks (WSNs), and the video analysis at the receiving end. The coordination of different components is important for improving the end-to-end video quality, especially under the communication resource constraint. Cross-layer control proves to be an efficient measure for optimal system configuration. In this dissertation, we address the problem of implementing cross-layer optimization in the wireless video surveillance system. The thesis work is based on three research projects. In the first project, a single PTU (pan-tilt-unit) camera is used for video object tracking. The problem studied is how to improve the quality of the received video by jointly considering the coding and transmission process. The cross-layer controller determines the optimal coding and transmission parameters, according to the dynamic channel condition and the transmission delay. Multiple error concealment strategies are developed utilizing the special property of the PTU camera motion. In the second project, the binocular PTU camera is adopted for video object tracking. The presented work studied the fast disparity estimation algorithm and the 3D video transcoding over the WSN for real-time applications. The disparity/depth information is estimated in a coarse-to-fine manner using both local and global methods. The transcoding is coordinated by the cross-layer controller based on the channel condition and the data rate constraint, in order to achieve the best view synthesis quality. The third project is applied for multi-camera motion capture in remote healthcare monitoring. The challenge is the resource allocation for multiple video sequences. The presented cross-layer design incorporates the delay sensitive, content-aware video coding and transmission, and the adaptive video coding and transmission to ensure the optimal and balanced quality for the multi-view videos. In these projects, interdisciplinary study is conducted to synergize the surveillance system under the cross-layer optimization framework. Experimental results demonstrate the efficiency of the proposed schemes. The challenges of cross-layer design in existing wireless video surveillance systems are also analyzed to enlighten the future work. Adviser: Song C

Visual Perception of Garments for their Robotic Manipulation

Tématem předložené práce je strojové vnímání textilií založené na obrazové informaci a využité pro jejich robotickou manipulaci. Práce studuje několik reprezentativních textilií v běžných kognitivně-manipulačních úlohách, jako je například třídění neznámých oděvů podle typu nebo jejich skládání. Některé z těchto činností by v budoucnu mohly být vykonávány domácími robotickými pomocníky. Strojová manipulace s textiliemi je poptávaná také v průmyslu. Hlavní výzvou řešeného problému je měkkost a s tím související vysoká deformovatelnost textilií, které se tak mohou nacházet v bezpočtu vizuálně velmi odlišných stavů.The presented work addresses the visual perception of garments applied for their robotic manipulation. Various types of garments are considered in the typical perception and manipulation tasks, including their classification, folding or unfolding. Our work is motivated by the possibility of having humanoid household robots performing these tasks for us in the future, as well as by the industrial applications. The main challenge is the high deformability of garments, which can be posed in infinitely many configurations with a significantly varying appearance

Proceedings of the NASA Conference on Space Telerobotics, volume 5

Papers presented at the NASA Conference on Space Telerobotics are compiled. The theme of the conference was man-machine collaboration in space. The conference provided a forum for researchers and engineers to exchange ideas on the research and development required for the application of telerobotics technology to the space systems planned for the 1990's and beyond. Volume 5 contains papers related to the following subject areas: robot arm modeling and control, special topics in telerobotics, telerobotic space operations, manipulator control, flight experiment concepts, manipulator coordination, issues in artificial intelligence systems, and research activities at the Johnson Space Center

Specification-Based Task Orchestration for Multi-Robot Aerial Teams

As humans begin working more frequently in environments with multi-agent systems, they are presented with challenges on how to control these systems in an intuitive manner. Current approaches tend to limit either the interaction ability of the user or limit the expressive capacity of instructions given to the robots. Applications that utilize temporal logics provide a human-readable syntax for systems that ensures formal guarantees for specification completion. By providing a modality for global task specification, we seek to reduce cognitive load and allow for high-level objectives to be communicated to a multi-agent system. In addition to this, we also seek to expand the capabilities of swarms to understand desired actions via interpretable commands retrieved from a human. In this thesis, we first present a method for specification-based control of a quadrotor. We utilize quadrotors as a highly agile and maneuverable application platform that has a wide variety of uses in complex problem domains. Leveraging specification-based control allows us to formulate a specification-based planning framework that will be utilized throughout the thesis. We then present methods for creating systems which allows us to provide task decomposition, allocation and planning for a team of quadrotors defined as task orchestration of multi-robot systems. Next, the task allocation portion of the task orchestration work is extended in the online case by considering cost agnostic sampling of trajectories from an online optimization problem. Then, we will introduce learning techniques where temporal logic specifications are learned and generated from a set of user given traces. Finally, we will conclude this thesis by presenting an extension to the Robotarium through hardware and software modifications that provides remote users access to control aerial swarms.Ph.D