Collaborative Robotics Strategies for Handling Non-Repetitive Micro-Drilling Tasks Characterized by Low Structural Mechanical Impedance

Mechanical micro-drilling finds widespread use in diverse applications ranging from advanced manufacturing to medical surgery. This dissertation aims to develop techniques that allow programming of robots to perform effective micro-drilling tasks. Accomplishing this goal is faced with several challenges. Micro-drills suffer from frequent breakage caused from variations in drill process parameters. Micro-drilling tasks afford extremely low feed rates and almost zero tolerance for any feed rate variations. The accompanying robot programming task is made difficult as mathematical models that capture the micro-drilling process complexities and sensitive variations in micro-drill parameters are highly difficult to obtain. Therefore, an experimental approach is adopted to identify the feasible parameter space by carrying out a systematic characterization of the tool-specimen interaction that is crucial for understanding the robotic micro-drilling process. The diameter of the hole to be drilled on a material is a primary defining factor for micro-drilling. For the purposes of this dissertation, micro-drills are defined as having a diameter less than or equal to 1 mm. The Sawyer and KUKA collaborative robots that meet the sensitive speed requirements have been chosen for this study. A regression analysis revealed a relationship between feed rate and reaction forces involved in the micro-drilling process that matched the underlying mathematical model of the tool-specimen interactions. Subsequently, this dissertation addresses the problem of destabilization in robotic micro-drilling caused by the low impedance of the collaborative robot’s cantilever structure. A semi-robotic method that combines force-controlled adaptive drill feed rate and human-assisted impedance enhancement strategy is developed to address the destabilization problem. This approach is inspired by the capability of humans to stabilize unstable dynamics while performing contact-based tasks by using selective control of arm mechanical impedance. A human-robot collaborative kinesthetic drilling mode was also developed using the selective compliance capability of the KUKA robot. Experimental results show that the Sawyer and KUKA robots can use the developed strategies to drill micro-holes of diameters up to a minimum of 0.6 mm and 0.2 mm, respectively. Finally, experiments involving drilling in different materials reveal the potential application of the collaborative robotic micro-drilling approach in composite repairs, micro-channels, dental drilling, and bone drilling

Miniature mobile sensor platforms for condition monitoring of structures

In this paper, a wireless, multisensor inspection system for nondestructive evaluation (NDE) of materials is described. The sensor configuration enables two inspection modes-magnetic (flux leakage and eddy current) and noncontact ultrasound. Each is designed to function in a complementary manner, maximizing the potential for detection of both surface and internal defects. Particular emphasis is placed on the generic architecture of a novel, intelligent sensor platform, and its positioning on the structure under test. The sensor units are capable of wireless communication with a remote host computer, which controls manipulation and data interpretation. Results are presented in the form of automatic scans with different NDE sensors in a series of experiments on thin plate structures. To highlight the advantage of utilizing multiple inspection modalities, data fusion approaches are employed to combine data collected by complementary sensor systems. Fusion of data is shown to demonstrate the potential for improved inspection reliability

Predicting human motion intention for pHRI assistive control

This work addresses human intention identification during physical Human-Robot Interaction (pHRI) tasks to include this information in an assistive controller. To this purpose, human intention is defined as the desired trajectory that the human wants to follow over a finite rolling prediction horizon so that the robot can assist in pursuing it. This work investigates a Recurrent Neural Network (RNN), specifically, Long-Short Term Memory (LSTM) cascaded with a Fully Connected layer. In particular, we propose an iterative training procedure to adapt the model. Such an iterative procedure is powerful in reducing the prediction error. Still, it has the drawback that it is time-consuming and does not generalize to different users or different co-manipulated objects. To overcome this issue, Transfer Learning (TL) adapts the pre-trained model to new trajectories, users, and co-manipulated objects by freezing the LSTM layer and fine-tuning the last FC layer, which makes the procedure faster. Experiments show that the iterative procedure adapts the model and reduces prediction error. Experiments also show that TL adapts to different users and to the co-manipulation of a large object. Finally, to check the utility of adopting the proposed method, we compare the proposed controller enhanced by the intention prediction with the other two standard controllers of pHRI

A brief review of neural networks based learning and control and their applications for robots

As an imitation of the biological nervous systems, neural networks (NN), which are characterized with powerful learning ability, have been employed in a wide range of applications, such as control of complex nonlinear systems, optimization, system identification and patterns recognition etc. This article aims to bring a brief review of the state-of-art NN for the complex nonlinear systems. Recent progresses of NNs in both theoretical developments and practical applications are investigated and surveyed. Specifically, NN based robot learning and control applications were further reviewed, including NN based robot manipulator control, NN based human robot interaction and NN based behavior recognition and generation

Manipulation Planning for Forceful Human-Robot-Collaboration

This thesis addresses the problem of manipulation planning for forceful human-robot collaboration. Particularly, the focus is on the scenario where a human applies a sequence of changing external forces through forceful operations (e.g. cutting a circular piece off a board) on an object that is grasped by a cooperative robot. We present a range of planners that 1) enable the robot to stabilize and position the object under the human applied forces by exploiting supports from both the object-robot and object-environment contacts; 2) improve task efficiency by minimizing the need of configuration and grasp changes required by the changing external forces; 3) improve human comfort during the forceful interaction by optimizing the defined comfort criteria. We first focus on the instance of using only robotic grasps, where the robot is supposed to grasp/regrasp the object multiple times to keep it stable under the changing external forces. We introduce a planner that can generate an efficient manipulation plan by intelligently deciding when the robot should change its grasp on the object as the human applies the forces, and choosing subsequent grasps such that they minimize the number of regrasps required in the long-term. The planner searches for such an efficient plan by first finding a minimal sequence of grasp configurations that are able to keep the object stable under the changing forces, and then generating connecting trajectories to switch between the planned configurations, i.e. planning regrasps. We perform the search for such a grasp (configuration) sequence by sampling stable configurations for the external forces, building an operation graph using these stable configurations and then searching the operation graph to minimize the number of regrasps. We solve the problem of bimanual regrasp planning under the assumption of no support surface, enabling the robot to regrasp an object in the air by finding intermediate configurations at which both the bimanual and unimanual grasps can hold the object stable under gravity. We present a variety of experiments to show the performance of our planner, particularly in minimizing the number of regrasps for forceful manipulation tasks and planning stable regrasps. We then explore the problem of using both the object-environment contacts and object-robot contacts, which enlarges the set of stable configurations and thus boosts the robot’s capability in stabilizing the object under external forces. We present a planner that can intelligently exploit the environment’s and robot’s stabilization capabilities within a unified planning framework to search for a minimal number of stable contact configurations. A big computational bottleneck in this planner is due to the static stability analysis of a large number of candidate configurations. We introduce a containment relation between different contact configurations, to efficiently prune the stability checking process. We present a set of real-robot and simulated experiments illustrating the effectiveness of the proposed framework. We present a detailed analysis of the proposed containment relationship, particularly in improving the planning efficiency. We present a planning algorithm to further improve the cooperative robot behaviour concerning human comfort during the forceful human-robot interaction. Particularly, we are interested in empowering the robot with the capability of grasping and positioning the object not only to ensure the object stability against the human applied forces, but also to improve human experience and comfort during the interaction. We address human comfort as the muscular activation level required to apply a desired external force, together with the human spatial perception, i.e. the so-called peripersonal-space comfort during the interaction. We propose to maximize both comfort metrics to optimize the robot and object configuration such that the human can apply a forceful operation comfortably. We present a set of human-robot drilling and cutting experiments which verify the efficiency of the proposed metrics in improving the overall comfort and HRI experience, without compromising the force stability. In addition to the above planning work, we present a conic formulation to approximate the distribution of a forceful operation in the wrench space with a polyhedral cone, which enables the planner to efficiently assess the stability of a system configuration even in the presence of force uncertainties that are inherent in the human applied forceful operations. We also develop a graphical user interface, which human users can easily use to specify various forceful tasks, i.e. sequences of forceful operations on selected objects, in an interactive manner. The user interface ties in human task specification, on-demand manipulation planning and robot-assisted fabrication together. We present a set of human-robot experiments using the interface demonstrating the feasibility of our system. In short, in this thesis we present a series of planners for object manipulation under changing external forces. We show the object contacts with the robot and the environment enable the robot to manipulate an object under external forces, while making the most of the object contacts has the potential to eliminate redundant changes during manipulation, e.g. regrasp, and thus improve task efficiency and smoothness. We also show the necessity of optimizing human comfort in planning for forceful human-robot manipulation tasks. We believe the work presented here can be a key component in a human-robot collaboration framework

Recent innovations and technological advancements in Dentistry and medicine

The newer innovations play an important role in the earlier diagnosis of dental caries based on near-infrared transillumination imaging such as Diagnocam and Soprocare intraoral digital cameras, which aid in the diagnosis of dental caries and also differentiate old and newer dental plaques, respectively. Nanoparticles used as carriers for effective and targeted drug therapies in the treatment of rheumatoid arthritis of the temporomandibular joint (TMJ) and candidiasis among chronic denture wearers provided better-increased bioavailability than oral formulations. Artificial intelligence (AI) uses machines to simulate intelligent human behavior. Augmented reality has been presented as a cognitive extension of AI in health care, emphasizing the auxiliary and supplemental roles of dental and medical practitioners. The article enlightens readers with newer diagnostic methods in dentistry and medicine and innovations such as Surface Topography-Assisted Robotic Superstructures (STARS), Prostate-Specific Membrane Antigen Ligand Positron Emission Tomography/Computed Tomography (PSMA-PETCT), toothbrushes with augmented reality, new hydrogel-based electrodes for Brain-Computer Interface (BCI) applications, and robotics in dentistry

The NASA SBIR product catalog

The purpose of this catalog is to assist small business firms in making the community aware of products emerging from their efforts in the Small Business Innovation Research (SBIR) program. It contains descriptions of some products that have advanced into Phase 3 and others that are identified as prospective products. Both lists of products in this catalog are based on information supplied by NASA SBIR contractors in responding to an invitation to be represented in this document. Generally, all products suggested by the small firms were included in order to meet the goals of information exchange for SBIR results. Of the 444 SBIR contractors NASA queried, 137 provided information on 219 products. The catalog presents the product information in the technology areas listed in the table of contents. Within each area, the products are listed in alphabetical order by product name and are given identifying numbers. Also included is an alphabetical listing of the companies that have products described. This listing cross-references the product list and provides information on the business activity of each firm. In addition, there are three indexes: one a list of firms by states, one that lists the products according to NASA Centers that managed the SBIR projects, and one that lists the products by the relevant Technical Topics utilized in NASA's annual program solicitation under which each SBIR project was selected

Combinatorial ink-jet printing for ceramic discovery

PhDAn aspirating and dispensing printer established inside a robot gantry equipped with furnace and measurement table is used to prepare thick-film combinatorial libraries. Implementation of series of screening tests for ceramic inks that address stability against sedimentation, evaporation and particle segregation during drying, has provided a series of calibration inks can be used for calibration of this printer. The instrument can assemble ceramic mixtures with compositional accuracy of 1-3 wt %. By changing the amount of dispersant used in the inks or by printing onto a porous substrate, the geometry of residues from dried ceramic ink droplets can be modified to facilitate property measurements and uniform composition, as planned, can be achieved. The same material prepared in three ways, in the form of dried ink, ink-jet printed as for a combinatorial sample and by conventional compaction gave similar dielectric measurements. A combinatorial system has been developed so that combinatorial libraries can be printed, fired and screened automatically. A ternary A1203-TiO2-ZrO2 system was first studied using the developed combinatorial method. The particle segregation during drying of multi-component ceramic ink drops is not due to preferential sedimentation unless dispersant addition is restricted. The segregation is due to the partitioning of particles between the growing peripheral 'foot' that develops during drying and the diminishing liquid pool which contains vigorous recirculation flows. Better dispersed particles remain in the pool and hence are found in excess on the upper surface of residues. Less well dispersed particles join the 'foot' earlier in the drying process. The contact angle and height of droplets containing large amounts of dispersant, steadily reduced during drying until a minimum value was reached; the contact diameter being almost unchanged during drying. These droplet residues retained a dome shape. Droplets of suspensions containing small additions of dispersant terminated in a 'doughnut' shaped residue

Robot manipulator skill learning and generalising through teleoperation

Robot manipulators have been widely used for simple repetitive, and accurate tasks in industrial plants, such as pick and place, assembly and welding etc., but it is still hard to deploy in human-centred environments for dexterous manipulation tasks, such as medical examination and robot-assisted healthcare. These tasks are not only related to motion planning and control but also to the compliant interaction behaviour of robots, e.g. motion control, force regulation and impedance adaptation simultaneously under dynamic and unknown environments. Recently, with the development of collaborative robotics (cobots) and machine learning, robot skill learning and generalising have attained increasing attention from robotics, machine learning and neuroscience communities. Nevertheless, learning complex and compliant manipulation skills, such as manipulating deformable objects, scanning the human body and folding clothes, is still challenging for robots. On the other hand, teleoperation, also namely remote operation or telerobotics, has been an old research area since 1950, and there have been a number of applications such as space exploration, telemedicine, marine vehicles and emergency response etc. One of its advantages is to combine the precise control of robots with human intelligence to perform dexterous and safety-critical tasks from a distance. In addition, telepresence allows remote operators could feel the actual interaction between the robot and the environment, including the vision, sound and haptic feedback etc. Especially under the development of various augmented reality (AR), virtual reality (VR) and wearable devices, intuitive and immersive teleoperation have received increasing attention from robotics and computer science communities. Thus, various human-robot collaboration (HRC) interfaces based on the above technologies were developed to integrate robot control and telemanipulation by human operators for robot skills learning from human beings. In this context, robot skill learning could benefit teleoperation by automating repetitive and tedious tasks, and teleoperation demonstration and interaction by human teachers also allow the robot to learn progressively and interactively. Therefore, in this dissertation, we study human-robot skill transfer and generalising through intuitive teleoperation interfaces for contact-rich manipulation tasks, including medical examination, manipulating deformable objects, grasping soft objects and composite layup in manufacturing. The introduction, motivation and objectives of this thesis are introduced in Chapter 1. In Chapter 2, a literature review on manipulation skills acquisition through teleoperation is carried out, and the motivation and objectives of this thesis are discussed subsequently. Overall, the main contents of this thesis have three parts: Part 1 (Chapter 3) introduces the development and controller design of teleoperation systems with multimodal feedback, which is the foundation of this project for robot learning from human demonstration and interaction. In Part 2 (Chapters 4, 5, 6 and 7), we studied primitive skill library theory, behaviour tree-based modular method, and perception-enhanced method to improve the generalisation capability of learning from the human demonstrations. And several applications were employed to evaluate the effectiveness of these methods.In Part 3 (Chapter 8), we studied the deep multimodal neural networks to encode the manipulation skill, especially the multimodal perception information. This part conducted physical experiments on robot-assisted ultrasound scanning applications.Chapter 9 summarises the contributions and potential directions of this thesis. Keywords: Learning from demonstration; Teleoperation; Multimodal interface; Human-in-the-loop; Compliant control; Human-robot interaction; Robot-assisted sonography