Human-Centric Machine Vision

Recently, the algorithms for the processing of the visual information have greatly evolved, providing efficient and effective solutions to cope with the variability and the complexity of real-world environments. These achievements yield to the development of Machine Vision systems that overcome the typical industrial applications, where the environments are controlled and the tasks are very specific, towards the use of innovative solutions to face with everyday needs of people. The Human-Centric Machine Vision can help to solve the problems raised by the needs of our society, e.g. security and safety, health care, medical imaging, and human machine interface. In such applications it is necessary to handle changing, unpredictable and complex situations, and to take care of the presence of humans

Human skill capturing and modelling using wearable devices

Industrial robots are delivering more and more manipulation services in manufacturing. However, when the task is complex, it is difficult to programme a robot to fulfil all the requirements because even a relatively simple task such as a peg-in-hole insertion contains many uncertainties, e.g. clearance, initial grasping position and insertion path. Humans, on the other hand, can deal with these variations using their vision and haptic feedback. Although humans can adapt to uncertainties easily, most of the time, the skilled based performances that relate to their tacit knowledge cannot be easily articulated. Even though the automation solution may not fully imitate human motion since some of them are not necessary, it would be useful if the skill based performance from a human could be firstly interpreted and modelled, which will then allow it to be transferred to the robot. This thesis aims to reduce robot programming efforts significantly by developing a methodology to capture, model and transfer the manual manufacturing skills from a human demonstrator to the robot. Recently, Learning from Demonstration (LfD) is gaining interest as a framework to transfer skills from human teacher to robot using probability encoding approaches to model observations and state transition uncertainties. In close or actual contact manipulation tasks, it is difficult to reliabley record the state-action examples without interfering with the human senses and activities. Therefore, wearable sensors are investigated as a promising device to record the state-action examples without restricting the human experts during the skilled execution of their tasks. Firstly to track human motions accurately and reliably in a defined 3-dimensional workspace, a hybrid system of Vicon and IMUs is proposed to compensate for the known limitations of the individual system. The data fusion method was able to overcome occlusion and frame flipping problems in the two camera Vicon setup and the drifting problem associated with the IMUs. The results indicated that occlusion and frame flipping problems associated with Vicon can be mitigated by using the IMU measurements. Furthermore, the proposed method improves the Mean Square Error (MSE) tracking accuracy range from 0.8˚ to 6.4˚ compared with the IMU only method. Secondly, to record haptic feedback from a teacher without physically obstructing their interactions with the workpiece, wearable surface electromyography (sEMG) armbands were used as an indirect method to indicate contact feedback during manual manipulations. A muscle-force model using a Time Delayed Neural Network (TDNN) was built to map the sEMG signals to the known contact force. The results indicated that the model was capable of estimating the force from the sEMG armbands in the applications of interest, namely in peg-in-hole and beater winding tasks, with MSE of 2.75N and 0.18N respectively. Finally, given the force estimation and the motion trajectories, a Hidden Markov Model (HMM) based approach was utilised as a state recognition method to encode and generalise the spatial and temporal information of the skilled executions. This method would allow a more representative control policy to be derived. A modified Gaussian Mixture Regression (GMR) method was then applied to enable motions reproduction by using the learned state-action policy. To simplify the validation procedure, instead of using the robot, additional demonstrations from the teacher were used to verify the reproduction performance of the policy, by assuming human teacher and robot learner are physical identical systems. The results confirmed the generalisation capability of the HMM model across a number of demonstrations from different subjects; and the reproduced motions from GMR were acceptable in these additional tests. The proposed methodology provides a framework for producing a state-action model from skilled demonstrations that can be translated into robot kinematics and joint states for the robot to execute. The implication to industry is reduced efforts and time in programming the robots for applications where human skilled performances are required to cope robustly with various uncertainties during tasks execution

Applied Mathematics to Mechanisms and Machines

This book brings together all 16 articles published in the Special Issue "Applied Mathematics to Mechanisms and Machines" of the MDPI Mathematics journal, in the section “Engineering Mathematics”. The subject matter covered by these works is varied, but they all have mechanisms as the object of study and mathematics as the basis of the methodology used. In fact, the synthesis, design and optimization of mechanisms, robotics, automotives, maintenance 4.0, machine vibrations, control, biomechanics and medical devices are among the topics covered in this book. This volume may be of interest to all who work in the field of mechanism and machine science and we hope that it will contribute to the development of both mechanical engineering and applied mathematics

A multi-step learning approach for in-process monitoring of depth-of-cuts in robotic countersinking operations

Robotic machining is a relatively new and promising technology that aims to substitute the conventional approach of Computer Numeric Control machine tools. Due to the low positional accuracy and variable stiffness of the industrial robots, the machining operations performed by robotic systems are subject to variations in the quality of the finished product. The main focus of this work is to provide a means of improving the performance of a robotic machining process by the use of in-process monitoring of key process variables that directly influence the quality of the machined part. To this end, an intelligent monitoring system is designed, which uses sensor signals collected during machining to predict the amount of errors that the robotic system introduces into the manufacturing process in terms of imperfections of the finished product. A multi-step learning procedure that allows training of process models to take place during normal operation of the process is proposed. Moreover, applying an iterative probabilistic approach, these models are able to estimate, given the current training dataset, whether the prediction is likely to be correct and further training data is requested if necessary. The proposed monitoring system was tested in a robotic countersinking experiment for the in-process prediction of the countersink depth-of-cut and the results showed good ability of the models to provide accurate and reliable predictions