Autonomy Infused Teleoperation with Application to BCI Manipulation

Robot teleoperation systems face a common set of challenges including latency, low-dimensional user commands, and asymmetric control inputs. User control with Brain-Computer Interfaces (BCIs) exacerbates these problems through especially noisy and erratic low-dimensional motion commands due to the difficulty in decoding neural activity. We introduce a general framework to address these challenges through a combination of computer vision, user intent inference, and arbitration between the human input and autonomous control schemes. Adjustable levels of assistance allow the system to balance the operator's capabilities and feelings of comfort and control while compensating for a task's difficulty. We present experimental results demonstrating significant performance improvement using the shared-control assistance framework on adapted rehabilitation benchmarks with two subjects implanted with intracortical brain-computer interfaces controlling a seven degree-of-freedom robotic manipulator as a prosthetic. Our results further indicate that shared assistance mitigates perceived user difficulty and even enables successful performance on previously infeasible tasks. We showcase the extensibility of our architecture with applications to quality-of-life tasks such as opening a door, pouring liquids from containers, and manipulation with novel objects in densely cluttered environments

Hybrid intelligent machine systems : design, modeling and control

To further improve performances of machine systems, mechatronics offers some opportunities. Traditionally, mechatronics deals with how to integrate mechanics and electronics without a systematic approach. This thesis generalizes the concept of mechatronics into a new concept called hybrid intelligent machine system. A hybrid intelligent machine system is a system where two or more elements combine to play at least one of the roles such as sensor, actuator, or control mechanism, and contribute to the system behaviour. The common feature with the hybrid intelligent machine system is thus the presence of two or more entities responsible for the system behaviour with each having its different strength complementary to the others. The hybrid intelligent machine system is further viewed from the system’s structure, behaviour, function, and principle, which has led to the distinction of (1) the hybrid actuation system, (2) the hybrid motion system (mechanism), and (3) the hybrid control system. This thesis describes a comprehensive study on three hybrid intelligent machine systems. In the case of the hybrid actuation system, the study has developed a control method for the “true” hybrid actuation configuration in which the constant velocity motor is not “mimicked” by the servomotor which is treated in literature. In the case of the hybrid motion system, the study has resulted in a novel mechanism structure based on the compliant mechanism which allows the micro- and macro-motions to be integrated within a common framework. It should be noted that the existing designs in literature all take a serial structure for micro- and macro-motions. In the case of hybrid control system, a novel family of control laws is developed, which is primarily based on the iterative learning of the previous driving torque (as a feedforward part) and various feedback control laws. This new family of control laws is rooted in the computer-torque-control (CTC) law with an off-line learned torque in replacement of an analytically formulated torque in the forward part of the CTC law. This thesis also presents the verification of these novel developments by both simulation and experiments. Simulation studies are presented for the hybrid actuation system and the hybrid motion system while experimental studies are carried out for the hybrid control system

Biomimetic Manipulator Control Design for Bimanual Tasks in the Natural Environment

As robots become more prolific in the human environment, it is important that safe operational procedures are introduced at the same time; typical robot control methods are often very stiff to maintain good positional tracking, but this makes contact (purposeful or accidental) with the robot dangerous. In addition, if robots are to work cooperatively with humans, natural interaction between agents will make tasks easier to perform with less effort and learning time. Stability of the robot is particularly important in this situation, especially as outside forces are likely to affect the manipulator when in a close working environment; for example, a user leaning on the arm, or task-related disturbance at the end-effector. Recent research has discovered the mechanisms of how humans adapt the applied force and impedance during tasks. Studies have been performed to apply this adaptation to robots, with promising results showing an improvement in tracking and effort reduction over other adaptive methods. The basic algorithm is straightforward to implement, and allows the robot to be compliant most of the time and only stiff when required by the task. This allows the robot to work in an environment close to humans, but also suggests that it could create a natural work interaction with a human. In addition, no force sensor is needed, which means the algorithm can be implemented on almost any robot. This work develops a stable control method for bimanual robot tasks, which could also be applied to robot-human interactive tasks. A dynamic model of the Baxter robot is created and verified, which is then used for controller simulations. The biomimetic control algorithm forms the basis of the controller, which is developed into a hybrid control system to improve both task-space and joint-space control when the manipulator is disturbed in the natural environment. Fuzzy systems are implemented to remove the need for repetitive and time consuming parameter tuning, and also allows the controller to actively improve performance during the task. Experimental simulations are performed, and demonstrate how the hybrid task/joint-space controller performs better than either of the component parts under the same conditions. The fuzzy tuning method is then applied to the hybrid controller, which is shown to slightly improve performance as well as automating the gain tuning process. In summary, a novel biomimetic hybrid controller is presented, with a fuzzy mechanism to avoid the gain tuning process, finalised with a demonstration of task-suitability in a bimanual-type situation.EPSR

A New Index for Detecting and Avoiding Type II Singularities for the Control of Non-Redundant Parallel Robots

[ES] Los robots paralelos (PR por sus siglas en inglés) son mecanismos donde el efector final está unido a la base, mediante al menos dos cadenas cinemáticas abiertas. Los PRs ofrecen una gran capacidad de carga y alta precisión, lo que los hace adecuados para diversas aplicaciones, entre ellas la interacción persona-robot. Sin embargo, en las proximidades de una singularidad Tipo II (singularidad dentro del espacio de trabajo), un PR pierde el control sobre los movimientos del efector final. La pérdida de control representa un riesgo importante para los usuarios, especialmente en rehabilitación robótica. En las últimas décadas, los PR se han popularizado en la rehabilitación de miembros inferiores debido al aumento del número de personas que viven con limitaciones físicas. Así, esta tesis trata sobre la detección y evitación de singularidades de Tipo II para asegurar total control de un PR no redundante para la rehabilitación y diagnóstico de rodilla, denominado 3UPS+RPU. En la literatura, existen varios índices para detectar y medir la cercanía a una singularidad basados en métodos analíticos y geométricos. Sin embargo, algunos de estos índices carecen de significado físico y son incapaces de identificar los actuadores responsables de la pérdida de control. Esta tesis aporta dos novedosos índices para detectar y medir la proximidad a una singularidad de Tipo II, capaces de identificar el par de actuadores responsables de la singularidad. Los dos índices son los ángulos entre los componentes lineal (T_i,j) y angular (O_i,j) de dos Twist Screw de Salida (OTS por sus siglas en inglés) normalizados i,j. Una singularidad Tipo II es detectada cuando T_i,j = O_i,j = 0 y su proximidad se mide mediante los mínimos ángulos T_i,j (minT) y O_i,j (minO) para los casos plano y espacial, respectivamente. La eficacia de los índices T_i,j y O_i,j se evalúa de forma teórica y experimental en un robot 3UPS+RPU y un mecanismo de cinco barras. Además, se propone un procedimiento experimental para el adecuado establecimiento del límite de cercanía a una singularidad de Tipo II mediante la aproximación progresiva del PR a una singularidad y la medición de la última posición controlable. Posteriormente, se desarrollan dos nuevos algoritmos deterministas para liberar y evitar una singularidad de Tipo II basados en minT y minO para PR no redundantes. minT y minO se utilizan para identificar los dos actuadores a mover para liberar o evitar el PR de una singularidad. Ambos algoritmos requieren una medición precisa de la pose alcanzada por el efector final. El algoritmo para liberar un PR de una configuración singular se aplica con éxito en un controlador híbrido basado en visión artificial para el PR 3UPS+RPU. El controlador utiliza un sistema de fotogrametría para medir la pose del robot debido a la degeneración del modelo cinemático en las proximidades de una singularidad. El algoritmo de evasión de singularidades Tipo II se aplica a la planificación offline y online de trayectorias no singulares para un mecanismo de cinco barras y el PR 3UPS+RPU. Estas aplicaciones verifican el bajo coste computacional y la mínima desviación introducida en la trayectoria original por los nuevos algoritmos. La implementación directa de un controlador de fuerza/posición en el PR 3UPS+RPU es insegura porque el paciente podría llevar involuntariamente al PR a una singularidad. Por lo tanto, esta tesis concluye presentando un novedoso controlador de fuerza/posición complementado con el algoritmo de evasión de singularidades de Tipo II. El nuevo controlador se evalúa durante rehabilitación activa de una pierna de maniquí y una pierna humana no lesionada. Los resultados muestran que el nuevo controlador combinado mantiene el PR 3UPS+RPU lejos de configuraciones singulares con una desviación mínima de la trayectoria original. Por lo tanto, esta tesis habilita el 3UPS+RPU PR para la rehabilitación segura de miembros inferiores lesionados.[CAT] Els robots paral·lels (PR per les seues sigles en anglés) són mecanismes on l'efector final està unit a la base, mitjançant almenys dues cadenes cinemàtiques obertes. Els PRs ofereixen una gran capacitat de càrrega i alta precisió, la qual cosa els fa adequats per a diverses aplicacions, entre elles la interacció persona-robot. No obstant això, en les proximitats d'una singularitat Tipus II (singularitat dins de l'espai de treball), un PR perd el control sobre els moviments de l'efector final. La pèrdua de control representa un risc important per als usuaris, especialment en rehabilitació robòtica. En les últimes dècades, els PR s'han popularitzat en la rehabilitació de membres inferiors a causa de l'augment del nombre de persones que viuen amb limitacions físiques. Així, aquesta tesi tracta sobre la detecció i evació de singularitats de Tipus II per a assegurar total control d'un PR no redundant per a la rehabilitació i diagnòstic de genoll, denominat 3UPS+RPU. En la literatura, existeixen diversos índexs per a detectar i mesurar la proximitat a una singularitat basats en mètodes analítics i geomètrics. No obstant això, alguns d'aquests índexs manquen de significat físic i són incapaços d'identificar els actuadors responsables de la pèrdua de control. Aquesta tesi aporta dos nous índexs per a detectar i mesurar la proximitat a una singularitat de Tipus II, capaços d'identificar el parell d'actuadors responsables de la singularitat. Els dos índexs són els angles entre els components lineal (T_i,j) i angular (O_i,j) de dues Twist Screw d'Eixida (OTS per les seues sigles en engonals) normalitzats i,j. Una singularitat Tipus II és detectada quan T_i,j = O_i,j = 0 i la seua proximitat es mesura mitjançant els minimos angles T_i,j (minT) i O_i,j (minO) per als casos pla i espacial, respectivament. L'eficàcia dels índexs T_i,j i O_i,j es evalua de manera teòrica i experimental en un robot 3UPS+RPU i un mecanisme de cinc barres. A més, es proposa un procediment experimental per a l'adequat establiment del límit de proximitat a una singularitat de Tipus II mitjançant l'aproximació progressiva del PR a una singularitat i el mesurament de l'última posició controlable. Posteriorment, es desenvolupen dos nous algorismes deterministes per a alliberar i evadir una singularitat de Tipus II basats en minT i minO per a PR no redundants. minT i minO s'utilitzen per a identificar els dos actuadors a moure per a alliberar o evadir el PR d'una singularitat. Aquests algorismes requereixen un mesurament precís de la posa aconseguida per l'efector final. L'algorisme per a alliberar un PR d'una configuració singular s'aplica amb èxit en un controlador híbrid basat en visió artificial per al PR 3UPS+RPU. El controlador utilitza un sistema de fotogrametria per a mesurar la posa del robot a causa de la degeneració del model cinemàtic en les proximitats d'una singularitat. L'algorisme d'evació de singularitats Tipus II s'aplica a la planificació offline i en línia de trajectòries no singulars per a un mecanisme de cinc barres i el PR 3UPS+RPU. Aquestes aplicacions verifiquen el baix cost computacional i la mínima desviació introduïda en la trajectòria original pels nous algorismes. La implementació directa d'un controlador de força/posició en el PR 3UPS+RPU és insegura perquè el pacient podria portar involuntàriament al PR a una singularitat. Per tant, aquesta tesi conclou presentant un nou controlador de força/posició complementat amb l'algorisme d'evació de singularitats de Tipus II. El nou controlador s'avalua durant la rehabilitació activa d'una cama de maniquí i una cama humana no lesionada. Els resultats mostren que el nou controlador combinat manté el PR 3UPS+RPU lluny de configuracions singulars amb una desviació mínima de la trajectòria original. Per tant, aquesta tesi habilita el 3UPS+RPU PR per a la rehabilitació segura dels membres inferiors lesionats.[EN] Parallel Robots (PR)s are mechanisms where the end-effector is linked to the base by at least two open kinematics chains. The PRs offer a high payload and high accuracy, making them suitable for various applications, including human robot interaction. However, in proximity to a Type II singularity (singularity within the workspace), a PR loses control over the movements of the end-effector. The loss of control represents a major risk for users, especially in robotic rehabilitation. In the last decades, PRs have become popular in lower limb rehabilitation because of the increment in the number of people living with physical limitations. Thus, this thesis is about the detection and avoidance of Type II singularities to ensure complete control of a non-redundant PR for knee rehabilitation and diagnosis named 3UPS+RPU. In the literature, several indices exist to detect and measure the closeness to a singular configuration based on analytical and geometrical methods. However, some of these indices have no physical meaning, and they are unable to identify the actuators responsible for the loss of control. This thesis contributes two novel indices to detect and measure the proximity to a Type II singularity capable of identifying the pair of actuators responsible for the singularity. The two indices are the angles between the linear (T_i,j) and the angular (O_i,j) components of two i,j normalised Output Twist Screws (OTSs). A Type II singularity is detected when the angles T_i,j = O_i,j = 0 and its closeness is measured by the minimum T_i,j (minT) and minimum O_i,j (minO) for planar and spatial cases, respectively. The effectiveness of the indices T_i,j and O_i,j is evaluated from a theoretical and experimental perspective in a 3UPS+RPU and a five bars mechanism. Moreover, an experimental procedure is proposed for setting a proper limit of closeness to a Type II singularity by the progressive approach of the PR to singular configuration and measuring the last controllable pose. Subsequently, two novel deterministic algorithms for releasing and avoiding Type II singularities based on minT and minO are developed for non-redundant PRs. The minT and minO are used to identify the two actuators to move for release or prevent the PR from the singularity. Both algorithms require an accurate measuring of the pose reached by the end-effector. The algorithm to release a PR from a singular configuration is successfully applied in a vision-based hybrid controller for the 3UPS+RPU PR. The controller uses a photogrammetry system to measure the pose of the robot due to the degeneration of the kinematic model in the vicinity of a singularity. The Type II singularity avoidance algorithm is applied to offline and online free-singularity trajectory planning for a five-bar mechanism and the 3UPS+RPU PR. These applications verify the low computation cost and the minimum deviation introduced in the original trajectory for both novel algorithms. The direct implementation of a force/position controller in the 3UPS+RPU PR is unsafe because the patient could unintentionally drive the PR to a Type II singularity. Therefore, this thesis concludes by presenting a novel force/position controller complemented with the Type II singularity avoidance algorithm. The complemented controller is evaluated during patient-active exercises in a mannequin leg and an uninjured human limb. The results show that the novel combined controller keeps the 3UPS+RPU PR far from singular configurations with a minimum deviation on the original trajectory. Hence, this thesis enables the 3UPS+RPU PR for the safe rehabilitation of injured lower limbs.Pulloquinga Zapata, JL. (2023). A New Index for Detecting and Avoiding Type II Singularities for the Control of Non-Redundant Parallel Robots [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/19427

A Framework of Hybrid Force/Motion Skills Learning for Robots

Human factors and human-centred design philosophy are highly desired in today’s robotics applications such as human-robot interaction (HRI). Several studies showed that endowing robots of human-like interaction skills can not only make them more likeable but also improve their performance. In particular, skill transfer by imitation learning can increase usability and acceptability of robots by the users without computer programming skills. In fact, besides positional information, muscle stiffness of the human arm, contact force with the environment also play important roles in understanding and generating human-like manipulation behaviours for robots, e.g., in physical HRI and tele-operation. To this end, we present a novel robot learning framework based on Dynamic Movement Primitives (DMPs), taking into consideration both the positional and the contact force profiles for human-robot skills transferring. Distinguished from the conventional method involving only the motion information, the proposed framework combines two sets of DMPs, which are built to model the motion trajectory and the force variation of the robot manipulator, respectively. Thus, a hybrid force/motion control approach is taken to ensure the accurate tracking and reproduction of the desired positional and force motor skills. Meanwhile, in order to simplify the control system, a momentum-based force observer is applied to estimate the contact force instead of employing force sensors. To deploy the learned motion-force robot manipulation skills to a broader variety of tasks, the generalization of these DMP models in actual situations is also considered. Comparative experiments have been conducted using a Baxter Robot to verify the effectiveness of the proposed learning framework on real-world scenarios like cleaning a table

Human-robot cooperation for robust surface treatment using non-conventional sliding mode control

[EN] This work presents a human-robot closely collaborative solution to cooperatively perform surface treatment tasks such as polishing, grinding, deburring, etc. The method considers two force sensors attached to the manipulator end-effector and tool: one sensor is used to properly accomplish the surface treatment task, while the second one is used by the operator to guide the robot tool. The proposed scheme is based on task priority and adaptive non-conventional sliding mode control. The applicability of the proposed approach is substantiated by experimental results using a redundant 7R manipulator: the Sawyer cobot.This work was supported in part by the Spanish Government under the project DPI2017-87656-C2-1-R and the Generalitat Valenciana under Grants VALi + d APOSTD/2016/044 and APOSTD/2017/055.Solanes Galbis, JE.; Gracia Calandin, LI.; Muñoz-Benavent, P.; Valls Miro, J.; Girbés, V.; Tornero Montserrat, J. (2018). Human-robot cooperation for robust surface treatment using non-conventional sliding mode control. ISA Transactions. 80(1):528-541. https://doi.org/10.1016/j.isatra.2018.05.013S52854180

Planning With Adaptive Dimensionality

Modern systems, such as robots or virtual agents, need to be able to plan their actions in increasingly more complex and larger state-spaces, incorporating many degrees of freedom. However, these high-dimensional planning problems often have low-dimensional representations that describe the problem well throughout most of the state-space. For example, planning for manipulation can be represented by planning a trajectory for the end-effector combined with an inverse kinematics solver through obstacle-free areas of the environment, while planning in the full joint space of the arm is only necessary in cluttered areas. Based on this observation, we have developed the framework for Planning with Adaptive Dimensionality, which makes effective use of state abstraction and dimensionality reduction in order to reduce the size and complexity of the state-space. It iteratively constructs and searches a hybrid state-space consisting of both abstract and non-abstract states. Initially the state-space consists only of abstract states, and regions of non-abstract states are selectively introduced into the state-space in order to maintain the feasibility of the resulting path and the strong theoretical guarantees of the algorithm---completeness and bounds on solution cost sub-optimality. The framework is able to make use of hierarchies of abstractions, as different abstractions can be more effective than others in different parts of the state-space. We have extended the framework to be able to utilize anytime and incremental graph search algorithms. Moreover, we have developed a novel general incremental graph search algorithm---tree-restoring weighted A*, which is able to minimize redundant computation between iterations while efficiently handling changes in the search graph. We have applied our framework to several different domains---navigation for unmanned aerial and ground vehicles, multi-robot collaborative navigation, manipulation and mobile manipulation, and navigation for humanoid robots