Network Latency in Teleoperation of Connected and Autonomous Vehicles:A Review of Trends, Challenges, and Mitigation Strategies

With remarkable advancements in the development of connected and autonomous vehicles (CAVs), the integration of teleoperation has become crucial for improving safety and operational efficiency. However, teleoperation faces substantial challenges, with network latency being a critical factor influencing its performance. This survey paper explores the impact of network latency along with state-of-the-art mitigation/compensation approaches. It examines cascading effects on teleoperation communication links (i.e., uplink and downlink) and how delays in data transmission affect the real-time perception and decision-making of operators. By elucidating the challenges and available mitigation strategies, the paper offers valuable insights for researchers, engineers, and practitioners working towards the seamless integration of teleoperation in the evolving landscape of CAVs

Position / force control of systems subjected to communicaton delays and interruptions in bilateral teleoperation

Thesis (Master)--Izmir Institute of Technology, Mechanical Engineering, Izmir, 2012Includes bibliographical references (leaves: 65-68)Text in English; Abstract: Turkish and Englishix, 76 leavesTeleoperation technology allows to remotely operate robotic (slave) systems located in hazardous, risky and distant environments. The human operator sends commands through the controller (master) system to execute the tasks from a distance. The operator is provided with necessary (visual, audio or haptic) feedback to accomplish the mission remotely. In bilateral teleoperation, continuous feedback from the remote environment is generated. Thus, the operator can handle the task as if the operator is in the remote environment relying on the relevant feedback. Since teleoperation deals with systems controlled from a distance, time delays and package losses in transmission of information are present. These communication failures affect the human perception and system stability, and thus, the ability of operator to handle the task successfully. The objective of this thesis is to investigate and develop a control algorithm, which utilizes model mediated teleoperation integrating parallel position/force controllers, to compensate for the instability issues and excessive forcing applied to the environment arising from communication failures. Model mediation technique is extended for three-degrees-of-freedom teleoperation and a parallel position/force controller, impedance controller, is integrated in the control algorithm. The proposed control method is experimentally tested by using Matlab Simulink blocksets for real-time experimentation in which haptic desktop devices, Novint Falcon and Phantom Desktop are configured as master and slave subsystems of the bilateral teleoperation. The results of these tests indicate that the stability and passivity of proposed bilateral teleoperation systems are preserved during constant and variable time delays and data losses while the position and force tracking test results provide acceptable performance with bounded errors

Minimally Invasive Expeditionary Surgical Care Using Human-Inspired Robots

This technical report serves as an updated collection of subject matter experts on surgical care using human-inspired robotics for human exploration. It is a summary of the Blue Sky Meeting, organized by the Florida Institute for Human and Machine Cognition (IHMC), Pensacola, Florida, and held on October 2-3, 2018. It contains an executive summary, the final report, all of the presentation materials, and an updated reference list

Smart Navigation in Surgical Robotics

La cirugía mínimamente invasiva, y concretamente la cirugía laparoscópica, ha supuesto un gran cambio en la forma de realizar intervenciones quirúrgicas en el abdomen. Actualmente, la cirugía laparoscópica ha evolucionado hacia otras técnicas aún menos invasivas, como es la cirugía de un solo puerto, en inglés Single Port Access Surgery. Esta técnica consiste en realizar una única incisión, por la que son introducidos los instrumentos y la cámara laparoscópica a través de un único trocar multipuerto. La principal ventaja de esta técnica es una reducción de la estancia hospitalaria por parte del paciente, y los resultados estéticos, ya que el trocar se suele introducir por el ombligo, quedando la cicatriz oculta en él. Sin embargo, el hecho de que los instrumentos estén introducidos a través del mismo trocar hace la intervención más complicada para el cirujano, que necesita unas habilidades específicas para este tipo de intervenciones. Esta tesis trata el problema de la navegación de instrumentos quirúrgicos mediante plataformas robóticas teleoperadas en cirugía de un solo puerto. En concreto, se propone un método de navegación que dispone de un centro de rotación remoto virtual, el cuál coincide con el punto de inserción de los instrumentos (punto de fulcro). Para estimar este punto se han empleado las fuerzas ejercidas por el abdomen en los instrumentos quirúrgicos, las cuales han sido medidas por sensores de esfuerzos colocados en la base de los instrumentos. Debido a que estos instrumentos también interaccionan con tejido blando dentro del abdomen, lo cual distorsionaría la estimación del punto de inserción, es necesario un método que permita detectar esta circunstancia. Para solucionar esto, se ha empleado un detector de interacción con tejido basado en modelos ocultos de Markov el cuál se ha entrenado para detectar cuatro gestos genéricos. Por otro lado, en esta tesis se plantea el uso de guiado háptico para mejorar la experiencia del cirujano cuando utiliza plataformas robóticas teleoperadas. En concreto, se propone la técnica de aprendizaje por demostración (Learning from Demonstration) para generar fuerzas que puedan guiar al cirujano durante la resolución de tareas específicas. El método de navegación propuesto se ha implantado en la plataforma quirúrgica CISOBOT, desarrollada por la Universidad de Málaga. Los resultados experimentales obtenidos validan tanto el método de navegación propuesto, como el detector de interacción con tejido blando. Por otro lado, se ha realizado un estudio preliminar del sistema de guiado háptico. En concreto, se ha empleado una tarea genérica, la inserción de una clavija, para realizar los experimentos necesarios que permitan demostrar que el método propuesto es válido para resolver esta tarea y otras similares

A Novel Predictor Based Framework to Improve Mobility of High Speed Teleoperated Unmanned Ground Vehicles

Teleoperated Unmanned Ground Vehicles (UGVs) have been widely used in applications when driver safety, mission eciency or mission cost is a major concern. One major challenge with teleoperating a UGV is that communication delays can significantly affect the mobility performance of the vehicle and make teleoperated driving tasks very challenging especially at high speeds. In this dissertation, a predictor based framework with predictors in a new form and a blended architecture are developed to compensate effects of delays through signal prediction, thereby improving vehicle mobility performance. The novelty of the framework is that minimal information about the governing equations of the system is required to compensate delays and, thus, the prediction is robust to modeling errors. This dissertation first investigates a model-free solution and develops a predictor that does not require information about the vehicle dynamics or human operators' motion for prediction. Compared to the existing model-free methods, neither assumptions about the particular way the vehicle moves, nor knowledge about the noise characteristics that drive the existing predictive filters are needed. Its stability and performance are studied and a predictor design procedure is presented. Secondly, a blended architecture is developed to blend the outputs of the model-free predictor with those of a steering feedforward loop that relies on minimal information about vehicle lateral response. Better prediction accuracy is observed based on open-loop virtual testing with the blended architecture compared to using either the model-free predictors or the model-based feedforward loop alone. The mobility performance of teleoperated vehicles with delays and the predictor based framework are evaluated in this dissertation with human-in-the-loop experiments using both simulated and physical vehicles in teleoperation mode. Predictor based framework is shown to provide a statistically significant improvement in vehicle mobility and drivability in the experiments performed.PHDMechanical EngineeringUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttps://deepblue.lib.umich.edu/bitstream/2027.42/146026/1/zhengys_1.pd

Effects of Control Device and Task Complexity on Performance and Task Shedding During a Robotic Arm Task

The use of robotic arms across domains is increasing, but the relationship between control features and performance is not fully understood. The goal of this research was to investigate the difference in task performance when using two different control devices at high and low task complexities when participants can shed tasks to automation. In this experiment, 40 undergraduates (24 females) used two control devices, a Leap Motion controller and an Xbox controller, to teleoperate a robotic arm in a high or low complexity peg placement task. Simultaneously, participants were tasked with scanning images for tanks. During the experiment, participants had the option to task shed the peg task to imperfect automation. Analyses indicated a significant main effect of control device on task completion rate and time to first grasp the peg, with completion rate higher and time lower when using the Leap. However, participants made significantly more errors with the Leap Motion controller than with the Xbox controller. Participants in both conditions task shed similarly with both control devices and task shed at similar times. The 2 x 2 mixed ANOVAs somewhat supported the proposed hypotheses. The results of this study indicate that control device impacts performance on a robotic arm task. The Leap Motion controller supports increased task completion rate and quicker peg grasps in high and low task complexity when compared with the Xbox controller. This supports the extension of Control Order Theory into three-dimensional space and suggests that the Leap Motion controller can be implemented in some domains. However, the criticality and frequency of errors should be carefully considered

Human activity classification with miniature inertial sensors

Ankara : The Department of Electrical and Electronics Engineering and the Institute of Engineering and Sciences of Bilkent University, 2009.Thesis (Master's) -- Bilkent University, 2009.Includes bibliographical references leaves 79-92.This thesis provides a comparative study on activity recognition using miniature inertial sensors (gyroscopes and accelerometers) and magnetometers worn on the human body. The classification methods used and compared in this study are: a rule-based algorithm (RBA) or decision tree, least-squares method (LSM), k-nearest neighbor algorithm (k-NN), dynamic time warping (DTW- 1 and DTW-2), and support vector machines (SVM). In the first part of this study, eight different leg motions are classified using only two single-axis gyroscopes. In the second part, human activities are classified using five sensor units worn on different parts of the body. Each sensor unit comprises a tri-axial gyroscope, a tri-axial accelerometer and a tri-axial magnetometer. Different feature sets extracted from the raw sensor data and these are used in the classification process. A number of feature extraction and reduction techniques (principal component analysis) as well as different cross-validation techniques have been implemented and compared. A performance comparison of these classification methods is provided in terms of their correct differentiation rates, confusion matrices, pre-processing and training times and classification times. Among the classification techniques we have considered and implemented, SVM, in general, gives the highest correct differentiation rate, followed by k-NN. The classification time for RBA is the shortest, followed by SVM or LSM, k-NN or DTW-1, and DTW-2 methods. SVM requires the longest training time, whereas DTW-2 takes the longest amount of classification time. Although there is not a significant difference between the correct differentiation rates obtained by different crossvalidation techniques, repeated random sub-sampling uses the shortest amount of classification time, whereas leave-one-out requires the longest.Tunçel, OrkunM.S

Human Manual Control as an Information Processing Channel

Human-machine interaction (HMI) can be modeled as information flows through bidirectional communication channels, where the human receives sensory information from the machine and sends command information back to the machine. The interaction between human and machine can thus be characterized by the dynamics of information exchange measured in bits per second (b/s). The information-transmission rate (ITR) from human to machine is expected to depend on the complexity of the machine dynamics as well as human capabilities and limitations. We propose to investigate this dependency quantitatively in order to provide a measure of human performance. The HMI task considered in our investigation is a one-dimensional manual control task of an unstable system. A set of experiments are conducted where the human subjects maneuver a joystick in order to stabilize an inverted pendulum simulation. Two scenarios are analyzed: when time delay affects the feedback control system, and when the degree of instability of the task is increased. Using information- and control-theoretic approaches we identify the lower bound for the ITR of the controller, which is dependent on the dynamics of the control task. From time series analysis, we suggest a method for ITR estimation from human experiments. We believe that the difference between these two quantities allows for assessment of human performance and for the prediction of its limitations. Additionally, the association between the recorded ITR and the bandwidth of the controller will be discussed in order to create a more complete picture of human manual control abilities and limitations

Satellite Communications

This study is motivated by the need to give the reader a broad view of the developments, key concepts, and technologies related to information society evolution, with a focus on the wireless communications and geoinformation technologies and their role in the environment. Giving perspective, it aims at assisting people active in the industry, the public sector, and Earth science fields as well, by providing a base for their continued work and thinking