Embedded Artificial Intelligent (AI) To Navigate Cart Follower

The concern of the societies in creating a quality life for everyone without laying aside of the right of disable person leads to research on designing and fabricating autonomous robot. Wheelchair user usually faces the problem of carrying luggage along during travel as they need both of their hands to navigate their wheelchair. One of the solution for the problem is to create an Artificial Intelligent (AI) cart follower. Therefore, this research is to create an AI system for the AI cart follower with a visual based sensor. The visual based sensor gathered the information of the width, height, angle, x and y coordination of the colour pattern board which situated behind the wheelchair and translate this information into relative position information which enable the cart to follow the wheelchair. This translation can be done in neural network. However, the data needs to be collected in such a way that the output distance is manipulated between 20cm to 69cm and the output angle is manipulated between -30 to 30 with its restriction for each case. The test MSE value is used to evaluate the performance of NN and validation MSE value is used to prevent overfitting. The weights and biases generated through the training process is depended on the training algorithm, initial weights and biases for training and the dataset used in the process. The training algorithm may also vary with different sets of parameters, number of neurons and activation function. The set of parameters used in traingd are lr, max_fail, min_grad, goal, time, and epochs. The final weights and biases generated with the minimum MSE performance after several run is used to train NN in the FPGA together with the structure of NN obtained in Simulink. The implementation of neural network on the FPGA can be done through software or hardware configuration. However, the floating-point operation circuit needs to be built to ensure the NN on FPGA is functioning

People tracking and following with a smart wheelchair using an omnidirectional camera and a RGB-D Camera

The project implements a new service that enables a smart wheelchair user and another person to have a normal talk while freely strolling around the environment, without the need of any interaction towards the wheelchair, called Jiaolong

Active visual tracking in multi-agent scenarios

PhD thesisCamera-equipped robots (agents) can autonomously follow people to provide continuous assistance in wide areas, e.g. museums and airports. Each agent serves one person (target) at a time and aims to maintain its target centred on the camera’s image plane with a certain size (active visual tracking) without colliding with other agents and targets in its proximity. It is essential that each agent accurately estimates the state of itself and that of nearby targets and agents over time (i.e. tracking) to perform collision-free active visual tracking. Agents can track themselves with either on-board sensors (e.g. cameras or inertial sensors) or external tracking systems (e.g. multi-camera systems). However, on-board sensing alone is not sufficient for tracking nearby targets due to occlusions in crowded scenes, where an external multi-camera system can help. To address scalability of wide-area applications and accurate tracking, this thesis proposes a novel collaborative framework where agents track nearby targets jointly with wireless ceiling-mounted static cameras in a distributed manner. Distributed tracking enables each agent to achieve agreed state estimates of targets via iteratively communicating with neighbouring static cameras. However, such iterative neighbourhood communication may cause poor communication quality (i.e. packet loss/error) due to limited bandwidth, which worsens tracking accuracy. This thesis proposes the formation of coalitions among static cameras prior to distributed tracking based on a marginal information utility that accounts for both the communication quality and the local tracking confidence. Agents move on demand when hearing requests from nearby static cameras. Each agent independently selects its target with limited scene knowledge and computes its robotic control for collision-free active visual tracking. Collision avoidance among robots and targets can be achieved by the Optimal Reciprocal Collision Avoidance (ORCA) method. To further address view maintenance during collision avoidance manoeuvres, this thesis proposes an ORCA-based method with adaptive responsibility sharing and heading-aware robotic control mapping. Experimental results show that the proposed methods achieve higher tracking accuracy and better view maintenance compared with the state-of-the-art methods.Queen Mary University of London and Chinese Scholarship Council

Planification de trajectoire pour la manipulation d'objets et l'interaction homme-robot

Le contexte de la robotique de service est caractérisé par la présence de l'homme dans l'espace de travail du robot. Les mouvements de ces robots ne doivent perturber ni la sécurité de l'homme ni son confort. D'un point de vu planification de mouvement, le planificateur doit d'une part éviter de heurter l'homme ou l'environnement et d'autre part adapter les limites cinématiques du robot en fonction de la proximité de l'homme. A chaque niveau du système (planification et exécution/contrôle), le robot doit garantir la sécurité et le confort de l'homme. Nous proposons une approche de la planification et du contrôle de mouvement basée sur des trajectoires polynomiales. Dans une première partie, nous présentons un générateur de trajectoires qui limite la vitesse, l'accélération et le jerk. Il génère des trajectoires composées de suites de segments de courbes cubiques. Le cas mono-dimensionnel est d'abord présenté puis étendu au cas multi-dimensionnel. Dans une deuxième partie, nous proposons d'approximer les trajectoires par des suites de triplets de segments de courbes cubiques. Cette méthode permet de calculer des trajectoires respectant une erreur maximale donnée. Ces générateurs de trajectoire sont intégrés au planificateur de chemin et produisent des trajectoires directement exécutables. Une application originale de l'approximation permet d'approximer une trajectoire définie dans l'espace cartésien par une trajectoire définie dans l'espace articulaire. Cette approche simplifie la structure du contrôleur du robot. La présence de l'homme dans l'espace de travail du robot nécessite une adaptation des trajectoires pendant l'exécution. Nous proposons une méthode pour adapter la loi de mouvement de la trajectoire multidimensionnelle pendant l'exécution. Ces travaux, menés dans le cadre du projet européen DEXMART et du projet ANR ASSIST, ont été intégrés et validés sur les plateformes Jido et PR2 du LAAS-CNRS.The context of service robotics is characterized by the presence of humans in the vicinity of the robot. The movements of these robots should not disturb the safety of humans or their comfort. From the motion planning point of view, the planner must both avoid hitting humans or colliding with the environment and also adapt the robot's kinematic limits depending on the proximity of humans. At each level of the system (Planning and execution / Control), the robot must ensure the safety and the comfort of humans. We propose an approach of motion planning and motion control based on polynomial trajectories. In the first part, we present a trajectory generator which limits the speed, the acceleration and the jerk (derivative of the acceleration). The motion planner generates trajectories consisting of series of segments of cubic polynomial curves. The mono-dimensional case is first introduced and then extended to the multi-dimensional one. In the second part, we propose to approximate the trajectories by sequences of triplets of segments of cubic curves. This method allows to find trajectories that respect a given maximum error. These trajectory generators are integrated into the path planner and produce directly executable motion. An original application of the trajectory approximation is the approximation of a trajectory defined in Cartesian space by a trajectory defined in the joint space. This approach simplifies the structure of the robot controller. The presence of humans in the workspace of the robot requires also an adaptation of the trajectories during the execution. We propose a method to adapt the motion law of the multidimensional path at runtime. This work, conducted as part of the European project DEXMART and the ANR project ASSIST, has been integrated and validated on the Jido and PR2 platforms of LAAS-CNRS