CHARMIE: a collaborative healthcare and home service and assistant robot for elderly care

The global population is ageing at an unprecedented rate. With changes in life expectancy across the world, three major issues arise: an increasing proportion of senior citizens; cognitive and physical problems progressively affecting the elderly; and a growing number of single-person households. The available data proves the ever-increasing necessity for efficient elderly care solutions such as healthcare service and assistive robots. Additionally, such robotic solutions provide safe healthcare assistance in public health emergencies such as the SARS-CoV-2 virus (COVID-19). CHARMIE is an anthropomorphic collaborative healthcare and domestic assistant robot capable of performing generic service tasks in non-standardised healthcare and domestic environment settings. The combination of its hardware and software solutions demonstrates map building and self-localisation, safe navigation through dynamic obstacle detection and avoidance, different human-robot interaction systems, speech and hearing, pose/gesture estimation and household object manipulation. Moreover, CHARMIE performs end-to-end chores in nursing homes, domestic houses, and healthcare facilities. Some examples of these chores are to help users transport items, fall detection, tidying up rooms, user following, and set up a table. The robot can perform a wide range of chores, either independently or collaboratively. CHARMIE provides a generic robotic solution such that older people can live longer, more independent, and healthier lives.This work has been supported by FCT—Fundação para a Ciência e Tecnologia within the R&D Units Project Scope: UIDB/00319/2020. The author T.R. received funding through a doctoral scholarship from the Portuguese Foundation for Science and Technology (Fundação para a Ciência e a Tecnologia) [grant number SFRH/BD/06944/2020], with funds from the Portuguese Ministry of Science, Technology and Higher Education and the European Social Fund through the Programa Operacional do Capital Humano (POCH). The author F.G. received funding through a doctoral scholarship from the Portuguese Foundation for Science and Technology (Fundação para a Ciência e a Tecnologia) [grant number SFRH/BD/145993/2019], with funds from the Portuguese Ministry of Science, Technology and Higher Education and the European Social Fund through the Programa Operacional do Capital Humano (POCH)

Adaptive and learning-based formation control of swarm robots

Autonomous aerial and wheeled mobile robots play a major role in tasks such as search and rescue, transportation, monitoring, and inspection. However, these operations are faced with a few open challenges including robust autonomy, and adaptive coordination based on the environment and operating conditions, particularly in swarm robots with limited communication and perception capabilities. Furthermore, the computational complexity increases exponentially with the number of robots in the swarm. This thesis examines two different aspects of the formation control problem. On the one hand, we investigate how formation could be performed by swarm robots with limited communication and perception (e.g., Crazyflie nano quadrotor). On the other hand, we explore human-swarm interaction (HSI) and different shared-control mechanisms between human and swarm robots (e.g., BristleBot) for artistic creation. In particular, we combine bio-inspired (i.e., flocking, foraging) techniques with learning-based control strategies (using artificial neural networks) for adaptive control of multi- robots. We first review how learning-based control and networked dynamical systems can be used to assign distributed and decentralized policies to individual robots such that the desired formation emerges from their collective behavior. We proceed by presenting a novel flocking control for UAV swarm using deep reinforcement learning. We formulate the flocking formation problem as a partially observable Markov decision process (POMDP), and consider a leader-follower configuration, where consensus among all UAVs is used to train a shared control policy, and each UAV performs actions based on the local information it collects. In addition, to avoid collision among UAVs and guarantee flocking and navigation, a reward function is added with the global flocking maintenance, mutual reward, and a collision penalty. We adapt deep deterministic policy gradient (DDPG) with centralized training and decentralized execution to obtain the flocking control policy using actor-critic networks and a global state space matrix. In the context of swarm robotics in arts, we investigate how the formation paradigm can serve as an interaction modality for artists to aesthetically utilize swarms. In particular, we explore particle swarm optimization (PSO) and random walk to control the communication between a team of robots with swarming behavior for musical creation

NAVIGATION STRATEGY FOR MOBILE ROBOT BASED ON COMPUTER VISION AND YOLOV5 NETWORK IN THE UNKNOWN ENVIRONMENT

Intelligent mobile robots must possess the ability to navigate in complex environments. The field of mobile robot navigation is continuously evolving, with various technologies being developed. Deep learning has gained attention from researchers, and numerous navigation models utilizing deep learning have been proposed. In this study, the YOLOv5 model is utilized to identify objects to aid the mobile robot in determining movement conditions. However, the limitation of deep learning models being trained on insufficient data, leading to inaccurate recognition in unforeseen scenarios, is addressed by introducing an innovative computer vision technology that detects lanes in real-time. Combining the deep learning model with computer vision technology, the robot can identify different types of objects, allowing it to estimate distance and adjust speed accordingly. Additionally, the paper investigates the recognition reliability in varying light intensities. The findings of this study offer promising directions for future breakthroughs in mobile robot navigatio

Dataset of Panoramic Images for People Tracking in Service Robotics

We provide a framework for constructing a guided robot for usage in hospitals in this thesis. The omnidirectional camera on the robot allows it to recognize and track the person who is following it. Furthermore, when directing the individual to their preferred position in the hospital, the robot must be aware of its surroundings and avoid accidents with other people or items. To train and evaluate our robot's performance, we developed an auto-labeling framework for creating a dataset of panoramic videos captured by the robot's omnidirectional camera. We labeled each person in the video and their real position in the robot's frame, enabling us to evaluate the accuracy of our tracking system and guide the development of the robot's navigation algorithms. Our research expands on earlier work that has established a framework for tracking individuals using omnidirectional cameras. We want to contribute to the continuing work to enhance the precision and dependability of these tracking systems, which is essential for the creation of efficient guiding robots in healthcare facilities, by developing a benchmark dataset. Our research has the potential to improve the patient experience and increase the efficiency of healthcare institutions by reducing staff time spent guiding patients through the facility.We provide a framework for constructing a guided robot for usage in hospitals in this thesis. The omnidirectional camera on the robot allows it to recognize and track the person who is following it. Furthermore, when directing the individual to their preferred position in the hospital, the robot must be aware of its surroundings and avoid accidents with other people or items. To train and evaluate our robot's performance, we developed an auto-labeling framework for creating a dataset of panoramic videos captured by the robot's omnidirectional camera. We labeled each person in the video and their real position in the robot's frame, enabling us to evaluate the accuracy of our tracking system and guide the development of the robot's navigation algorithms. Our research expands on earlier work that has established a framework for tracking individuals using omnidirectional cameras. We want to contribute to the continuing work to enhance the precision and dependability of these tracking systems, which is essential for the creation of efficient guiding robots in healthcare facilities, by developing a benchmark dataset. Our research has the potential to improve the patient experience and increase the efficiency of healthcare institutions by reducing staff time spent guiding patients through the facility

Sistem Pengikut Manusia pada Robot Servis Menggunakan Model YOLO dan Kamera Stereo

Kemampuan mengikuti seseorang merupakan fitur penting bagi robot servis yang bekerja berdampingan dengan manusia. Untuk merancang sistem pengikut manusia pada robot servis, diperlukan akurasi yang tinggi tapi juga tanpa mengorbankan kecepatan komputasi agar sistem berjalan secara real-time. Penelitian ini bertujuan untuk merancang sebuah sistem pengikut manusia untuk robot servis dengan memanfaatkan model pendeteksi objek You Only Look Once (YOLO) dan kamera stereo. Sistem ini dirancang agar robot dapat menjaga jarak yang tetap dari target yang diikuti dan menjaga orientasinya sehingga target tetap berada di tengah pandangan robot. Perancangan sistem ini juga memanfaatkan algoritma pelacak dari OpenCV yang dikoreksi dengan model YOLOv7 setiap 20 frame untuk menghasilkan proses yang lebih cepat. Pengontrol PID digunakan untuk menghasilkan kecepatan linear dan angular robot berdasarkan jarak relatif orang yang dijadikan target dari robot dan posisinya pada frame. Robot Operating System (ROS) digunakan untuk mem-publish kecepatan pada node yang sesuai. Berdasarkan hasil pengujian algoritma pelacak, pelacak Boosting memiliki hasil terbaik untuk digunakan. Selanjutnya, sistem ini diuji untuk mengontrol robot servis di dalam ruangan dengan berbagai variasi kondisi. Dari pengujian-pengujian tersebut, robot berhasil untuk mengikuti seseorang dengan eror RMS sebesar 41,88 mm dan standar deviasi sebesar 35,59 mm saat robot berhenti di jarak 1 m dari target. Nilai eror terbesar yang didapat bernilai 320,369 mm yang terjadi ketika sistem dijalankan pada ruangan gelap. Sistem ini berjalan dengan frame rate rata-rata sebesar 17,18 FPS

Edge Artificial Intelligence for Real-Time Target Monitoring

The key enabling technology for the exponentially growing cellular communications sector is location-based services. The need for location-aware services has increased along with the number of wireless and mobile devices. Estimation problems, and particularly parameter estimation, have drawn a lot of interest because of its relevance and engineers' ongoing need for higher performance. As applications expanded, a lot of interest was generated in the accurate assessment of temporal and spatial properties. In the thesis, two different approaches to subject monitoring are thoroughly addressed. For military applications, medical tracking, industrial workers, and providing location-based services to the mobile user community, which is always growing, this kind of activity is crucial. In-depth consideration is given to the viability of applying the Angle of Arrival (AoA) and Receiver Signal Strength Indication (RSSI) localization algorithms in real-world situations. We presented two prospective systems, discussed them, and presented specific assessments and tests. These systems were put to the test in diverse contexts (e.g., indoor, outdoor, in water...). The findings showed the localization capability, but because of the low-cost antenna we employed, this method is only practical up to a distance of roughly 150 meters. Consequently, depending on the use-case, this method may or may not be advantageous. An estimation algorithm that enhances the performance of the AoA technique was implemented on an edge device. Another approach was also considered. Radar sensors have shown to be durable in inclement weather and bad lighting conditions. Frequency Modulated Continuous Wave (FMCW) radars are the most frequently employed among the several sorts of radar technologies for these kinds of applications. Actually, this is because they are low-cost and can simultaneously provide range and Doppler data. In comparison to pulse and Ultra Wide Band (UWB) radar sensors, they also need a lower sample rate and a lower peak to average ratio. The system employs a cutting-edge surveillance method based on widely available FMCW radar technology. The data processing approach is built on an ad hoc-chain of different blocks that transforms data, extract features, and make a classification decision before cancelling clutters and leakage using a frame subtraction technique, applying DL algorithms to Range-Doppler (RD) maps, and adding a peak to cluster assignment step before tracking targets. In conclusion, the FMCW radar and DL technique for the RD maps performed well together for indoor use-cases. The aforementioned tests used an edge device and Infineon Technologies' Position2Go FMCW radar tool-set