Cameras and Inertial/Magnetic Sensor Units Alignment Calibration

Due to the external acceleration interference/ magnetic disturbance, the inertial/magnetic measurements are usually fused with visual data for drift-free orientation estimation, which plays an important role in a wide variety of applications, ranging from virtual reality, robot, and computer vision to biomotion analysis and navigation. However, in order to perform data fusion, alignment calibration must be performed in advance to determine the difference between the sensor coordinate system and the camera coordinate system. Since orientation estimation performance of the inertial/magnetic sensor unit is immune to the selection of the inertial/magnetic sensor frame original point, we therefore ignore the translational difference by assuming the sensor and camera coordinate systems sharing the same original point and focus on the rotational alignment difference only in this paper. By exploiting the intrinsic restrictions among the coordinate transformations, the rotational alignment calibration problem is formulated by a simplified hand–eye equation AX = XB (A, X, and B are all rotation matrices). A two-step iterative algorithm is then proposed to solve such simplified handeye calibration task. Detailed laboratory validation has been performed and the good experimental results have illustrated the effectiveness of the proposed alignment calibration method

Survey of Motion Tracking Methods Based on Inertial Sensors: A Focus on Upper Limb Human Motion

Motion tracking based on commercial inertial measurements units (IMUs) has been widely studied in the latter years as it is a cost-effective enabling technology for those applications in which motion tracking based on optical technologies is unsuitable. This measurement method has a high impact in human performance assessment and human-robot interaction. IMU motion tracking systems are indeed self-contained and wearable, allowing for long-lasting tracking of the user motion in situated environments. After a survey on IMU-based human tracking, five techniques for motion reconstruction were selected and compared to reconstruct a human arm motion. IMU based estimation was matched against motion tracking based on the Vicon marker-based motion tracking system considered as ground truth. Results show that all but one of the selected models perform similarly (about 35 mm average position estimation error)

Robust human motion tracking using wireless and inertial sensors

Recently, miniature inertial measurement units (IMUs) have been deployed as wearable devices to monitor human motion in an ambulatory fashion. This thesis presents a robust human motion tracking algorithm using the IMU and radio-based wireless sensors, such as the Bluetooth Low Energy (BLE) and ultra-wideband (UWB). First, a novel indoor localization method using the BLE and IMU is proposed. The BLE trilateration residue is deployed to adaptively weight the estimates from these sensor modalities. Second, a robust sensor fusion algorithm is developed to accurately track the location and capture the lower body motion by integrating the estimates from the UWB system and IMUs, but also taking advantage of the estimated height and velocity obtained from an aiding lower body biomechanical model. The experimental results show that the proposed algorithms can maintain high accuracy for tracking the location of a sensor/subject in the presence of the BLE/UWB outliers and signal outages

Fusion of wearable and visual sensors for human motion analysis

Human motion analysis is concerned with the study of human activity recognition, human motion tracking, and the analysis of human biomechanics. Human motion analysis has applications within areas of entertainment, sports, and healthcare. For example, activity recognition, which aims to understand and identify different tasks from motion can be applied to create records of staff activity in the operating theatre at a hospital; motion tracking is already employed in some games to provide an improved user interaction experience and can be used to study how medical staff interact in the operating theatre; and human biomechanics, which is the study of the structure and function of the human body, can be used to better understand athlete performance, pathologies in certain patients, and assess the surgical skill of medical staff. As health services strive to improve the quality of patient care and meet the growing demands required to care for expanding populations around the world, solutions that can improve patient care, diagnosis of pathology, and the monitoring and training of medical staff are necessary. Surgical workflow analysis, for example, aims to assess and optimise surgical protocols in the operating theatre by evaluating the tasks that staff perform and measurable outcomes. Human motion analysis methods can be used to quantify the activities and performance of staff for surgical workflow analysis; however, a number of challenges must be overcome before routine motion capture of staff in an operating theatre becomes feasible. Current commercial human motion capture technologies have demonstrated that they are capable of acquiring human movement with sub-centimetre accuracy; however, the complicated setup procedures, size, and embodiment of current systems make them cumbersome and unsuited for routine deployment within an operating theatre. Recent advances in pervasive sensing have resulted in camera systems that can detect and analyse human motion, and small wear- able sensors that can measure a variety of parameters from the human body, such as heart rate, fatigue, balance, and motion. The work in this thesis investigates different methods that enable human motion to be more easily, reliably, and accurately captured through ambient and wearable sensor technologies to address some of the main challenges that have limited the use of motion capture technologies in certain areas of study. Sensor embodiment and accuracy of activity recognition is one of the challenges that affect the adoption of wearable devices for monitoring human activity. Using a single inertial sensor, which captures the movement of the subject, a variety of motion characteristics can be measured. For patients, wearable inertial sensors can be used in long-term activity monitoring to better understand the condition of the patient and potentially identify deviations from normal activity. For medical staff, inertial sensors can be used to capture tasks being performed for automated workflow analysis, which is useful for staff training, optimisation of existing processes, and early indications of complications within clinical procedures. Feature extraction and classification methods are introduced in thesis that demonstrate motion classification accuracies of over 90% for five different classes of walking motion using a single ear-worn sensor. To capture human body posture, current capture systems generally require a large number of sensors or reflective reference markers to be worn on the body, which presents a challenge for many applications, such as monitoring human motion in the operating theatre, as they may restrict natural movements and make setup complex and time consuming. To address this, a method is proposed, which uses a regression method to estimate motion using a subset of fewer wearable inertial sensors. This method is demonstrated using three sensors on the upper body and is shown to achieve mean estimation accuracies as low as 1.6cm, 1.1cm, and 1.4cm for the hand, elbow, and shoulders, respectively, when compared with the gold standard optical motion capture system. Using a subset of three sensors, mean errors for hand position reach 15.5cm. Unlike human motion capture systems that rely on vision and reflective reference point markers, commonly known as marker-based optical motion capture, wearable inertial sensors are prone to inaccuracies resulting from an accumulation of inaccurate measurements, which becomes increasingly prevalent over time. Two methods are introduced in this thesis, which aim to solve this challenge using visual rectification of the assumed state of the subject. Using a ceiling-mounted camera, a human detection and human motion tracking method is introduced to improve the average mean accuracy of tracking to within 5.8cm in a laboratory of 3m × 5m. To improve the accuracy of capturing the position of body parts and posture for human biomechanics, a camera is also utilised to track the body part movements and provide visual rectification of human pose estimates from inertial sensing. For most subjects, deviations of less than 10% from the ground truth are achieved for hand positions, which exhibit the greatest error, and the occurrence of sources of other common visual and inertial estimation errors, such as measurement noise, visual occlusion, and sensor calibration are shown to be reduced.Open Acces

Composite prototyping and vision based hierarchical control of a quad tilt-wing UAV

As the attention to unmanned systems is increasing, unmanned aerial vehicles (UAVs) are becoming more popular based on the rapid advances in technology and growth in operational experience. The main motivation in this vast research field is to diminish the human driven tasks by employing UAVs in critical civilian and military tasks such as traffic monitoring, disasters, surveillance, reconnaissance and border security. Researchers have been developing featured UAVs with intelligent navigation and control systems on more efficient designs aiming to increase the functionality, flight time and maneuverability. This thesis focuses on the composite prototyping and vision based hierarchical control of a quad tilt-wing aerial vehicle (SUAVI: Sabanci University Unmanned Aerial VehIcle). With the tilt-wing mechanism, SUAVI is one of the most challenging UAV concepts by combining advantages of vertical take-off and landing (VTOL) and horizontal flight. Various composite materials are tested for their mechanical properties and the most suitable one is used for prototyping of the aerial vehicle. A hierarchical control structure which consists of high-level and low-level controllers is developed. A vision based high-level controller generates attitude references for the low-level controllers. A Kalman filter fuses data from low-cost inertial sensors to obtain reliable orientation information. Low-level controllers are typically gravity compensated PID controllers. An image based visual servoing (IBVS) algorithm for VTOL, hovering and trajectory tracking is successfully implemented in simulations. Real flight tests demonstrate satisfactory performance of the developed control algorithms

DESIGN OF MOBILE DATA COLLECTOR BASED CLUSTERING ROUTING PROTOCOL FOR WIRELESS SENSOR NETWORKS

Wireless Sensor Networks (WSNs) consisting of hundreds or even thousands of nodes, canbe used for a multitude of applications such as warfare intelligence or to monitor the environment. A typical WSN node has a limited and usually an irreplaceable power source and the efficient use of the available power is of utmost importance to ensure maximum lifetime of eachWSNapplication. Each of the nodes needs to transmit and communicate sensed data to an aggregation point for use by higher layer systems. Data and message transmission among nodes collectively consume the largest amount of energy available in WSNs. The network routing protocols ensure that every message reaches thedestination and has a direct impact on the amount of transmissions to deliver messages successfully. To this end, the transmission protocol within the WSNs should be scalable, adaptable and optimized to consume the least possible amount of energy to suite different network architectures and application domains. The inclusion of mobile nodes in the WSNs deployment proves to be detrimental to protocol performance in terms of nodes energy efficiency and reliable message delivery. This thesis which proposes a novel Mobile Data Collector based clustering routing protocol for WSNs is designed that combines cluster based hierarchical architecture and utilizes three-tier multi-hop routing strategy between cluster heads to base station by the help of Mobile Data Collector (MDC) for inter-cluster communication. In addition, a Mobile Data Collector based routing protocol is compared with Low Energy Adaptive Clustering Hierarchy and A Novel Application Specific Network Protocol for Wireless Sensor Networks routing protocol. The protocol is designed with the following in mind: minimize the energy consumption of sensor nodes, resolve communication holes issues, maintain data reliability, finally reach tradeoff between energy efficiency and latency in terms of End-to-End, and channel access delays. Simulation results have shown that the Mobile Data Collector based clustering routing protocol for WSNs could be easily implemented in environmental applications where energy efficiency of sensor nodes, network lifetime and data reliability are major concerns