Sparse Inertial Poser: Automatic 3D Human Pose Estimation from Sparse IMUs

We address the problem of making human motion capture in the wild more practical by using a small set of inertial sensors attached to the body. Since the problem is heavily under-constrained, previous methods either use a large number of sensors, which is intrusive, or they require additional video input. We take a different approach and constrain the problem by: (i) making use of a realistic statistical body model that includes anthropometric constraints and (ii) using a joint optimization framework to fit the model to orientation and acceleration measurements over multiple frames. The resulting tracker Sparse Inertial Poser (SIP) enables 3D human pose estimation using only 6 sensors (attached to the wrists, lower legs, back and head) and works for arbitrary human motions. Experiments on the recently released TNT15 dataset show that, using the same number of sensors, SIP achieves higher accuracy than the dataset baseline without using any video data. We further demonstrate the effectiveness of SIP on newly recorded challenging motions in outdoor scenarios such as climbing or jumping over a wall.Comment: 12 pages, Accepted at Eurographics 201

3D Sensor Placement and Embedded Processing for People Detection in an Industrial Environment

Papers I, II and III are extracted from the dissertation and uploaded as separate documents to meet post-publication requirements for self-arciving of IEEE conference papers.At a time when autonomy is being introduced in more and more areas, computer vision plays a very important role. In an industrial environment, the ability to create a real-time virtual version of a volume of interest provides a broad range of possibilities, including safety-related systems such as vision based anti-collision and personnel tracking. In an offshore environment, where such systems are not common, the task is challenging due to rough weather and environmental conditions, but the result of introducing such safety systems could potentially be lifesaving, as personnel work close to heavy, huge, and often poorly instrumented moving machinery and equipment. This thesis presents research on important topics related to enabling computer vision systems in industrial and offshore environments, including a review of the most important technologies and methods. A prototype 3D sensor package is developed, consisting of different sensors and a powerful embedded computer. This, together with a novel, highly scalable point cloud compression and sensor fusion scheme allows to create a real-time 3D map of an industrial area. The question of where to place the sensor packages in an environment where occlusions are present is also investigated. The result is algorithms for automatic sensor placement optimisation, where the goal is to place sensors in such a way that maximises the volume of interest that is covered, with as few occluded zones as possible. The method also includes redundancy constraints where important sub-volumes can be defined to be viewed by more than one sensor. Lastly, a people detection scheme using a merged point cloud from six different sensor packages as input is developed. Using a combination of point cloud clustering, flattening and convolutional neural networks, the system successfully detects multiple people in an outdoor industrial environment, providing real-time 3D positions. The sensor packages and methods are tested and verified at the Industrial Robotics Lab at the University of Agder, and the people detection method is also tested in a relevant outdoor, industrial testing facility. The experiments and results are presented in the papers attached to this thesis.publishedVersio

Imaging with a small number of photons

Low-light-level imaging techniques have application in many diverse fields, ranging from biological sciences to security. We demonstrate a single-photon imaging system based on a time-gated inten- sified CCD (ICCD) camera in which the image of an object can be inferred from very few detected photons. We show that a ghost-imaging configuration, where the image is obtained from photons that have never interacted with the object, is a useful approach for obtaining images with high signal-to-noise ratios. The use of heralded single-photons ensures that the background counts can be virtually eliminated from the recorded images. By applying techniques of compressed sensing and associated image reconstruction, we obtain high-quality images of the object from raw data comprised of fewer than one detected photon per image pixel.Comment: 9 pages, 4 figure

Image registration algorithm for molecular tagging velocimetry applied to unsteady flow in Hele-Shaw cell

In order to develop velocimetry methods for confined geometries, we propose to combine image registration and volumetric reconstruction from a monocular video of the draining of a Hele-Shaw cell filled with water. The cell’s thickness is small compared to the other two dimensions (e.g. 1x400 x 800 mm3). We use a technique known as molecular tagging which consists in marking by photobleaching a pattern in the fluid and then tracking its deformations. The evolution of the pattern is filmed with a camera whose principal axis coincides with the cell’s gap. The velocity of the fluid along this direction is not constant. Consequently, tracking the pattern cannot be achieved with classical methods because what is observed is the integral of the marked molecules over the entire cell’s gap. The proposed approach is built on top of direct image registration that we extend to specifically model the volumetric image formation. It allows us to accurately measure the motion and the velocity profiles for the entire volume (including the cell’s gap) which is something usually hard to achieve. The results we obtained are consistent with the theoretical hydrodynamic behaviour for this flow which is known as the Poiseuille flow

Through-life modelling of nano-satellite power system dynamics

This paper presents a multi-fidelity approach to finding optimal, mission-specific power system configurations for CubeSats. The methodology begins with propagation of the orbit elements over the mission lifetime, via a continuous-time model, accounting for orbital perturbations (drag, solar radiation and non-spherical geo-potential). Analytical sizing of the power system is then achieved at discrete long-term intervals, to account for the effects of variations in environmental conditions over the mission life. This sizing is based on worst case power demand and provides inputs to a numerical assessment of the in-flight energy collection for each potential solar array deployment configuration. Finally, two objective functions (minimum deviation about the orbit average power and maximum average power over the entire mission) are satisfied to identify the configurations most suitable for the specific mission requirement. Most Nano-satellites are designed with relatively simple, static-models only and tend to be over-engineered as a result, often leading to a power-limited system. The approach described here aims to reduce the uncertainty in energy collection during flight and provide a robust approach to finding the optimal solution for a given set of mission requirements

Application of Saliency Maps for Optimizing Camera Positioning in Deep Learning Applications

In the fields of process control engineering and robotics, especially in automatic control, optimization challenges frequently manifest as complex problems with expensive evaluations. This thesis zeroes in on one such problem: the optimization of camera positions for Convolutional Neural Networks (CNNs). CNNs have specific attention points in images that are often not intuitive to human perception, making camera placement critical for performance. The research is guided by two primary questions. The first investigates the role of Explainable Artificial Intelligence (XAI), specifically GradCAM++ visual explanations, in Computer Vision for aiding in the evaluation of different camera positions. Building on this, the second question assesses a novel algorithm that leverages these XAI features against traditional black-box optimization methods. To answer these questions, the study employs a robotic auto-positioning system for data collection, CNN model training, and performance evaluation. A case study focused on classifying flow regimes in industrial-grade bioreactors validates the method. The proposed approach shows improvements over established techniques like Grid Search, Random Search, Bayesian optimization, and Simulated Annealing. Future work will focus on gathering more data and including noise for generalized conclusions.:Contents 1 Introduction 1.1 Motivation 1.2 Problem Analysis 1.3 Research Question 1.4 Structure of the Thesis 2 State of the Art 2.1 Literature Research Methodology 2.1.1 Search Strategy 2.1.2 Inclusion and Exclusion Criteria 2.2 Blackbox Optimization 2.3 Mathematical Notation 2.4 Bayesian Optimization 2.5 Simulated Annealing 2.6 Random Search 2.7 Gridsearch 2.8 Explainable A.I. and Saliency Maps 2.9 Flowregime Classification in Stirred Vessels 2.10 Performance Metrics 2.10.1 R2 Score and Polynomial Regression for Experiment Data Analysis 2.10.2 Blackbox Optimization Performance Metrics 2.10.3 CNN Performance Metrics 3 Methodology 3.1 Requirement Analysis and Research Hypothesis 3.2 Research Approach: Case Study 3.3 Data Collection 3.4 Evaluation and Justification 4 Concept 4.1 System Overview 4.2 Data Flow 4.3 Experimental Setup 4.4 Optimization Challenges and Approaches 5 Data Collection and Experimental Setup 5.1 Hardware Components 5.2 Data Recording and Design of Experiments 5.3 Data Collection 5.4 Post-Experiment 6 Implementation 6.1 Simulation Unit 6.2 Recommendation Scalar from Saliency Maps 6.3 Saliency Map Features as Guidance Mechanism 6.4 GradCam++ Enhanced Bayesian Optimization 6.5 Benchmarking Unit 6.6 Benchmarking 7 Results and Evaluation 7.1 Experiment Data Analysis 7.2 Recommendation Scalar 7.3 Benchmarking Results and Quantitative Analysis 7.3.1 Accuracy Results from the Benchmarking Process 7.3.2 Cumulative Results Interpretation 7.3.3 Analysis of Variability 7.4 Answering the Research Questions 7.5 Summary 8 Discussion 8.1 Critical Examination of Limitations 8.2 Discussion of Solutions to Limitations 8.3 Practice-Oriented Discussion of Findings 9 Summary and OutlookIm Bereich der Prozessleittechnik und Robotik, speziell bei der automatischen Steuerung, treten oft komplexe Optimierungsprobleme auf. Diese Arbeit konzentriert sich auf die Optimierung der Kameraplatzierung in Anwendungen, die Convolutional Neural Networks (CNNs) verwenden. Da CNNs spezifische, für den Menschen nicht immer ersichtliche, Merkmale in Bildern hervorheben, ist die intuitive Platzierung der Kamera oft nicht optimal. Zwei Forschungsfragen leiten diese Arbeit: Die erste Frage untersucht die Rolle von Erklärbarer Künstlicher Intelligenz (XAI) in der Computer Vision zur Bereitstellung von Merkmalen für die Bewertung von Kamerapositionen. Die zweite Frage vergleicht einen darauf basierenden Algorithmus mit anderen Blackbox-Optimierungstechniken. Ein robotisches Auto-Positionierungssystem wird zur Datenerfassung und für Experimente eingesetzt. Als Lösungsansatz wird eine Methode vorgestellt, die XAI-Merkmale, insbesondere solche aus GradCAM++ Erkenntnissen, mit einem Bayesschen Optimierungsalgorithmus kombiniert. Diese Methode wird in einer Fallstudie zur Klassifizierung von Strömungsregimen in industriellen Bioreaktoren angewendet und zeigt eine gesteigerte performance im Vergleich zu etablierten Methoden. Zukünftige Forschung wird sich auf die Sammlung weiterer Daten, die Inklusion von verrauschten Daten und die Konsultation von Experten für eine kostengünstigere Implementierung konzentrieren.:Contents 1 Introduction 1.1 Motivation 1.2 Problem Analysis 1.3 Research Question 1.4 Structure of the Thesis 2 State of the Art 2.1 Literature Research Methodology 2.1.1 Search Strategy 2.1.2 Inclusion and Exclusion Criteria 2.2 Blackbox Optimization 2.3 Mathematical Notation 2.4 Bayesian Optimization 2.5 Simulated Annealing 2.6 Random Search 2.7 Gridsearch 2.8 Explainable A.I. and Saliency Maps 2.9 Flowregime Classification in Stirred Vessels 2.10 Performance Metrics 2.10.1 R2 Score and Polynomial Regression for Experiment Data Analysis 2.10.2 Blackbox Optimization Performance Metrics 2.10.3 CNN Performance Metrics 3 Methodology 3.1 Requirement Analysis and Research Hypothesis 3.2 Research Approach: Case Study 3.3 Data Collection 3.4 Evaluation and Justification 4 Concept 4.1 System Overview 4.2 Data Flow 4.3 Experimental Setup 4.4 Optimization Challenges and Approaches 5 Data Collection and Experimental Setup 5.1 Hardware Components 5.2 Data Recording and Design of Experiments 5.3 Data Collection 5.4 Post-Experiment 6 Implementation 6.1 Simulation Unit 6.2 Recommendation Scalar from Saliency Maps 6.3 Saliency Map Features as Guidance Mechanism 6.4 GradCam++ Enhanced Bayesian Optimization 6.5 Benchmarking Unit 6.6 Benchmarking 7 Results and Evaluation 7.1 Experiment Data Analysis 7.2 Recommendation Scalar 7.3 Benchmarking Results and Quantitative Analysis 7.3.1 Accuracy Results from the Benchmarking Process 7.3.2 Cumulative Results Interpretation 7.3.3 Analysis of Variability 7.4 Answering the Research Questions 7.5 Summary 8 Discussion 8.1 Critical Examination of Limitations 8.2 Discussion of Solutions to Limitations 8.3 Practice-Oriented Discussion of Findings 9 Summary and Outloo

Modelling of building interiors with mobile phone sensor data

Creating as-built plans of building interiors is a challenging task. In this paper we present a semi-automatic modelling system for creating residential building interior plans and their integration with existing map data to produce building models. Taking a set of imprecise measurements made with an interactive mobile phone room mapping application, the system performs spatial adjustments in accordance with soft and hard constraints imposed on the building plan geometry. The approach uses an optimisation model that exploits a high accuracy building outline, such as can be found in topographic map data, and the building topology to improve the quality of interior measurements and generate a standardised output. We test our system on building plans of five residential homes. Our evaluation shows that the approach enables construction of accurate interior plans from imprecise measurements. The experiments report an average accuracy of 0.24 m, close to the 0.20 m recommended by the CityGML LoD4 specificatio

Thermal error modelling of machine tools based on ANFIS with fuzzy c-means clustering using a thermal imaging camera

Thermal errors are often quoted as being the largest contributor to CNC machine tool errors, but they can be effectively reduced using error compensation. The performance of a thermal error compensation system depends on the accuracy and robustness of the thermal error model and the quality of the inputs to the model. The location of temperature measurement must provide a representative measurement of the change in temperature that will affect the machine structure. The number of sensors and their locations are not always intuitive and the time required to identify the optimal locations is often prohibitive, resulting in compromise and poor results. In this paper, a new intelligent compensation system for reducing thermal errors of machine tools using data obtained from a thermal imaging camera is introduced. Different groups of key temperature points were identified from thermal images using a novel schema based on a Grey model GM (0, N) and Fuzzy c-means (FCM) clustering method. An Adaptive Neuro-Fuzzy Inference System with Fuzzy c-means clustering (FCM-ANFIS) was employed to design the thermal prediction model. In order to optimise the approach, a parametric study was carried out by changing the number of inputs and number of membership functions to the FCM-ANFIS model, and comparing the relative robustness of the designs. According to the results, the FCM-ANFIS model with four inputs and six membership functions achieves the best performance in terms of the accuracy of its predictive ability. The residual value of the model is smaller than ± 2 μm, which represents a 95% reduction in the thermally-induced error on the machine. Finally, the proposed method is shown to compare favourably against an Artificial Neural Network (ANN) model

Optimizing Fault-Tolerant Quality-Guaranteed Sensor Deployments for UAV Localization in Critical Areas via Computational Geometry

The increasing spreading of small commercial Unmanned Aerial Vehicles (UAVs, aka drones) presents serious threats for critical areas such as airports, power plants, governmental and military facilities. In fact, such UAVs can easily disturb or jam radio communications, collide with other flying objects, perform espionage activity, and carry offensive payloads, e.g., weapons or explosives. A central problem when designing surveillance solutions for the localization of unauthorized UAVs in critical areas is to decide how many triangulating sensors to use, and where to deploy them to optimise both coverage and cost effectiveness. In this article, we compute deployments of triangulating sensors for UAV localization, optimizing a given blend of metrics, namely: coverage under multiple sensing quality levels, cost-effectiveness, fault-tolerance. We focus on large, complex 3D regions, which exhibit obstacles (e.g., buildings), varying terrain elevation, different coverage priorities, constraints on possible sensors placement. Our novel approach relies on computational geometry and statistical model checking, and enables the effective use of off-the-shelf AI-based black-box optimizers. Moreover, our method allows us to compute a closed-form, analytical representation of the region uncovered by a sensor deployment, which provides the means for rigorous, formal certification of the quality of the latter. We show the practical feasibility of our approach by computing optimal sensor deployments for UAV localization in two large, complex 3D critical regions, the Rome Leonardo Da Vinci International Airport (FCO) and the Vienna International Center (VIC), using NOMAD as our state-of-the-art underlying optimization engine. Results show that we can compute optimal sensor deployments within a few hours on a standard workstation and within minutes on a small parallel infrastructure