Pushing the envelope for estimating poses and actions via full 3D reconstruction

Estimating poses and actions of human bodies and hands is an important task in the computer vision community due to its vast applications, including human computer interaction, virtual reality and augmented reality, medical image analysis. Challenges: There are many in-the-wild challenges in this task (see chapter 1). Among them, in this thesis, we focused on two challenges which could be relieved by incorporating the 3D geometry: (1) inherent 2D-to-3D ambiguity driven by the non-linear 2D projection process when capturing 3D objects. (2) lack of sufficient and quality annotated datasets due to the high-dimensionality of subjects' attribute space and inherent difficulty in annotating 3D coordinate values. Contributions: We first tried to jointly tackle the 2D-to-3D ambiguity and insufficient data issues by (1) explicitly reconstructing 2.5D and 3D samples and use them as new training data to train a pose estimator. Next, we tried to (2) encode 3D geometry in the training process of the action recognizer to reduce the 2D-to-3D ambiguity. In appendix, we proposed a (3) new hand pose synthetic dataset that can be used for more complete attribute changes and multi-modal experiments in the future. Experiments: Throughout experiments, we found interesting facts: (1) 2.5D depth map reconstruction and data augmentation can improve the accuracy of the depth-based hand pose estimation algorithm, (2) 3D mesh reconstruction can be used to generate a new RGB data and it improves the accuracy of RGB-based dense hand pose estimation algorithm, (3) 3D geometry from 3D poses and scene layouts could be successfully utilized to reduce the 2D-to-3D ambiguity in the action recognition problem.Open Acces

Simulation-based Planning of Machine Vision Inspection Systems with an Application to Laser Triangulation

Nowadays, vision systems play a central role in industrial inspection. The experts typically choose the configuration of measurements in such systems empirically. For complex inspections, however, automatic inspection planning is essential. This book proposes a simulation-based approach towards inspection planning by contributing to all components of this problem: simulation, evaluation, and optimization. As an application, inspection of a complex cylinder head by laser triangulation is studied

Enhancing numerical modelling efficiency for electromagnetic simulation of physical layer components.

The purpose of this thesis is to present solutions to overcome several key difficulties that limit the application of numerical modelling in communication cable design and analysis. In particular, specific limiting factors are that simulations are time consuming, and the process of comparison requires skill and is poorly defined and understood. When much of the process of design consists of optimisation of performance within a well defined domain, the use of artificial intelligence techniques may reduce or remove the need for human interaction in the design process. The automation of human processes allows round-the-clock operation at a faster throughput. Achieving a speedup would permit greater exploration of the possible designs, improving understanding of the domain. This thesis presents work that relates to three facets of the efficiency of numerical modelling: minimizing simulation execution time, controlling optimization processes and quantifying comparisons of results. These topics are of interest because simulation times for most problems of interest run into tens of hours. The design process for most systems being modelled may be considered an optimisation process in so far as the design is improved based upon a comparison of the test results with a specification. Development of software to automate this process permits the improvements to continue outside working hours, and produces decisions unaffected by the psychological state of a human operator. Improved performance of simulation tools would facilitate exploration of more variations on a design, which would improve understanding of the problem domain, promoting a virtuous circle of design. The minimization of execution time was achieved through the development of a Parallel TLM Solver which did not use specialized hardware or a dedicated network. Its design was novel because it was intended to operate on a network of heterogeneous machines in a manner which was fault tolerant, and included a means to reduce vulnerability of simulated data without encryption. Optimisation processes were controlled by genetic algorithms and particle swarm optimisation which were novel applications in communication cable design. The work extended the range of cable parameters, reducing conductor diameters for twisted pair cables, and reducing optical coverage of screens for a given shielding effectiveness. Work on the comparison of results introduced ―Colour maps‖ as a way of displaying three scalar variables over a two-dimensional surface, and comparisons were quantified by extending 1D Feature Selective Validation (FSV) to two dimensions, using an ellipse shaped filter, in such a way that it could be extended to higher dimensions. In so doing, some problems with FSV were detected, and suggestions for overcoming these presented: such as the special case of zero valued DC signals. A re-description of Feature Selective Validation, using Jacobians and tensors is proposed, in order to facilitate its implementation in higher dimensional spaces

A comparative analysis of algorithms for satellite operations scheduling

Scheduling is employed in everyday life, ranging from meetings to manufacturing and operations among other activities. One instance of scheduling in a complex real-life setting is space mission operations scheduling, i.e. instructing a satellite to perform fitting tasks during predefined time periods with a varied frequency to achieve its mission goals. Mission operations scheduling is pivotal to the success of any space mission, choreographing every task carefully, accounting for technological and environmental limitations and constraints along with mission goals.;It remains standard practice to this day, to generate operations schedules manually ,i.e. to collect requirements from individual stakeholders, collate them into a timeline, compare against feasibility and available satellite resources, and find potential conflicts. Conflict resolution is done by hand, checked by a simulator and uplinked to the satellite weekly. This process is time consuming, bears risks and can be considered sub-optimal.;A pertinent question arises: can we automate the process of satellite mission operations scheduling? And if we can, what method should be used to generate the schedules? In an attempt to address this question, a comparison of algorithms was deemed suitable in order to explore their suitability for this particular application.;The problem of mission operations scheduling was initially studied through literature and numerous interviews with experts. A framework was developed to approximate a generic Low Earth Orbit satellite, its environment and its mission requirements. Optimisation algorithms were chosen from different categories such as single-point stochastic without memory (Simulated Annealing, Random Search), multi-point stochastic with memory (Genetic Algorithm, Ant Colony System, Differential Evolution) and were run both with and without Local Search.;The aforementioned algorithmic set was initially tuned using a single 89-minute Low Earth Orbit of a scientific mission to Mars. It was then applied to scheduling operations during one high altitude Low Earth Orbit (2.4hrs) of an experimental mission.;It was then applied to a realistic test-case inspired by the European Space Agency PROBA-2 mission, comprising a 1 day schedule and subsequently a 7 day schedule - equal to a Short Term Plan as defined by the European Space Agency.;The schedule fitness - corresponding to the Hamming distance between mission requirements and generated schedule - are presented along with the execution time of each run. Algorithmic performance is discussed and put at the disposal of mission operations experts for consideration.Scheduling is employed in everyday life, ranging from meetings to manufacturing and operations among other activities. One instance of scheduling in a complex real-life setting is space mission operations scheduling, i.e. instructing a satellite to perform fitting tasks during predefined time periods with a varied frequency to achieve its mission goals. Mission operations scheduling is pivotal to the success of any space mission, choreographing every task carefully, accounting for technological and environmental limitations and constraints along with mission goals.;It remains standard practice to this day, to generate operations schedules manually ,i.e. to collect requirements from individual stakeholders, collate them into a timeline, compare against feasibility and available satellite resources, and find potential conflicts. Conflict resolution is done by hand, checked by a simulator and uplinked to the satellite weekly. This process is time consuming, bears risks and can be considered sub-optimal.;A pertinent question arises: can we automate the process of satellite mission operations scheduling? And if we can, what method should be used to generate the schedules? In an attempt to address this question, a comparison of algorithms was deemed suitable in order to explore their suitability for this particular application.;The problem of mission operations scheduling was initially studied through literature and numerous interviews with experts. A framework was developed to approximate a generic Low Earth Orbit satellite, its environment and its mission requirements. Optimisation algorithms were chosen from different categories such as single-point stochastic without memory (Simulated Annealing, Random Search), multi-point stochastic with memory (Genetic Algorithm, Ant Colony System, Differential Evolution) and were run both with and without Local Search.;The aforementioned algorithmic set was initially tuned using a single 89-minute Low Earth Orbit of a scientific mission to Mars. It was then applied to scheduling operations during one high altitude Low Earth Orbit (2.4hrs) of an experimental mission.;It was then applied to a realistic test-case inspired by the European Space Agency PROBA-2 mission, comprising a 1 day schedule and subsequently a 7 day schedule - equal to a Short Term Plan as defined by the European Space Agency.;The schedule fitness - corresponding to the Hamming distance between mission requirements and generated schedule - are presented along with the execution time of each run. Algorithmic performance is discussed and put at the disposal of mission operations experts for consideration

An Adjectival Interface for procedural content generation

Includes abstract.Includes bibliographical references.In this thesis, a new interface for the generation of procedural content is proposed, in which the user describes the content that they wish to create by using adjectives. Procedural models are typically controlled by complex parameters and often require expert technical knowledge. Since people communicate with each other using language, an adjectival interface to the creation of procedural content is a natural step towards addressing the needs of non-technical and non-expert users. The key problem addressed is that of establishing a mapping between adjectival descriptors, and the parameters employed by procedural models. We show how this can be represented as a mapping between two multi-dimensional spaces, adjective space and parameter space, and approximate the mapping by applying novel function approximation techniques to points of correspondence between the two spaces. These corresponding point pairs are established through a training phase, in which random procedural content is generated and then described, allowing one to map from parameter space to adjective space. Since we ultimately seek a means of mapping from adjective space to parameter space, particle swarm optimisation is employed to select a point in parameter space that best matches any given point in adjective space. The overall result, is a system in which the user can specify adjectives that are then used to create appropriate procedural content, by mapping the adjectives to a suitable set of procedural parameters and employing the standard procedural technique using those parameters as inputs. In this way, none of the control offered by procedural modelling is sacrificed â although the adjectival interface is simpler, it can at any point be stripped away to reveal the standard procedural model and give users access to the full set of procedural parameters. As such, the adjectival interface can be used for rapid prototyping to create an approximation of the content desired, after which the procedural parameters can be used to fine-tune the result. The adjectival interface also serves as a means of intermediate bridging, affording users a more comfortable interface until they are fully conversant with the technicalities of the underlying procedural parameters. Finally, the adjectival interface is compared and contrasted to an interface that allows for direct specification of the procedural parameters. Through user experiments, it is found that the adjectival interface presented in this thesis is not only easier to use and understand, but also that it produces content which more accurately reflects usersâ intentions

Model-based human upper body tracking using interest points in real-time video

Vision-based human motion analysis has received huge attention from researchers because of the number of applications, such as automated surveillance, video indexing, human machine interaction, traffic monitoring, and vehicle navigation. However, it contains several open problems. To date, despite very promising proposed approaches, no explicit solution has been found to solve these open problems efficiently. In this regard, this thesis presents a model-based human upper body pose estimation and tracking system using interest points (IPs) in real-time video. In the first stage, we propose a novel IP-based background-subtraction algorithm to segment the foreground IPs of each frame from the background ones. Afterwards, the foreground IPs of any two consecutive frames are matched to each other using a dynamic hybrid localspatial IP matching algorithm, proposed in this research. The IP matching algorithm starts by using the local feature descriptors of the IPs to find an initial set of possible matches. Then two filtering steps are applied to the results to increase the precision by deleting the mismatched pairs. To improve the recall, a spatial matching process is applied to the remaining unmatched points. Finally, a two-stage hierarchical-global model-based pose estimation and tracking algorithm based on Particle Swarm Optimiation (PSO) is proposed to track the human upper body through consecutive frames. Given the pose and the foreground IPs in the previous frame and the matched points in the current frame, the proposed PSO-based pose estimation and tracking algorithm estimates the current pose hierarchically by minimizing the discrepancy between the hypothesized pose and the real matched observed points in the first stage. Then a global PSO is applied to the pose estimated by the first stage to do a consistency check and pose refinement

Heterogeneous Ant Colony Optimisation Methods and their Application to the Travelling Salesman and PCB Drilling Problems

Ant Colony Optimization (ACO) is an optimization algorithm that is inspired by the foraging behaviour of real ants in locating and transporting food source to their nest. It is designed as a population-based metaheuristic and have been successfully implemented on various NP-hard problems such as the well-known Traveling Salesman Problem (TSP), Vehicle Routing Problem (VRP) and many more. However, majority of the studies in ACO focused on homogeneous artificial ants although animal behaviour researchers suggest that real ants exhibit heterogeneous behaviour thus improving the overall efficiency of the ant colonies. Equally important is that most, if not all, optimization algorithms require proper parameter tuning to achieve optimal performance. However, it is well-known that parameters are problem-dependant as different problems or even different instances have different optimal parameter settings. Parameter tuning through the testing of parameter combinations is a computationally expensive procedure that is infeasible on large-scale real-world problems. One method to mitigate this is to introduce heterogeneity by initializing the artificial agents with individual parameters rather than colony level parameters. This allows the algorithm to either actively or passively discover good parameter settings during the search. The approach undertaken in this study is to randomly initialize the ants from both uniform and Gaussian distribution respectively within a predefined range of values. The approach taken in this study is one of biological plausibility for ants with similar roles, but differing behavioural traits, which are being drawn from a mathematical distribution. This study also introduces an adaptive approach to the heterogeneous ant colony population that evolves the alpha and beta controlling parameters for ACO to locate near-optimal solutions. The adaptive approach is able to modify the exploitation and exploration characteristics of the algorithm during the search to reflect the dynamic nature of search. An empirical analysis of the proposed algorithm tested on a range of Travelling Salesman Problem (TSP) instances shows that the approach has better algorithmic performance when compared against state-of-the-art algorithms from the literature

Robust vision based slope estimation and rocks detection for autonomous space landers

As future robotic surface exploration missions to other planets, moons and asteroids become more ambitious in their science goals, there is a rapidly growing need to significantly enhance the capabilities of entry, descent and landing technology such that landings can be carried out with pin-point accuracy at previously inaccessible sites of high scientific value. As a consequence of the extreme uncertainty in touch-down locations of current missions and the absence of any effective hazard detection and avoidance capabilities, mission designers must exercise extreme caution when selecting candidate landing sites. The entire landing uncertainty footprint must be placed completely within a region of relatively flat and hazard free terrain in order to minimise the risk of mission ending damage to the spacecraft at touchdown. Consequently, vast numbers of scientifically rich landing sites must be rejected in favour of safer alternatives that may not offer the same level of scientific opportunity. The majority of truly scientifically interesting locations on planetary surfaces are rarely found in such hazard free and easily accessible locations, and so goals have been set for a number of advanced capabilities of future entry, descent and landing technology. Key amongst these is the ability to reliably detect and safely avoid all mission critical surface hazards in the area surrounding a pre-selected landing location. This thesis investigates techniques for the use of a single camera system as the primary sensor in the preliminary development of a hazard detection system that is capable of supporting pin-point landing operations for next generation robotic planetary landing craft. The requirements for such a system have been stated as the ability to detect slopes greater than 5 degrees and surface objects greater than 30cm in diameter. The primary contribution in this thesis, aimed at achieving these goals, is the development of a feature-based,self-initialising, fully adaptive structure from motion (SFM) algorithm based on a robust square-root unscented Kalman filtering framework and the fusion of the resulting SFM scene structure estimates with a sophisticated shape from shading (SFS) algorithm that has the potential to produce very dense and highly accurate digital elevation models (DEMs) that possess sufficient resolution to achieve the sensing accuracy required by next generation landers. Such a system is capable of adapting to potential changes in the external noise environment that may result from intermittent and varying rocket motor thrust and/or sudden turbulence during descent, which may translate to variations in the vibrations experienced by the platform and introduce varying levels of motion blur that will affect the accuracy of image feature tracking algorithms. Accurate scene structure estimates have been obtained using this system from both real and synthetic descent imagery, allowing for the production of accurate DEMs. While some further work would be required in order to produce DEMs that possess the resolution and accuracy needed to determine slopes and the presence of small objects such as rocks at the levels of accuracy required, this thesis presents a very strong foundation upon which to build and goes a long way towards developing a highly robust and accurate solution