3D Reconstruction with Uncalibrated Cameras Using the Six-Line Conic Variety

We present new algorithms for the recovery of the Euclidean structure from a projective calibration of a set of cameras with square pixels but otherwise arbitrarily varying intrinsic and extrinsic parameters. Our results, based on a novel geometric approach, include a closed-form solution for the case of three cameras and two known vanishing points and an efficient one-dimensional search algorithm for the case of four cameras and one known vanishing point. In addition, an algorithm for a reliable automatic detection of vanishing points on the images is presented. These techniques fit in a 3D reconstruction scheme oriented to urban scenes reconstruction. The satisfactory performance of the techniques is demonstrated with tests on synthetic and real data

Robotic Cameraman for Augmented Reality based Broadcast and Demonstration

In recent years, a number of large enterprises have gradually begun to use vari-ous Augmented Reality technologies to prominently improve the audiences’ view oftheir products. Among them, the creation of an immersive virtual interactive scenethrough the projection has received extensive attention, and this technique refers toprojection SAR, which is short for projection spatial augmented reality. However,as the existing projection-SAR systems have immobility and limited working range,they have a huge difficulty to be accepted and used in human daily life. Therefore,this thesis research has proposed a technically feasible optimization scheme so thatit can be practically applied to AR broadcasting and demonstrations. Based on three main techniques required by state-of-art projection SAR applica-tions, this thesis has created a novel mobile projection SAR cameraman for ARbroadcasting and demonstration. Firstly, by combining the CNN scene parsingmodel and multiple contour extractors, the proposed contour extraction pipelinecan always detect the optimal contour information in non-HD or blurred images.This algorithm reduces the dependency on high quality visual sensors and solves theproblems of low contour extraction accuracy in motion blurred images. Secondly, aplane-based visual mapping algorithm is introduced to solve the difficulties of visualmapping in these low-texture scenarios. Finally, a complete process of designing theprojection SAR cameraman robot is introduced. This part has solved three mainproblems in mobile projection-SAR applications: (i) a new method for marking con-tour on projection model is proposed to replace the model rendering process. Bycombining contour features and geometric features, users can identify objects oncolourless model easily. (ii) a camera initial pose estimation method is developedbased on visual tracking algorithms, which can register the start pose of robot to thewhole scene in Unity3D. (iii) a novel data transmission approach is introduced to establishes a link between external robot and the robot in Unity3D simulation work-space. This makes the robotic cameraman can simulate its trajectory in Unity3D simulation work-space and project correct virtual content. Our proposed mobile projection SAR system has made outstanding contributionsto the academic value and practicality of the existing projection SAR technique. Itfirstly solves the problem of limited working range. When the system is running ina large indoor scene, it can follow the user and project dynamic interactive virtualcontent automatically instead of increasing the number of visual sensors. Then,it creates a more immersive experience for audience since it supports the user hasmore body gestures and richer virtual-real interactive plays. Lastly, a mobile systemdoes not require up-front frameworks and cheaper and has provided the public aninnovative choice for indoor broadcasting and exhibitions

Searches for continuous gravitational waves : sensitivity estimation and deep learning as a novel search method

The first direct detections of gravitational waves from merging black holes and neutron stars started the era of gravitational-wave astronomy. Since then, observing merging compact objects has become routine. Other exciting sources still remain undetected. Rapidly-rotating neutron stars are predicted to emit weak, long-lasting quasi-monochromatic waves called continuous gravitational waves (CWs). In the current detector generation, advanced LIGO and Virgo, various noise sources create far more signal output than a potential CW signal. CW data analysis tries to overcome the weakness of the signals by integrating over long stretches of data. Analyzing large amounts of data usually corresponds to large computing cost. For that reason, CW searches for signals from unknown neutron stars are limited in their sensitivity by computational cost. This thesis is concerned with estimating and improving the sensitivity of continuous gravita- tional wave searches. The first main research work presented in this thesis is a new sensitivity estimator that can swiftly and accurately predict the sensitivity of a CW search before it is started. This makes optimizing the search algorithms and therefore improving the sensitivity easier. The accuracy of the estimator is studied by applying it to many different CW searches. The work is expanded with an extensive sensitivity review of past CW searches by calculating their sensitivity depth. The second main part of this thesis is the development of a new CW search method based on deep neural networks (DNNs). DNNs are extremely fast once trained and therefore might present an interesting possibility of circumventing the computational limitations and creating a more sensitive CW search. In this thesis such a DNN CW search is developed first as a single-detector search for signals from all over the sky and then expanded to a multi-detector all-sky search and to directed multi-detector searches for signals from a single position in the sky. The DNNs’ performance is compared to coherent matched-filtering searches in terms of detection probability at fixed false-alarm level first on idealized Gaussian noise and then on realistic LIGO detector noise. This thesis finds that the DNNs show a lot of potential: For short timespans of about one day the networks only lose a few percent in sensitivity depth compared to coherent matched- filtering. For longer timespans the networks’ performance gradually deteriorates making further research necessary. As an outlook to future research, this thesis proposes the combination of short-timespan network outputs, similar to semi-coherent matched-filtering, as a DNN search method over longer timespans.Die ersten direkten Detektionen von Gravitationswellen von verschmelzenden Schwarzen Löchern und Neutronensternen haben die Ära der Gravitationswellenastronomie eingeläutet. Seitdem ist die Beobachtung von verschmelzenden kompakten Objekten zur Routine geworden. Andere interessante Quellen von Gravitationswellen sind jedoch noch unentdeckt. Schnell rotierende Neutronensterne können schwache, langanhaltende quasi-monochromatische Wellen aussenden, genannt Kontinuierliche Gravitationswellen (engl.: continuous waves CWs). In der aktuellen Detektorgeneration, advanced LIGO und Virgo, wird mehr Detektoroutput durch diverse Rauschquellen erzeugt als durch potenzielle CW Signale. Die Datenanalyse für CWs versucht die Schwäche der Signale zu überwinden, indem die Daten über lange Zeitspan- nen integriert werden. Große Mengen von Daten zu analysieren ist jedoch für gewöhnlich mit großen Ansprüchen an die Rechenleistung verbunden. Deshalb sind Suchen nach CWs von un- bekannten Neutronensternen in ihrer Empfindlichkeit limitiert durch die begrenzt vorhandene Rechenleistung. Diese Doktorarbeit beschäftigt sich mit dem Abschätzen und Verbessern der Empfindlichkeit von Suchen nach Kontinuierlichen Gravitationswellen. Das erste Hauptforschungsergebnis dieser Arbeit ist ein neuartiger Abschätzer, der die Empfindlichkeit einer CW-Suche schnell und genau vorhersagen kann bevor die Suche gestartet wird. Dies vereinfacht die Verbesserung der Suchal- gorithmen und kann deshalb zu empfindlicheren Suchen führen. Die Genauigkeit des Abschätzers wird anhand von vielen verschiedenen CW-Suchen untersucht. Die Untersuchung wird ergänzt durch eine ausführliche Studie der Empfindlichkeit von vergangenen CW-Suchen. Dazu wird deren Empfindlichkeit in die gemeinsame Größe der Empfindlichkeitstiefe (engl.: sensitivity depth) umgerechnet. Das zweite Hauptforschungsergebnis dieser Dissertation ist eine neuartige CW-Suchmethode mit Hilfe von tiefen neuronalen Netzwerken (engl.: deep neural networks, DNNs). Fertig trainierte DNNs können extrem schnell angewendet werden und stellen deshalb eine interes- sante Art und Weise dar, wie möglicherweise mit der Limitierung durch fehlende Rechenleistung umgegangen und eine empfindlichere Suche konstruiert werden kann. In dieser Arbeit wird eine solche DNN nutzende CW-Suchmethode präsentiert: zuerst als Suche mit Daten von einem einzigen Detektor nach Signalen vom gesamten Himmel und dann erweitert zu Suchen mit Daten von mehreren Detektoren nach Signalen vom gesamten Himmel oder nach Signalen von einer speziellen Himmelsposition. Die Leistungsfähigkeit der DNNs wird dabei verglichen mit kohärenten Optimalfiltermethoden im Hinblick auf ihre Detektionswahrscheinlichkeit bei festem Fehlalarmniveau. Diese Arbeit zeigt das diesbezüglich große Potential von DNNs: Bei der Analyse von kurzen Zeitspannen von etwa einem Tag verliert das Netwerk nur wenige Prozent in Empfindlichkeitstiefe gegenüber kohärenten Optimalfiltermethoden. Für längere Zeitspannen nimmt die Leistungsfähigkeit der Netzwerke im Vergleich jedoch nach und nach ab. An dieser Stelle wird deshalb weitere Forschungsarbeit benötigt um die Leistungsfähigkeit der DNNs zu verbessern. Ein Ansatz, der in dieser Arbeit für zukünftige Forschung vorgeschlagen wird, ist die Kombination von Ergebnissen, die Netzwerke auf kurzen Zeitspannen erreicht haben, als Ergebnis für längere Zeitspannen zu nutzen. Dieser Ansatz ist ähnlich zum semi-kohärenten Optimalfilter, der in klassischen CW-Suchen benutzt wird

Airborne vision-based attitude estimation and localisation

Vision plays an integral part in a pilot's ability to navigate and control an aircraft. Therefore Visual Flight Rules have been developed around the pilot's ability to see the environment outside of the cockpit in order to control the attitude of the aircraft, to navigate and to avoid obstacles. The automation of these processes using a vision system could greatly increase the reliability and autonomy of unmanned aircraft and flight automation systems. This thesis investigates the development and implementation of a robust vision system which fuses inertial information with visual information in a probabilistic framework with the aim of aircraft navigation. The horizon appearance is a strong visual indicator of the attitude of the aircraft. This leads to the first research area of this thesis, visual horizon attitude determination. An image processing method was developed to provide high performance horizon detection and extraction from camera imagery. A number of horizon models were developed to link the detected horizon to the attitude of the aircraft with varying degrees of accuracy. The second area investigated in this thesis was visual localisation of the aircraft. A terrain-aided horizon model was developed to estimate the position, altitude as well as attitude of the aircraft. This gives rough positions estimates with highly accurate attitude information. The visual localisation accuracy was improved by incorporating ground feature-based map-aided navigation. Road intersections were detected using a developed image processing algorithm and then they were matched to a database to provide positional information. The developed vision system show comparable performance to other non-vision-based systems while removing the dependence on external systems for navigation. The vision system and techniques developed in this thesis helps to increase the autonomy of unmanned aircraft and flight automation systems for manned flight

Automatic assessment of honey bee cells using deep learning

Temporal assessment of honey bee colony strength is required for different applications in many research projects, which often involves counting the number of comb cells with brood and food reserves multiple times a year. There are thousands of cells in each comb, which makes manual counting a time-consuming, tedious and thereby an error-prone task. Therefore, the automation of this task using modern imaging processing techniques represents a major advance. Herein, we developed a software capable of (i) detecting each cell from comb images, (ii) classifying its content and (iii) display the results to the researcher in a simple way. The cells’ contents typically display a high variation of patterns which make their classification by software a challenging endeavour. To address this challenge, we used Deep Neural Networks (DNNs). DNNs are known for achieving the state of art in many fields of study including image classification, because they can learn features that best describe the content being classified by themselves. Our DNN model was trained with over 70,000 manually labelled cell images whose cells were separated into seven classes. Our contribution is an end-to-end software capable of doing automatic background removal, cell detection, and classification of cell content based on an input comb image. With this software, colony assessment achieves an average accuracy of 94% across the seven classes in our dataset, representing a substantial progress regarding the approximation methods (e.g. Lieberfeld) currently used by honey bee researchers and previous techniques based on machine learning that used handmade features like colour and texture.A análise temporal sobre a qualidade e força de colônias de abelha melífera (Apis mellifera L.) é necessária em muitos projetos de pesquisa. Ela pode ser realizada contando alvéolos com alimento (pólen e néctar) e criação. É comum que ela seja feita diversas vezes ao ano. A grande quantidade de alvéolos em cada favo torna a tarefa demorada e tediosa ao pesquisador. Assim, frequentemente essa contagem é feita forma aproximada usando métodos como o de Lieberfeld. Automatizar este processo usando técnicas modernas de processamento de imagem representa um grande avanço, pois resultados mais precisos e padronizados poderão ser obtidos em menos tempo. O objetivo deste trabalho é construir de um software capaz de detectar, classificar e contar alvéolos a partir de uma imagem. Após, ele deverá apresentar os dados de forma simplificada ao usuário. Para tratar da alta variação de padrões como textura, cor e iluminação presente nas alvéolos, usaremos Deep Neural Network (DNN), que são modelos computacionais conhecidos por terem alcançado o estado da arte em várias tarefas relacionadas a processamento de sinais e imagens. Para o treinamento desses modelos utilizamos mais de 70.000 alvéolos anotadas por um apicultor experiente, separadas em sete classes. Entre nossas contribuições estão métodos de préprocessamento que garantem uma alta taxa de detecção de alvéolos, aliados a modelos de segmentação baseados em DNNs que asseguram uma baixa taxa de falsos positivos. Com nossos classificadores conseguimos uma acurácia média de 94% em nosso dataset e obtivemos resultados superiores a outros métodos baseados em contagens aproximadas e técnicas de análise por imagem que não utilizam DNNs.This research was conducted in the framework of the project BEEHOPE, funded through the 2013-2014 BiodivERsA/FACCE-JPI Joint call for research proposals, with the national founders FCT(Portugal), CNRS(France), and MEC(Spain)

Visual localisation of electricity pylons for power line inspection

Inspection of power infrastructure is a regular maintenance event. To date the inspection process has mostly been done manually, but there is growing interest in automating the process. The automation of the inspection process will require an accurate means for the localisation of the power infrastructure components. In this research, we studied the visual localisation of a pylon. The pylon is the most prominent component of the power infrastructure and can provide a context for the inspection of the other components. Point-based descriptors tend to perform poorly on texture less objects such as pylons, therefore we explored the localisation using convolutional neural networks and geometric constraints. The crossings of the pylon, or vertices, are salient points on the pylon. These vertices aid with recognition and pose estimation of the pylon. We were successfully able to use a convolutional neural network for the detection of the vertices. A model-based technique, geometric hashing, was used to establish the correspondence between the stored pylon model and the scene object. We showed the effectiveness of the method as a voting technique to determine the pose estimation from a single image. In a localisation framework, the method serves as the initialization of the tracking process. We were able to incorporate an extended Kalman filter for subsequent incremental tracking of the camera relative to the pylon. Also, we demonstrated an alternative tracking using heatmap details from the vertex detection. We successfully demonstrated the proposed algorithms and evaluated their effectiveness using a model pylon we built in the laboratory. Furthermore, we revalidated the results on a real-world outdoor electricity pylon. Our experiments illustrate that model-based techniques can be deployed as part of the navigation aspect of a robot

Quantitative electron microscopy for microstructural characterisation

Development of materials for high-performance applications requires accurate and useful analysis tools. In parallel with advances in electron microscopy hardware, we require analysis approaches to better understand microstructural behaviour. Such improvements in characterisation capability permit informed alloy design. New approaches to the characterisation of metallic materials are presented, primarily using signals collected from electron microscopy experiments. Electron backscatter diffraction is regularly used to investigate crystallography in the scanning electron microscope, and combined with energy-dispersive X-ray spectroscopy to simultaneusly investigate chemistry. New algorithms and analysis pipelines are developed to permit accurate and routine microstructural evaluation, leveraging a variety of machine learning approaches. This thesis investigates the structure and behaviour of Co/Ni-base superalloys, derived from V208C. Use of the presently developed techniques permits informed development of a new generation of advanced gas turbine engine materials.Open Acces