A Computer Vision System for the Automatic Inspection of Geometric Distortions in Television Displays

The ability to automatically measure the image quality of a television display is a valuable resource. In display manufacturing, automatic inspection enables automatic television alignment, which reduces manufacturing costs and improves product quality. Automatic inspection also comes in handy during competitive analysis and engineering review. Primarily though, commercial inspection systems are built and used for manufacturing. In the past two decades, the advent of microcomputers has made automatic inspection feasible through the use of computer vision. Most of the approaches developed can be divided into two groups: fixtured systems, with fixed or movable cameras, and position independent systems, which can use one or more cameras. A fixtured system involves placing the television-under-test in a rig with attached cameras. The cameras are either fixed or moved robotically. On the other hand, a position independent system allows one or more cameras to be placed anywhere as long as the cameras can view the television’s image. This thesis describes the development of a position-independent, two-camera television inspection system. Chapter One defines the problem and gives an overview of existing systems. Chapter Two covers camera calibration, the mathematical modeling of the way a camera forms an image. Camera calibration makes it possible to use off-the-shelf cameras as measurement sensors. Chapter Three discusses how to take stereo measurements from a pair of cameras. Stereo measurements taken from two or more cameras result in the determination of three-dimensional positions. Finally, Chapter Four mentions some results taken with the developed inspection system

Automated computer vision system for real-time drilling cuttings monitoring

In rotary drilling operations, cuttings are continuously transported to the surface by drilling fluid. Real-time monitoring of cuttings and cavings is crucial for early detection and remediation of drilling problems such as stuck pipe, lost circulation, high torque and drag, reduction in rate of penetration, and other wellbore instability issues. These incidents are large contributors to drilling-related Non-Productive Time (NPT). At the current stage, a mud logger performs monitoring manually. This work proposes to use computer vision techniques to automate this procedure. To achieve this application, specific requirements should be established to design an automated machine vision system to maintain drilling safety and speed. Cuttings ramp has been identified as an ideal location to perform the measurement, where cuttings and caving are sliding down a slope at a steady speed. To accomplish this task, an intelligent image processing system must be able to track cuttings speed, measure volume, analyze size, and generate a surface model. Through a detailed review and testing of available 3D sensing techniques, a system consisting of a 2D high-resolution camera and 3D laser profile scanner was designed. By implementing image processing techniques, the cuttings speed on the ramp was estimated which was then synchronized to the 3D depth data from a laser scanner. Finally, the volume of moving cuttings was estimated and a 3D surface profile was reconstructed using point cloud data. Experimental results in the lab environment validated that such a system can be applied to quantify cuttings volume, size distribution, and reconstruct a 3D profile of cuttings and cavings. This measured result can be stored for further analysis. Overall, this work established a foundation for the design of a sophisticated real-time monitoring system for hole cleaning and wellbore risk reduction.Petroleum and Geosystems Engineerin

A cost-effective, mobile platform-based, photogrammetric approach for continuous structural deformation monitoring

PhD ThesisWith the evolution of construction techniques and materials technology, the design of modern civil engineering infrastructure has become increasingly advanced and complex. In parallel to this, the development and application of appropriate and efficient monitoring technologies has become essential. Improvement in the performance of structural monitoring systems, reduction of labour and total implementation costs have therefore become important issues that scientists and engineers are committed to solving. In this research, a non-intrusive structural monitoring system was developed based on close-range photogrammetric principles. This research aimed to combine the merits of photogrammetry and latest mobile phone technology to propose a cost-effective, compact (portable) and precise solution for structural monitoring applications. By combining the use of low-cost imaging devices (two or more mobile phone handsets) with in-house control software, a monitoring project can be undertaken within a relatively low budget when compared to conventional methods. The system uses programmable smart phones (Google Android v.2.2 OS) to replace conventional in-situ photogrammetric imaging stations. The developed software suite is able to control multiple handsets to continuously capture high-quality, synchronized image sequences for short or long-term structural monitoring purposes. The operations are fully automatic and the system can be remotely controlled, exempting the operator from having to attend the site, and thus saving considerable labour expense in long-term monitoring tasks. In order to prevent the system from crashing during a long-term monitoring scheme, an automatic system state monitoring program and a system recovery module were developed to enhance the stability. In considering that the image resolution for current mobile phone cameras is relatively low (in comparison to contemporary digital SLR cameras), a target detection algorithm was developed for the mobile platform that, when combined with dedicated target patterns, was found to improve the quality of photogrammetric target measurement. Comparing the photogrammetric results with physical measurements, which were measured using a Zeiss P3 analytical plotter, the returned accuracy achieved was 1/67,000. The feasibility of the system has been proven through the implementation of an indoor simulation test and an outdoor experiment. In terms of using this system for actual structural monitoring applications, the optimal relative accuracy of distance measurement was determined to be approximately 1/28,000 under laboratory conditions, and the outdoor experiment returned a relative accuracy of approximately 1/16,400

A Review of Sensor Technologies for Perception in Automated Driving

After more than 20 years of research, ADAS are common in modern vehicles available in the market. Automated Driving systems, still in research phase and limited in their capabilities, are starting early commercial tests in public roads. These systems rely on the information provided by on-board sensors, which allow to describe the state of the vehicle, its environment and other actors. Selection and arrangement of sensors represent a key factor in the design of the system. This survey reviews existing, novel and upcoming sensor technologies, applied to common perception tasks for ADAS and Automated Driving. They are put in context making a historical review of the most relevant demonstrations on Automated Driving, focused on their sensing setup. Finally, the article presents a snapshot of the future challenges for sensing technologies and perception, finishing with an overview of the commercial initiatives and manufacturers alliances that will show future market trends in sensors technologies for Automated Vehicles.This work has been partly supported by ECSEL Project ENABLE- S3 (with grant agreement number 692455-2), by the Spanish Government through CICYT projects (TRA2015- 63708-R and TRA2016-78886-C3-1-R)

Optical measurement of shape and deformation fields on challenging surfaces

A multiple-sensor optical shape measurement system (SMS) based on the principle of white-light fringe projection has been developed and commercialised by Loughborough University and Phase Vision Ltd for over 10 years. The use of the temporal phase unwrapping technique allows precise and dense shape measurements of complex surfaces; and the photogrammetry-based calibration technique offers the ability to calibrate multiple sensors simultaneously in order to achieve 360° measurement coverage. Nevertheless, to enhance the applicability of the SMS in industrial environments, further developments are needed (i) to improve the calibration speed for quicker deployment, (ii) to broaden the application range from shape measurement to deformation field measurement, and (iii) to tackle practically-challenging surfaces of which specular components may disrupt the acquired data and result in spurious measurements. The calibration process typically requires manual positioning of an artefact (i.e., reference object) at many locations within the view of the sensors. This is not only timeconsuming but also complicated for an operator with average knowledge of metrology. This thesis introduces an automated artefact positioning system which enables automatic and optimised distribution of the artefacts, automatic prediction of their whereabouts to increase the artefact detection speed and robustness, and thereby greater overall calibration performance. This thesis also describes a novel technique that integrates the digital image correlation (DIC) technique into the present fringe projection SMS for the purpose of simultaneous shape and deformation field measurement. This combined technique offers three key advantages: (a) the ability to deal with geometrical discontinuities which are commonly present on mechanical surfaces and currently challenging to most deformation measurement methods, (b) the ability to measure 3D displacement fields with a basic single-camera single-projector SMS with no additional hardware components, and (c) the simple implementation on a multiple-sensor hardware platform to achieve complete coverage of large-scale and complex samples, with the resulting displacement fields automatically lying in a single global coordinate system. A displacement measurement iii accuracy of ≅1/12,000 of the measurement volume, which is comparable to that of an industry-standard DIC system, has been achieved. The applications of this novel technique to several structural tests of aircraft wing panels on-site at the research centre of Airbus UK in Filton are also presented. Mechanical components with shiny surface finish and complex geometry may introduce another challenge to present fringe projection techniques. In certain circumstances, multiple reflections of the projected fringes on an object surface may cause ambiguity in the phase estimation process and result in incorrect coordinate measurements. This thesis presents a new technique which adopts a Fourier domain ranging (FDR) method to correctly identifying multiple phase signals and enables unambiguous triangulation for a measured coordinate. Experiments of the new FDR technique on various types of surfaces have shown promising results as compared to the traditional phase unwrapping techniques

An advanced unmanned aerial vehicle (UAV) approach via learning-based control for overhead power line monitoring: a comprehensive review

Detection and prevention of faults in overhead electric lines is critical for the reliability and availability of electricity supply. The disadvantages of conventional methods range from cumbersome installations to costly maintenance and from lack of adaptability to hazards for human operators. Thus, transmission inspections based on unmanned aerial vehicles (UAV) have been attracting the attention of researchers since their inception. This article provides a comprehensive review for the development of UAV technologies in the overhead electric power lines patrol process for monitoring and identifying faults, explores its advantages, and realizes the potential of the aforementioned method and how it can be exploited to avoid obstacles, especially when compared with the state-of-the-art mechanical methods. The review focuses on the development of advanced Learning Control strategies for higher manoeuvrability of the quadrotor. It also explores suitable recharging strategies and motor control for improved mission autonomy

Design and management of image processing pipelines within CPS: Acquired experience towards the end of the FitOptiVis ECSEL Project

Cyber-Physical Systems (CPSs) are dynamic and reactive systems interacting with processes, environment and, sometimes, humans. They are often distributed with sensors and actuators, characterized for being smart, adaptive, predictive and react in real-time. Indeed, image- and video-processing pipelines are a prime source for environmental information for systems allowing them to take better decisions according to what they see. Therefore, in FitOptiVis, we are developing novel methods and tools to integrate complex image- and video-processing pipelines. FitOptiVis aims to deliver a reference architecture for describing and optimizing quality and resource management for imaging and video pipelines in CPSs both at design- and run-time. The architecture is concretized in low-power, high-performance, smart components, and in methods and tools for combined design-time and run-time multi-objective optimization and adaptation within system and environment constraints

Virtual Reality

At present, the virtual reality has impact on information organization and management and even changes design principle of information systems, which will make it adapt to application requirements. The book aims to provide a broader perspective of virtual reality on development and application. First part of the book is named as "virtual reality visualization and vision" and includes new developments in virtual reality visualization of 3D scenarios, virtual reality and vision, high fidelity immersive virtual reality included tracking, rendering and display subsystems. The second part named as "virtual reality in robot technology" brings forth applications of virtual reality in remote rehabilitation robot-based rehabilitation evaluation method and multi-legged robot adaptive walking in unstructured terrains. The third part, named as "industrial and construction applications" is about the product design, space industry, building information modeling, construction and maintenance by virtual reality, and so on. And the last part, which is named as "culture and life of human" describes applications of culture life and multimedia-technology

A Versatile Sensor Data Processing Framework for Resource Technology

Die Erweiterung experimenteller Infrastrukturen um neuartige Sensor eröffnen die Möglichkeit, qualitativ neuartige Erkenntnisse zu gewinnen. Um diese Informationen vollständig zu erschließen ist ein Abdecken der gesamten Verarbeitungskette von der Datenauslese bis zu anwendungsbezogenen Auswertung erforderlich. Eine Erweiterung bestehender wissenschaftlicher Instrumente beinhaltet die strukturelle und zeitbezogene Integration der neuen Sensordaten in das Bestandssystem. Das hier vorgestellte Framework bietet durch seinen flexiblen Ansatz das Potenzial, unterschiedliche Sensortypen in unterschiedliche, leistungsfähige Plattformen zu integrieren. Zwei unterschiedliche Integrationsansätze zeigen die Flexibilität dieses Ansatzes, wobei einer auf die Steigerung der Sensitivität einer Anlage zur Sekundärionenmassenspektroskopie und der andere auf die Bereitstellung eines Prototypen zur Untersuchung von Rezyklaten ausgerichtet ist. Die sehr unterschiedlichen Hardwarevoraussetzungen und Anforderungen der Anwendung bildeten die Basis zur Entwicklung eines flexiblen Softwareframeworks. Um komplexe und leistungsfähige Applikationsbausteine bereitzustellen wurde eine Softwaretechnologie entwickelt, die modulare Pipelinestrukturen mit Sensor- und Ausgabeschnittstellen sowie einer Wissensbasis mit entsprechenden Konfigurations- und Verarbeitungsmodulen kombiniert.:1. Introduction 2. Hardware Architecture and Application Background 3. Software Concept 4. Experimental Results 5. Conclusion and OutlookNovel sensors with the ability to collect qualitatively new information offer the potential to improve experimental infrastructure and methods in the field of research technology. In order to get full access to this information, the entire range from detector readout data transfer over proper data and knowledge models up to complex application functions has to be covered. The extension of existing scientific instruments comprises the integration of diverse sensor information into existing hardware, based on the expansion of pivotal event schemes and data models. Due to its flexible approach, the proposed framework has the potential to integrate additional sensor types and offers migration capabilities to high-performance computing platforms. Two different implementation setups prove the flexibility of this approach, one extending the material analyzing capabilities of a secondary ion mass spectrometry device, the other implementing a functional prototype setup for the online analysis of recyclate. Both setups can be regarded as two complementary parts of a highly topical and ground-breaking unique scientific application field. The requirements and possibilities resulting from different hardware concepts on one hand and diverse application fields on the other hand are the basis for the development of a versatile software framework. In order to support complex and efficient application functions under heterogeneous and flexible technical conditions, a software technology is proposed that offers modular processing pipeline structures with internal and external data interfaces backed by a knowledge base with respective configuration and conclusion mechanisms.:1. Introduction 2. Hardware Architecture and Application Background 3. Software Concept 4. Experimental Results 5. Conclusion and Outloo