Coupling Vanishing Point Tracking with Inertial Navigation to Estimate Attitude in a Structured Environment

This research aims to obtain accurate and stable estimates of a vehicle\u27s attitude by coupling consumer-grade inertial and optical sensors. This goal is pursued by first modeling both inertial and optical sensors and then developing a technique for identifying vanishing points in perspective images of a structured environment. The inertial and optical processes are then coupled to enable each one to aid the other. The vanishing point measurements are combined with the inertial data in an extended Kalman filter to produce overall attitude estimates. This technique is experimentally demonstrated in an indoor corridor setting using a motion profile designed to simulate flight. Through comparison with a tactical-grade inertial sensor, the combined consumer-grade inertial and optical data are shown to produce a stable attitude solution accurate to within 1.5 degrees. A measurement bias is manifested which degrades the accuracy by up to another 2.5 degrees

Optimization of Machine Learning Models with Segmentation to Determine the Pose of Cattle

Image pattern recognition poses numerous challenges, particularly in feature recognition, making it a complex problem for machine learning algorithms. This study focuses on the problem of cow pose detection, involving the classification of cow images into categories like front, right, left, and others. With the increasing popularity of image-based applications, such as object recognition in smartphone technologies, there is a growing need for accurate and efficient classification algorithms based on shape and color. In this paper, we propose a machine learning approach utilizing Support Vector Machine (SVM) and Random Forest (RF) algorithms for cow pose detection. To achieve an optimal model, we employ data augmentation techniques, including Gaussian blur, brightness adjustments, and segmentation. The proposed segmentation methods used are Canny and Kmeans. We compare several machine learning algorithms to identify the optimal approach in terms of accuracy. The success of our method is measured by accuracy and Receiver Operating Characteristic (ROC) analysis. The results indicate that using the Canny segmentation, SVM achieved 74.31% accuracy with a testing ratio of 90:10, while RF achieved 99.60% accuracy with the same testing ratio. Furthermore, testing with SVM and K-means segmentation reached an accuracy of 98.61% with a test ratio of 80:20. The study demonstrates the effectiveness of SVM and Random Forest algorithms in cow pose detection, with Kmeans segmentation yielding highly accurate results. These findings hold promising implications for real-world applications in image-based recognition systems. Based on the results of the model obtained, it is very important in pattern recognition to use segmentation based on color even though shape recognition

Real Time Stereo Cameras System Calibration Tool and Attitude and Pose Computation with Low Cost Cameras

The Engineering in autonomous systems has many strands. The area in which this work falls, the artificial vision, has become one of great interest in multiple contexts and focuses on robotics. This work seeks to address and overcome some real difficulties encountered when developing technologies with artificial vision systems which are, the calibration process and pose computation of robots in real-time. Initially, it aims to perform real-time camera intrinsic (3.2.1) and extrinsic (3.3) stereo camera systems calibration needed to the main goal of this work, the real-time pose (position and orientation) computation of an active coloured target with stereo vision systems. Designed to be intuitive, easy-to-use and able to run under real-time applications, this work was developed for use either with low-cost and easy-to-acquire or more complex and high resolution stereo vision systems in order to compute all the parameters inherent to this same system such as the intrinsic values of each one of the cameras and the extrinsic matrices computation between both cameras. More oriented towards the underwater environments, which are very dynamic and computationally more complex due to its particularities such as light reflections. The available calibration information, whether generated by this tool or loaded configurations from other tools allows, in a simplistic way, to proceed to the calibration of an environment colorspace and the detection parameters of a specific target with active visual markers (4.1.1), useful within unstructured environments. With a calibrated system and environment, it is possible to detect and compute, in real time, the pose of a target of interest. The combination of position and orientation or attitude is referred as the pose of an object. For performance analysis and quality of the information obtained, this tools are compared with others already existent.A engenharia de sistemas autónomos actua em diversas vertentes. Uma delas, a visão artificial, em que este trabalho assenta, tornou-se uma das de maior interesse em múltiplos contextos e focos na robótica. Assim, este trabalho procura abordar e superar algumas dificuldades encontradas aquando do desenvolvimento de tecnologias baseadas na visão artificial. Inicialmente, propõe-se a fornecer ferramentas para realizar as calibrações necessárias de intrínsecos (3.2.1) e extrínsecos (3.3) de sistemas de visão stereo em tempo real para atingir o objectivo principal, uma ferramenta de cálculo da posição e orientação de um alvo activo e colorido através de sistemas de visão stereo. Desenhadas para serem intuitivas, fáceis de utilizar e capazes de operar em tempo real, estas ferramentas foram desenvolvidas tendo em vista a sua integração quer com camaras de baixo custo e aquisição fácil como com camaras mais complexas e de maior resolução. Propõem-se a realizar a calibração dos parâmetros inerentes ao sistema de visão stereo como os intrínsecos de cada uma das camaras e as matrizes de extrínsecos que relacionam ambas as camaras. Este trabalho foi orientado para utilização em meio subaquático onde se presenciam ambientes com elevada dinâmica visual e maior complexidade computacional devido `a suas particularidades como reflexões de luz e má visibilidade. Com a informação de calibração disponível, quer gerada pelas ferramentas fornecidas, quer obtida a partir de outras, pode ser carregada para proceder a uma calibração simplista do espaço de cor e dos parâmetros de deteção de um alvo específico com marcadores ativos coloridos (4.1.1). Estes marcadores são ´uteis em ambientes não estruturados. Para análise da performance e qualidade da informação obtida, as ferramentas de calibração e cálculo de pose (posição e orientação), serão comparadas com outras já existentes

Application of computer vision for roller operation management

Compaction is the last and possibly the most important phase in construction of asphalt concrete (AC) pavements. Compaction densifies the loose (AC) mat, producing a stable surface with low permeability. The process strongly affects the AC performance properties. Too much compaction may cause aggregate degradation and low air void content facilitating bleeding and rutting. On the other hand too little compaction may result in higher air void content facilitating oxidation and water permeability issues, rutting due to further densification by traffic and reduced fatigue life. Therefore, compaction is a critical issue in AC pavement construction.;The common practice for compacting a mat is to establish a roller pattern that determines the number of passes and coverages needed to achieve the desired density. Once the pattern is established, the roller\u27s operator must maintain the roller pattern uniformly over the entire mat.;Despite the importance of uniform compaction to achieve the expected durability and performance of AC pavements, having the roller operator as the only mean to manage the operation can involve human errors.;With the advancement of technology in recent years, the concept of intelligent compaction (IC) was developed to assist the roller operators and improve the construction quality. Commercial IC packages for construction rollers are available from different manufacturers. They can provide precise mapping of a roller\u27s location and provide the roller operator with feedback during the compaction process.;Although, the IC packages are able to track the roller passes with impressive results, there are also major hindrances. The high cost of acquisition and potential negative impact on productivity has inhibited implementation of IC.;This study applied computer vision technology to build a versatile and affordable system to count and map roller passes. An infrared camera is mounted on top of the roller to capture the operator view. Then, in a near real-time process, image features were extracted and tracked to estimate the incremental rotation and translation of the roller. Image featured are categorized into near and distant features based on the user defined horizon. The optical flow is estimated for near features located in the region below the horizon. The change in roller\u27s heading is constantly estimated from the distant features located in the sky region. Using the roller\u27s rotation angle, the incremental translation between two frames will be calculated from the optical flow. The roller\u27s incremental rotation and translation will put together to develop a tracking map.;During system development, it was noted that in environments with thermal uniformity, the background of the IR images exhibit less featured as compared to images captured with optical cameras which are insensitive to temperature. This issue is more significant overnight, since nature elements are not able to reflect the heat energy from sun. Therefore to improve roller\u27s heading estimation where less features are available in the sky region a unique methodology that allows heading detection based on the asphalt mat edges was developed for this research. The heading measurements based on the slope of the asphalt hot edges will be added to the pool of the headings measured from sky region. The median of all heading measurements will be used as the incremental roller\u27s rotation for the tracking analysis.;The record of tracking data is used for QC/QA purposes and verifying the proper implementation of the roller pattern throughout a job constructed under the roller pass specifications.;The system developed during this research was successful in mapping roller location for few projects tested. However the system should be independently validated

Vision Science and Technology at NASA: Results of a Workshop

A broad review is given of vision science and technology within NASA. The subject is defined and its applications in both NASA and the nation at large are noted. A survey of current NASA efforts is given, noting strengths and weaknesses of the NASA program

Thermal Robotic Arm Controlled Spraying via Robotic Arm and Vision System

The Tribology Surface Engineering industry is a worldwide multi billion euro industry with significant health and safety risks. The thermal spraying sector of this industry employs the technique of applying molten surface coating material to a substrate via a thermal spray process which is implemented either by manual spraying or pre-programmed robotic systems. The development of autonomous robotic systems for thermal spraying surface coating would significantly improve production and profitability over pre-programmed systems and improve health and safety over manual spraying. The aim of this research was to investigate and develop through software simulation, physical modelling and testing the development of robotic subsystems that are required to provide autonomous robotic control for the thermal spraying process. Computer based modelling programs were developed to investigate the control strategy identified for the thermal spaying process. The algorithms included fifth order polynomial trajectories and the complete dynamic model where gravitational, inertia, centrifugal and coriolis torques are considered. Tests provide detail of the load torques that must be driven by the robot electric actuator for various structural changes to the thermal spraying robot and for variations in trajectory boundary conditions during thermal spraying. The non-linear and coupled forward and inverse kinematic equations of a five axis articulated robot with continuous rotation joints were developed and tested via computer based modelling and miniature physical robot modelling. Both the computer based modelling and physical model confirmed the closed form kinematic solutions. A solution to running cables through the continuous rotation joints for power and data is present which uses polytetrafloraethylene (PTFE) electroless nickel. This material was identified during the literature review of surface coating materials. It has excellent wear, friction and conductivity properties. Physical tests on a slip ring and brushes test rig using electroless nickel are presented which confirm the viability of using PTFE electroless nickel as a slip ring. Measurement of the substrate during thermal spraying so as to autonomously control the thermal spaying robot is a significant challenge. This research presents solutions for the measurement of the substrate using a low cost camera system and lasers in a single wavelength environment. Tests were carried out which resulted in the removal of a butane flame obscuring a test piece requiring measurement from the camera image so that substrate measurements can be made using image processing and analysis techniques such as canny edge detection and centroid measurements. Test results for the low cost vision system provide depth measure errors of ±0.6 % and structural measurements such as area and perimeter in the range -5% to -7.5%. These results confirm the efficacy of this novel flame removal technique

Machine Learning based Mountainous Skyline Detection and Visual Geo-Localization

With the ubiquitous availability of geo-tagged imagery and increased computational power, geo-localization has captured a lot of attention from researchers in computer vision and image retrieval communities. Significant progress has been made in urban environments with stable man-made structures and geo-referenced street imagery of frequently visited tourist attractions. However, geo-localization of natural/mountain scenes is more challenging due to changed vegetations, lighting, seasonal changes and lack of geo-tagged imagery. Conventional approaches for mountain/natural geo-localization mostly rely on mountain peaks and valley information, visible skylines and ridges etc. Skyline (boundary segmenting sky and non-sky regions) has been established to be a robust natural feature for mountainous images, which can be matched with the synthetic skylines generated from publicly available terrain maps such as Digital Elevation Models (DEMs). Skyline or visible horizon finds further applications in various other contexts e.g. smooth navigation of Unmanned Aerial Vehicles (UAVs)/Micro Aerial Vehicles (MAVs), port security, ship detection and outdoor robot/vehicle localization.\parProminent methods for skyline/horizon detection are based on non-realistic assumptions and rely on mere edge detection and/or linear line fitting using Hough transform. We investigate the use of supervised machine learning for skyline detection. Specifically we propose two novel machine learning based methods, one relying on edge detection and classification while other solely based on classification. Given a query image, an edge or classification map is first built and converted into a multi-stage graph problem. Dynamic programming is then used to find a shortest path which conforms to the detected skyline in the given image. For the first method, we provide a detailed quantitative analysis for various texture features (Scale Invariant Feature Transform (SIFT), Local Binary Patterns (LBP), Histogram of Oriented Gradients (HOG) and their combinations) used to train a Support Vector Machine (SVM) classifier and different choices (binary edges, classified edge score, gradient score and their combinations) for the nodal costs for Dynamic Programming (DP). For the second method, we investigate the use of dense classification maps for horizon line detection. We use Support Vector Machines (SVMs) and Convolutional Neural Networks (CNNs) as our classifier choices and use normalized intensity patches as features. Both proposed formulations are compared with a prominent edge based method on two different data sets.\par We propose a fusion strategy which boosts the performance of the edge-less approach using edge information. The fusion approach, which has been tested on an additional challenging data set, outperforms each of the two methods alone. Further, we demonstrate the capability of our formulations to detect absence of horizon boundary and detection of partial horizon lines. This could be of great value in applications where a confidence measure of the detection is necessary e.g. localization of planetary rovers/robots. In an extended work, we compare our edge-less skyline detection approach against deep learning networks recently proposed for semantic segmentation on an additional data set. Specifically, we compare our proposed fusion formulation with Fully Convolutional Network (FCN), SegNet and another classical supervised learning based method.\par We further propose a visual geo-localization pipeline based on evolutionary computing; where Particle Swarm Optimization (PSO) is adopted to find/refine an orientation estimate by minimizing the cost function based on horizon-ness probability of pixels. The dense classification score image resulting from our edge-less/fusion approach is used as a fitness measure to guide the particles toward best solution where the rendered horizon from DEM perfectly aligns with the actual horizon from the image without even requiring its explicit detection. The effectiveness of the proposed geo-localization pipeline is evaluated on a decent sized data set

Proceedings of the 5th Baltic Mechatronics Symposium - Espoo April 17, 2020

The Baltic Mechatronics Symposium is annual symposium with the objective to provide a forum for young scientists from Baltic countries to exchange knowledge, experience, results and information in large variety of fields in mechatronics. The symposium was organized in co-operation with Taltech and Aalto University. Due to Coronavirus COVID-19 the symposium was organized as a virtual conference. The content of the proceedings1. Monitoring Cleanliness of Public Transportation with Computer Vision2. Device for Bending and Cutting Coaxial Wires for Cryostat in Quantum Computing3. Inertial Measurement Method and Application for Bowling Performance Metrics4. Mechatronics Escape Room5. Hardware-In-the-Loop Test Setup for Tuning Semi-Active Hydraulic Suspension Systems6. Newtonian Telescope Design for Stand-off Laser Induced Breakdown Spectroscopy7. Simulation and Testing of Temperature Behavior in Flat Type Linear Motor Carrier8. Powder Removal Device for Metal Additive Manufacturing9. Self-Leveling Spreader Beam for Adjusting the Orientation of an Overhead Crane Loa

An investigation of the microscale geometry and liquid flow through an isolated foam channel network

Liquid foams represent an extremely diverse and highly functional form of soft matter, whose application is widespread throughout industry. These can range from luxurious and low calorie applications in food and beverages, structural and insulating properties in building and manufacture, right through to dynamic and transport abilities in the petrochemical industry, among others. A common feature of all of these foams is they are required to exhibit longevity, however as foams are thermodynamically unstable systems, this is not always a trivial feat. Foams are highly complex systems, with dynamic processes occurring on the molecular scale that influence properties at the scale of individual bubbles and subsequently at the macroscopic scale of bulk foams. A particular challenge of foam research is to unite these length scale processes, requiring robust theoretical and experimental studies to be made at all size regimes. This PhD thesis is concerned specifically with the microscale process of liquid flow between bubbles, as these liquid channels form the primary network through which liquid ‘drains’ through a foam under the force of gravity; one of the key mechanisms governing foam instability. The initial focus of this PhD thesis was the design and implementation of an experimental technique to isolate and image liquid foam channels formed under controlled liquid flow rates. This was developed with a view to producing highly accurate and reproducible measurements of the channel geometries, which would enable the comparison to theory derived to describe such systems. Measurements of low molecular weight surfactants and higher molecular weight emulsifiers clearly demonstrated three previously unseen geometries of foam channel that could not be described using existing theory. Instead, a new geometric model was developed which was able to account for these differences, relating the bulk and surface properties of the foam channel to its length and the rate of liquid flow passing through it. When used as a fitting parameter, the new model was able to clearly demarcate between the characteristic low and high surface viscosities of the surfactant and emulsifier species respectively. The surface viscosity of the surfactant foam channel interfaces was examined throughout this PhD study, as the values extracted from model fitting were consistently lower than the majority found in literature, but in line with predictions made from hyper-sensitive measurement techniques. Ultimately, it was proposed that these differences could be attributed to a combination of the limited measurement sensitivity of commercial systems, combined with a liquid flow velocity dependence of surfactant concentration at the channel surface. It was suggested that, in the case of low molecular weight surfactants, a surface tension gradient can exist along the length of a foam channel, that is dependent upon the rate of liquid flow, the concentration of surfactant and the rate of surfactant adsorption to the interface. In the case of high liquid flow velocities, it was shown that surface tensions in some channel regions could be almost as high as pure water, despite surfactant concentrations being above the CMC. As such, this could have significant consequences for stability in macroscopic foams where these conditions are present