SPEEDING-UP IMAGE REGISTRATION FOR REPETITIVE MOTION SCENARIOS

ABSTRACT We propose a novel approach for real-time image registration for image sequences of organs subject to repetitive movement, such as breathing. The method exploits the redundancy within the images and consists of a training and an application phase. During training, the images are registered and then the relationship between the image appearance and the spatial transformation is learned by employing dimensionality reduction to the images and storage of the corresponding displacements. For each image in the application phase, the most similar images in the training set are selected for predicting the associated displacements. Registration and update of the training data is only performed for outliers. The method is assessed on 2D sequences (4 MRI, 1 ultrasound) of the liver during free breathing. The performance is evaluated on manually selected landmarks, such as vessel centers and the distal point of the inferior segment. The proposed algorithm is real-time (9 ms per frame) and the prediction error is on average 1.2 mm for both MRI and ultrasound

Efficient generation of occlusion-aware multispectral and thermographic point clouds

The reconstruction of 3D point clouds from image datasets is a time-consuming task that has been frequently solved by performing photogrammetric techniques on every data source. This work presents an approach to efficiently build large and dense point clouds from co-acquired images. In our case study, the sensors coacquire visible as well as thermal and multispectral imagery. Hence, RGB point clouds are reconstructed with traditional methods, whereas the rest of the data sources with lower resolution and less identifiable features are projected into the first one, i.e., the most complete and dense. To this end, the mapping process is accelerated using the Graphics Processing Unit (GPU) and multi-threading in the CPU (Central Processing Unit). The accurate colour aggregation in 3D points is guaranteed by taking into account the occlusion of foreground surfaces. Accordingly, our solution is shown to reconstruct much more dense point clouds than notable commercial software (286% on average), e.g., Pix4Dmapper and Agisoft Metashape, in much less time (−70% on average with respect to the best alternative).Spanish Ministry of Science, Innovation and Universities via a doctoral grant to the first author (FPU19/00100)Project TED2021- 132120B-I00 funded by MCIN/AEI/10.13039/501100011033/ and ERDF funds ‘‘A way of doing Europe’

Improved Visible Light Communication Receiver Performance by Leveraging the Spatial Dimension

In wireless communications systems, signals can be transmitted as time (temporal) or spatial variants across 3D space, and in both ways. However, using temporal variant communication channels in high-speed data transmission introduces inter-symbol interference (ISI) which makes the systems unreliable. On the other hand, spatial diversity in signal processing reduces the ISI and improves the system throughput or performance by allowing more signals from different spatial locations at the same time. Therefore, the spatial features or properties of visible light signals can be very useful in designing a reliable visible light communication (VLC) system with higher system throughput and making it more robust against ambient noise and interference. By allowing only the signals of interest, spatial separability in VLC can minimize the noise to a greater extent to improve signal-to-noise ratio (SNR) which can ensure higher data rates (in the order of Gbps-Tbps) in VLC. So, designing a VLC system with spatial diversity is an exciting area to explore and might set the foundation for future VLC system architectures and enable different VLC based applications such as vehicular VLC, multi-VLC, localization, and detection using VLC, etc. This thesis work is motivated by the fundamental challenges in reusing spatial information in VLC systems to increase the system throughput or gain through novel system designing and their prototype implementations

TARGETLESS REGISTRATION METHODS BETWEEN UAV LIDAR AND WEARABLE MMS POINT CLOUDS

Fixed-wing Unmanned Aerial Vehicles (UAV) and wearable or portable Mobile Mapping Systems (MMS) are two widely used platforms for point cloud acquisition with Light Detection And Ranging (LiDAR) sensors. The two platforms acquire from distant viewpoints and produce complementary point clouds, one describing predominantly horizontal surfaces and the other primarily vertical. Thus, the registration of the two data is not straightforward. This paper presents a test of targetless registration between a UAV LiDAR point cloud and terrestrial MMS surveys. The case study is a vegetated hilly landscape characterized by the presence of a structure of interest; the UAV acquisition allows the entire area to be acquired from above, while the terrestrial MMS acquisitions will enable the construction of interest to be detailed. The paper describes the survey phase with both techniques. It focuses on processing and registration strategies to fuse the two data together. Our approach is based on the ICP (Iterative Closest Point) method by exploiting the data processing algorithms available in the Heron Desktop post-processing software for handling data acquired with the Heron Backpack MMS instrument. Two co-registration methods are compared. Both ways use the UAV point cloud as a reference and derive the registration of the terrestrial MMS data by finding ICP matches between the ground acquisition and the reference cloud exploiting only a few areas of overlap. The two methods are detailed in the paper, and both allow us to complete the co-registration task

Cloud-Induced Uncertainty for Visual Navigation

This research addresses the numerical distortion of features due to the presence of clouds in an image. The research aims to quantify the probability of a mismatch between two features in a single image, which will describe the likelihood that a visual navigation system incorrectly tracks a feature throughout an image sequence, leading to position miscalculations. First, an algorithm is developed for calculating transparency of clouds in images at the pixel level. The algorithm determines transparency based on the distance between each pixel color and the average pixel color of the clouds. The algorithm is used to create a dataset of cloudy aerial images. Matching features are then detected between the original and cloudy images, which allows a direct comparison between features with and without clouds. The transparency values are used to segment the detected features into three categories, based on whether the features are located in the regions without clouds, along edges of clouds, or with clouds. The error between features on the cloudy and cloud-free images is determined, and used as a basis for generating a synthetic dataset with statistically similar properties. Lastly, Monte Carlo techniques are used to find the probability of mismatching

An Open System for Collection and Automatic Recognition of Pottery through Neural Network Algorithms

In the last ten years, artificial intelligence (AI) techniques have been applied in archaeology. The ArchAIDE project realised an AI-based application to recognise archaeological pottery. Pottery is of paramount importance for understanding archaeological contexts. However, recognition of ceramics is still a manual, time-consuming activity, reliant on analogue catalogues. The project developed two complementary machine-learning tools to propose identifications based on images captured on-site, for optimising and economising this process, while retaining key decision points necessary to create trusted results. One method relies on the shape of a potsherd; the other is based on decorative features. For the shape-based recognition, a novel deep-learning architecture was employed, integrating shape information from points along the inner and outer profile of a sherd. The decoration classifier is based on relatively standard architectures used in image recognition. In both cases, training the algorithms meant facing challenges related to real-world archaeological data: the scarcity of labelled data; extreme imbalance between instances of different categories; and the need to take note of minute differentiating features. Finally, the creation of a desktop and mobile application that integrates the AI classifiers provides an easy-to-use interface for pottery classification and storing pottery data

Dexterous Grasping by Manipulability Selection for Mobile Manipulator with Visual Guidance

Industry 4.0 demands the heavy usage of robotic mobile manipulators with high autonomy and intelligence. The goal is to accomplish dexterous manipulation tasks without prior knowledge of the object status in unstructured environments. It is important for the mobile manipulator to recognize and detect the objects, determine manipulation pose, and adjust its pose in the workspace fast and accurately. In this research, we developed a stereo vision algorithm for the object pose estimation using point cloud data from multiple stereo vision systems. An improved iterative closest point algorithm method is developed for the pose estimation. With the pose input, algorithms and several criteria are studied for the robot to select and adjust its pose by maximizing its manipulability on a given manipulation task. The performance of each technical module and the complete robotic system is finally shown by the virtual robot in the simulator and real robot in experiments. This study demonstrates a setup of autonomous mobile manipulator for various flexible manufacturing and logistical scenarios

Visual SLAM with RGB-D cameras based on pose graph optimization

En este trabajo abordamos el problema de localización y mapeo simultáneo (SLAM) utilizando únicamente información obtenida mediante una cámara RGB-D. El objetivo principal es desarrollar un sistema SLAM capaz de estimar la trayectoria completa del sensor y generar una representación 3D consistente del entorno en tiempo real. Para lograr este objetivo, el sistema se basa en un método de estimación del movimiento del sensor a partir de información de profundidad densa y en técnicas de reconocimiento de lugares a partir de características visuales. A partir de estos algoritmos, se extraen restricciones espaciales entre fotogramas cuidadosamente seleccionados. Con estas restricciones espaciales se construye un grafo de poses, empleado para inferir la trayectoria más verosímil. El sistema se ha diseñado para ejecutarse en dos hilos paralelos: uno para el seguimiento y el otro para la construcción de la representación consistente. El sistema se evalúa en conjuntos de datos públicamente accesible, alcanzando una precisión comparable a sistemas de SLAM del estado del arte. Además, el hilo de seguimiento se ejecuta a una frecuencia de 60 Hz en un ordenador portátil de prestaciones modestas. También se realizan pruebas en situaciones más realistas, procesando observaciones adquiridas mientras se movía el sensor por dos entornos de interiores distintos