A New Vehicle Localization Scheme Based on Combined Optical Camera Communication and Photogrammetry

The demand for autonomous vehicles is increasing gradually owing to their enormous potential benefits. However, several challenges, such as vehicle localization, are involved in the development of autonomous vehicles. A simple and secure algorithm for vehicle positioning is proposed herein without massively modifying the existing transportation infrastructure. For vehicle localization, vehicles on the road are classified into two categories: host vehicles (HVs) are the ones used to estimate other vehicles' positions and forwarding vehicles (FVs) are the ones that move in front of the HVs. The FV transmits modulated data from the tail (or back) light, and the camera of the HV receives that signal using optical camera communication (OCC). In addition, the streetlight (SL) data are considered to ensure the position accuracy of the HV. Determining the HV position minimizes the relative position variation between the HV and FV. Using photogrammetry, the distance between FV or SL and the camera of the HV is calculated by measuring the occupied image area on the image sensor. Comparing the change in distance between HV and SLs with the change in distance between HV and FV, the positions of FVs are determined. The performance of the proposed technique is analyzed, and the results indicate a significant improvement in performance. The experimental distance measurement validated the feasibility of the proposed scheme

Semantic descriptor for intelligence services

The exposition and discovery of intelligence especially for connected devices and autonomous systems have become an important area of the research towards an all-intelligent world. In this article, it a semantic description of functions is proposed and used to provide intelligence services mainly for networked devices. The semantic descriptors aim to provide interoperability between multiple domains' vocabularies, data models, and ontologies, so that device applications become able to deploy them autonomously once they are onboarded in the device or system platform. The proposed framework supports the discovery, onboarding, and updating of the services by providing descriptions of their execution environment, software dependencies, policies and data inputs required, as well as the outputs produced, to enable application decoupling from the AI functions

Comunicações com câmara para aplicações de platooning

Platooning is a technology that corresponds to all the coordinated movements of a collection of vehicles, or, in the case of mobile robotics, to all the coordinated movements of a collection of mobile robots. It brings several advantages to driving, such as, improved safety, accurate speed control, lower CO2 emission rates, and higher energy efficiency. This dissertation describes the development of a laboratory scale demonstrator of platooning based on optical camera communications, using two generic wheel steered robots. For this purpose, one of the robots is equipped with a Light Emitting Diode (LED) matrix and the other with a camera. The LED matrix acts as an Optical Camera Communication (OCC) transmitter, providing status information of the robot attitude. The camera acts as both image acquisition and as an OCC receiver. The gathered information is processed using the algorithm You Only Look Once (YOLO) to infer the robot motion. The YOLO object detector continuously checks the movement of the robot in front. Performance evaluation of 5 different YOLO models (YOLOv3, YOLOv3-tiny, YOLOv4, YOLOv4-tiny, YOLOv4-tiny-3l) was conducted to assess which model works best for this project. The outcomes demonstrate that YOLOv4-tiny surpasses the other models in terms of timing, making it the ideal choice for real-time performance. Object detection using YOLOv4-tiny was performed on the computer. This was chosen since it has a processing speed of 3.09 fps as opposed to the Raspberry Pi’s 0.2 fps.O platooning é uma tecnologia que corresponde a todas as movimentações coordenadas de um conjunto de veículos, ou, no caso da robótica movel, a todas as movimentações coordenadas de um conjunto de robots móveis. Traz várias vantagens para a condução, tais como, maior segurança, um controlo preciso da velocidade, menores taxas de emissão de CO2 e maior eficiência energética. Esta dissertação descreve o desenvolvimento de um demonstrador de platooning em escala laboratorial baseado em comunicações com câmera, usando dois robôs móveis genéricos. Para este propósito, um dos robôs é equipado com uma matriz de Light Emitting Diodes (LEDs) e o outro é equipado com uma câmera. A matriz de LEDs funciona como transmissor, fornecendo informações de estado do robô. A câmera funciona como recetor, realizando a aquisição de imagens. As informações recolhidas são processadas usando o algoritmo You Only Look Once (YOLO) de forma a prever o movimento do robô. O YOLO verifica continuamente o movimento do robô da frente. A avaliação de desempenho de 5 modelos de YOLO diferentes (YOLOv3, YOLOv3-tiny, YOLOv4, YOLOv4-tiny, YOLOv4-tiny-3l) foi realizada para identificar qual o modelo que funciona melhor no contexto deste projeto. Os resultados demonstram que o YOLOv4-tiny supera os outros modelos em termos de tempo, tornando-o a escolha ideal para desempenho em tempo real. A deteção de objetos usando YOLOv4-tiny foi realizada no computador. Esta escolhe deveuse ao facto de o computador ter uma velocidade de processamento de 3,09 fps em oposição aos 0,2 fps da Raspberry Pi.Mestrado em Engenharia Eletrónica e Telecomunicaçõe

Development,Validation, and Integration of AI-Driven Computer Vision System and Digital-twin System for Traffic Safety Dignostics

The use of data and deep learning algorithms in transportation research have become increasingly popular in recent years. Many studies rely on real-world data. Collecting accurate traffic data is crucial for analyzing traffic safety. Still, traditional traffic data collection methods that rely on loop detectors and radar sensors are limited to collect macro-level data, and it may fail to monitor complex driver behaviors like lane changing and interactions between road users. With the development of new technologies like in-vehicle cameras, Unmanned Aerial Vehicle (UAV), and surveillance cameras, vehicle trajectory data can be collected from the recorded videos for more comprehensive and microscopic traffic safety analysis. This research presents the development, validation, and integration of three AI-driven computer vision systems for vehicle trajectory extraction and traffic safety research: 1) A.R.C.I.S, an automated framework for safety diagnosis utilizing multi-object detection and tracking algorithm for UAV videos. 2)N.M.E.D.S., A new framework with the ability to detect and predict the key points of vehicles and provide more precise vehicle occupying locations for traffic safety analysis. 3)D.V.E.D.S applied deep learning models to extract information related to drivers\u27 visual environment from the Google Street View (GSV) images. Based on the drone video collected and processed by A.R.C.I.S at various locations, CitySim: a new drone recorded vehicle trajectory dataset that aim to facilitate safety research was introduced. CitySim has vehicle interaction trajectories extracted from 1140- minutes of video recordings, which provide a large-scale naturalistic vehicle trajectory that covers a variety of locations, including basic freeway segments, freeway weaving segments, expressway segments, signalized intersections, stop-controlled intersections, and unique intersections without sign/signal control. The advantage of CitySim over other datasets is that it contains more critical safety events in quantity and severity and provides supporting scenarios for safety-oriented research. In addition, CitySim provides digital twin features, including the 3D base maps and signal timings, which enables a more comprehensive testing environment for safety research, such as autonomous vehicle safety. Based on these digital twin features provided by CitySim, we proposed a Digital Twin framework for CV and pedestrian in-the-loop simulation, which is based on Carla-Sumo Co-simulation and Cave automatic virtual environment (CAVE). The proposed framework is expected to guide the future Digital Twin research, and the architecture we build can serve as the testbed for further research and development

Multi-modal Machine Learning for Vehicle Rating Predictions Using Image, Text, and Parametric Data

Accurate vehicle rating prediction can facilitate designing and configuring good vehicles. This prediction allows vehicle designers and manufacturers to optimize and improve their designs in a timely manner, enhance their product performance, and effectively attract consumers. However, most of the existing data-driven methods rely on data from a single mode, e.g., text, image, or parametric data, which results in a limited and incomplete exploration of the available information. These methods lack comprehensive analyses and exploration of data from multiple modes, which probably leads to inaccurate conclusions and hinders progress in this field. To overcome this limitation, we propose a multi-modal learning model for more comprehensive and accurate vehicle rating predictions. Specifically, the model simultaneously learns features from the parametric specifications, text descriptions, and images of vehicles to predict five vehicle rating scores, including the total score, critics score, performance score, safety score, and interior score. We compare the multi-modal learning model to the corresponding unimodal models and find that the multi-modal model's explanatory power is 4% - 12% higher than that of the unimodal models. On this basis, we conduct sensitivity analyses using SHAP to interpret our model and provide design and optimization directions to designers and manufacturers. Our study underscores the importance of the data-driven multi-modal learning approach for vehicle design, evaluation, and optimization. We have made the code publicly available at http://decode.mit.edu/projects/vehicleratings/.Comment: The paper submitted to IDETC/CIE2023, the International Design Engineering Technical Conferences & Computers and Information in Engineering Conference, has been accepte

A Systematic Survey of ML Datasets for Prime CV Research Areas-Media and Metadata

The ever-growing capabilities of computers have enabled pursuing Computer Vision through Machine Learning (i.e., MLCV). ML tools require large amounts of information to learn from (ML datasets). These are costly to produce but have received reduced attention regarding standardization. This prevents the cooperative production and exploitation of these resources, impedes countless synergies, and hinders ML research. No global view exists of the MLCV dataset tissue. Acquiring it is fundamental to enable standardization. We provide an extensive survey of the evolution and current state of MLCV datasets (1994 to 2019) for a set of specific CV areas as well as a quantitative and qualitative analysis of the results. Data were gathered from online scientific databases (e.g., Google Scholar, CiteSeerX). We reveal the heterogeneous plethora that comprises the MLCV dataset tissue; their continuous growth in volume and complexity; the specificities of the evolution of their media and metadata components regarding a range of aspects; and that MLCV progress requires the construction of a global standardized (structuring, manipulating, and sharing) MLCV "library". Accordingly, we formulate a novel interpretation of this dataset collective as a global tissue of synthetic cognitive visual memories and define the immediately necessary steps to advance its standardization and integration

Perception Intelligence Integrated Vehicle-to-Vehicle Optical Camera Communication.

Ubiquitous usage of cameras and LEDs in modern road and aerial vehicles open up endless opportunities for novel applications in intelligent machine navigation, communication, and networking. To this end, in this thesis work, we hypothesize the benefit of dual-mode usage of vehicular built-in cameras through novel machine perception capabilities combined with optical camera communication (OCC). Current key conception of understanding a line-of-sight (LOS) scenery is from the aspect of object, event, and road situation detection. However, the idea of blending the non-line-of-sight (NLOS) information with the LOS information to achieve a see-through vision virtually is new. This improves the assistive driving performance by enabling a machine to see beyond occlusion. Another aspect of OCC in the vehicular setup is to understand the nature of mobility and its impact on the optical communication channel quality. The research questions gathered from both the car-car mobility modelling, and evaluating a working setup of OCC communication channel can also be inherited to aerial vehicular situations like drone-drone OCC. The aim of this thesis is to answer the research questions along these new application domains, particularly, (i) how to enable a virtual see-through perception in the car assisting system that alerts the human driver about the visible and invisible critical driving events to help drive more safely, (ii) how transmitter-receiver cars behaves while in the mobility and the overall channel performance of OCC in motion modality, (iii) how to help rescue lost Unmanned Aerial Vehicles (UAVs) through coordinated localization with fusion of OCC and WiFi, (iv) how to model and simulate an in-field drone swarm operation experience to design and validate UAV coordinated localization for group of positioning distressed drones. In this regard, in this thesis, we present the end-to-end system design, proposed novel algorithms to solve the challenges in applying such a system, and evaluation results through experimentation and/or simulation