Video Object Detection with an Aligned Spatial-Temporal Memory

We introduce Spatial-Temporal Memory Networks for video object detection. At its core, a novel Spatial-Temporal Memory module (STMM) serves as the recurrent computation unit to model long-term temporal appearance and motion dynamics. The STMM's design enables full integration of pretrained backbone CNN weights, which we find to be critical for accurate detection. Furthermore, in order to tackle object motion in videos, we propose a novel MatchTrans module to align the spatial-temporal memory from frame to frame. Our method produces state-of-the-art results on the benchmark ImageNet VID dataset, and our ablative studies clearly demonstrate the contribution of our different design choices. We release our code and models at http://fanyix.cs.ucdavis.edu/project/stmn/project.html

Advances and Applications of Computer Vision Techniques in Vehicle Trajectory Generation and Surrogate Traffic Safety Indicators

The application of Computer Vision (CV) techniques massively stimulates microscopic traffic safety analysis from the perspective of traffic conflicts and near misses, which is usually measured using Surrogate Safety Measures (SSM). However, as video processing and traffic safety modeling are two separate research domains and few research have focused on systematically bridging the gap between them, it is necessary to provide transportation researchers and practitioners with corresponding guidance. With this aim in mind, this paper focuses on reviewing the applications of CV techniques in traffic safety modeling using SSM and suggesting the best way forward. The CV algorithm that are used for vehicle detection and tracking from early approaches to the state-of-the-art models are summarized at a high level. Then, the video pre-processing and post-processing techniques for vehicle trajectory extraction are introduced. A detailed review of SSMs for vehicle trajectory data along with their application on traffic safety analysis is presented. Finally, practical issues in traffic video processing and SSM-based safety analysis are discussed, and the available or potential solutions are provided. This review is expected to assist transportation researchers and engineers with the selection of suitable CV techniques for video processing, and the usage of SSMs for various traffic safety research objectives

Experimental investigation of the tire wear process using camera-assisted observation assessed by numerical modeling

This paper presents a novel experimental method to study the abrasion mechanism of car tires. It is based on the detection of microscopic movements associated with material damage (cracking) on the rubber tread. This is referred to as degrading layer relaxation. It correlates with the wear rate and, interestingly, the direction of the pattern's movement is opposite to the lateral forces during cornering. To measure and analyze the microscopic movements, a new camera-based method with feature point matching using video stabilization was developed. Besides extensive experimental investigation, the formation and propagation of microcracks are investigated using a simplified numerical model in which a phase field approach coupled with a viscoelastic constitutive behavior is implemented in a finite element framework

Challenges of Video Monitoring for Phenomenological Diagnostics in Present and Future Tokamaks

With the development of heterogeneous camera networks working at different wavelengths and frame rates and covering a large surface of vacuum vessel, the visual observation of a large variety of plasma and thermal phenomena (e.g., hot spots, ELMs, MARFE, arcs, dusts, etc.) becomes possible. In the domain of machine protection, a phenomenological diagnostic is a key-element towards plasma/thermal event dangerousness assessment during real time operation. It is also of primary importance to automate the extraction and the storage of phenomena information for further off-line event retrieval and analysis, thus leading to a better use of massive image data bases for plasma physics studies. To this end, efforts have been devoted to the development of image processing algorithms dedicated to the recognition of specific events. But a need arises now for the integration of techniques developed so far in both hardware and software directions. We present in this paper our latests results in the field of real time phenomena recognition and management through our image understanding software platform. This platform has been validated on Tore Supra during operation and is under evaluation for the foreseen imaging diagnostic of ITER

Embarking on the Autonomous Journey: A Strikingly Engineered Car Control System Design

openThis thesis develops an autonomous car control system with Raspberry Pi. Two predictive models are implemented: a convolutional neural network (CNN) using machine learning and an input-based decision tree model using sensor data. The Raspberry Module controls the car hardware and acquires real-time camera data with OpenCV. A dedicated web server and event stream processor process data in real-time using the trained neural network model, facilitating real-time decision-making. Unity and Meta Quest 2 VR set create the VR interface, while a generic DIY kit from Amazon and Raspberry PI provide the car hardware inputs. This research demonstrates the potential of VR in automotive communication, enhancing autonomous car testing and user experience.This thesis develops an autonomous car control system with Raspberry Pi. Two predictive models are implemented: a convolutional neural network (CNN) using machine learning and an input-based decision tree model using sensor data. The Raspberry Module controls the car hardware and acquires real-time camera data with OpenCV. A dedicated web server and event stream processor process data in real-time using the trained neural network model, facilitating real-time decision-making. Unity and Meta Quest 2 VR set create the VR interface, while a generic DIY kit from Amazon and Raspberry PI provide the car hardware inputs. This research demonstrates the potential of VR in automotive communication, enhancing autonomous car testing and user experience

Measuring traffic lane-changing by converting video into space–time still images

Empirical data is needed in order to extend our knowledge of traffic behavior. Video recordings are used to enrich typical data from loop detectors. In this context, data extraction from videos becomes a challenging task. Setting automatic video processing systems is costly, complex, and the accuracy achieved is usually not enough to improve traffic flow models. In contrast “visual” data extraction by watching the recordings requires extensive human intervention. A semiautomatic video processing methodology to count lane-changing in freeways is proposed. The method allows counting lane changes faster than with the visual procedure without falling into the complexities and errors of full automation. The method is based on converting the video into a set of space–time still images, from where to visually count. This methodology has been tested at several freeway locations near Barcelona (Spain) with good results. A user-friendly implementation of the method is available on http://bit.ly/2yUi08M.Peer ReviewedPostprint (published version