The NVIDIA AI City Challenge

Web image analysis has witnessed an AI renaissance. The ILSVRC benchmark has been instrumental in providing a corpus and standardized evaluation. The NVIDIA AI City Challenge is envisioned to provide similar impetus to the analysis of image and video data that helps make cities smarter and safer. In its first year, this Challenge has focused on traffic video data. While millions of traffic video cameras around the world capture data, albeit low-quality, very little automated analysis and value creation results. Lack of labeled data, and trained models that can be deployed at the edge of the city fabric, ensure that most traffic video data goes through little or no automated analysis. Real-time and batch analysis of this data can provide vital breakthroughs in real-time traffic management as well as pedestrian safety. The NVIDIA AI City Challenge brought together 29 teams from universities in 4 continents to collaboratively annotate a 125 hour data set and then compete on detection, localization and classification tasks as well as traffic and safety application analytics tasks. The result is the largest high quality annotated data set, a set of models trained using NVIDIA AI City Edge to Cloud platform and ready to be deployed at the edge solving traffic and safety problems for cities worldwide

The 2018 NVIDIA AI City Challenge

The NVIDIA AI City Challenge has been created to accelerate intelligent video analysis that helps make cities smarter and safer. With millions of traffic video cameras acting as sensors around the world, there is a significant opportunity for real-time and batch analysis of these videos to provide actionable insights. These insights will benefit a wide variety of agencies, from traffic control to public safety. The second edition of the NVIDIA AI City Challenge, being organized as a CVPR workshop, provided a forum to more than 70 academic and industrial research teams to compete and solve real-world problems using traffic camera video data. The Challenge was launched with three tracks — speed estimation, anomaly detection, and vehicle re-identification. Each track was chosen in consultation with traffic and public safety officials based on the value of potential solutions. With the largest available dataset for such tasks, and ground truth for each track, the Challenge enabled 22 teams to evaluate their solutions. Given how complex these tasks are, the results are encouraging and reflect increased value addition year over year for the Challenge

Vehicle Tracking and Speed Estimation from Traffic Videos

The rapid recent advancements in the computation ability of everyday computers have made it possible to widely apply deep learning methods to the analysis of traffic surveillance videos. Traffic flow prediction, anomaly detection, vehicle re-identification, and vehicle tracking are basic components in traffic analysis. Among these applications, traffic flow prediction, or vehicle speed estimation, is one of the most important research topics of recent years. Good solutions to this problem could prevent traffic collisions and help improve road planning by better estimating transit demand. In the 2018 NVIDIA AI City Challenge, we combine modern deep learning models with classic computer vision approaches to propose an efficient way to predict vehicle speed. In this paper, we introduce some state-of-the-art approaches in vehicle speed estimation, vehicle detection, and object tracking, as well as our solution for Track 1 of the Challenge

Robust classification of city roadway objects for traffic related applications

The increasing prevalence of video data, particularly from traffic and surveillance cameras, is accompanied by a growing need for improved object detection, tracking, and classification techniques. In order to encourage development in this area, the AI City Challenge, sponsored by IEEE Smart World and NVIDIA, cultivated a competitive environment in which teams from all over the world sought to demonstrate the effectiveness of their models after training and testing on a common dataset of 114,766 unique traffic camera keyframes. Models were constructed for two distinct purposes; track 1 designs addressed object detection, localization and classification, while track 2 designs aimed to produce novel approaches towards traffic related application development. Careful tuning of the Darknet framework\u27s YOLO (You Only Look Once) architecture allowed us to achieve 2nd place scores in track 1 of the competition. Our model was able to achieve inference beyond 50 frames per second (FPS) when performing on the NVIDIA DGX-1\u27s Tesla P100 GPU and up to 37 FPS on a NVIDIA GTX 1070 GPU. However, the NVIDIA Jetson TX2 edge device had a lackluster 2 FPS inference speed. To produce truly competitive automated traffic control systems, either more preferment edge device hardware or revolutionary neural network architectures are required. While our track 2 model approach demonstrated that it is reasonable to obtain useful traffic related metrics without the use of the region proposal networks and classification methods utilized in other models typically associated with traffic control systems

Multi-task mutual learning for vehicle re-identification

Vehicle re-identification (Re-ID) aims to search a specific vehicle instance across non-overlapping camera views. The main challenge of vehicle Re-ID is that the visual appearance of vehicles may drastically changes according to diverse viewpoints and illumination. Most existing vehicle Re-ID models cannot make full use of various complementary vehicle information, e.g. vehicle type and orientation. In this paper, we propose a novel Multi-Task Mutual Learning (MTML) deep model to learn discriminative features simultaneously from multiple branches. Specifically, we design a consensus learning loss function by fusing features from the final convolutional feature maps from all branches. Extensive comparative evaluations demonstrate the effectiveness of our proposed MTML method in comparison to the state-of-the-art vehicle Re-ID techniques on a large-scale benchmark dataset, VeRi-776. We also yield competitive performance on the NVIDIA 2019 AI City Challenge Track 2

Dual Embedding Expansion for Vehicle Re-identification

Vehicle re-identification plays a crucial role in the management of transportation infrastructure and traffic flow. However, this is a challenging task due to the large view-point variations in appearance, environmental and instance-related factors. Modern systems deploy CNNs to produce unique representations from the images of each vehicle instance. Most work focuses on leveraging new losses and network architectures to improve the descriptiveness of these representations. In contrast, our work concentrates on re-ranking and embedding expansion techniques. We propose an efficient approach for combining the outputs of multiple models at various scales while exploiting tracklet and neighbor information, called dual embedding expansion (DEx). Additionally, a comparative study of several common image retrieval techniques is presented in the context of vehicle re-ID. Our system yields competitive performance in the 2020 NVIDIA AI City Challenge with promising results. We demonstrate that DEx when combined with other re-ranking techniques, can produce an even larger gain without any additional attribute labels or manual supervision