1st Workshop on Maritime Computer Vision (MaCVi) 2023: Challenge Results

The 1$^{\text{st}}$ Workshop on Maritime Computer Vision (MaCVi) 2023 focused on maritime computer vision for Unmanned Aerial Vehicles (UAV) and Unmanned Surface Vehicle (USV), and organized several subchallenges in this domain: (i) UAV-based Maritime Object Detection, (ii) UAV-based Maritime Object Tracking, (iii) USV-based Maritime Obstacle Segmentation and (iv) USV-based Maritime Obstacle Detection. The subchallenges were based on the SeaDronesSee and MODS benchmarks. This report summarizes the main findings of the individual subchallenges and introduces a new benchmark, called SeaDronesSee Object Detection v2, which extends the previous benchmark by including more classes and footage. We provide statistical and qualitative analyses, and assess trends in the best-performing methodologies of over 130 submissions. The methods are summarized in the appendix. The datasets, evaluation code and the leaderboard are publicly available at https://seadronessee.cs.uni-tuebingen.de/macvi.Comment: MaCVi 2023 was part of WACV 2023. This report (38 pages) discusses the competition as part of MaCV

Augmented Reality Based Interactive Cooking Guide

Cooking at home is a critical survival skill. We propose a new cooking assistance system in which a user only needs to wear an all-in-one augmented reality (AR) headset without having to install any external sensors or devices in the kitchen. Utilizing the built-in camera and cutting-edge computer vision (CV) technology, the user can direct the AR headset to recognize available food ingredients by simply looking at them. Based on the types of the recognized food ingredients, suitable recipes are suggested accordingly. A step-by-step video tutorial providing details of the selected recipe is then displayed with the AR glasses. The user can conveniently interact with the proposed system using eight kinds of natural hand gestures without needing to touch any devices throughout the entire cooking process. Compared with the deep learning models ResNet and ResNeXt, experimental results show that the YOLOv5 achieves lower accuracy for ingredient recognition, but it can locate and classify multiple ingredients in one shot and make the scanning process easier for users. Twenty participants test the prototype system and provide feedback via two questionnaires. Based on the analysis results, 19 of the 20 participants would recommend others to use the proposed system, and all participants are overall satisfied with the prototype system

Appearance-Based Multimodal Human Tracking and Identification for Healthcare in the Digital Home

There is an urgent need for intelligent home surveillance systems to provide home security, monitor health conditions, and detect emergencies of family members. One of the fundamental problems to realize the power of these intelligent services is how to detect, track, and identify people at home. Compared to RFID tags that need to be worn all the time, vision-based sensors provide a natural and nonintrusive solution. Observing that body appearance and body build, as well as face, provide valuable cues for human identification, we model and record multi-view faces, full-body colors and shapes of family members in an appearance database by using two Kinects located at a home’s entrance. Then the Kinects and another set of color cameras installed in other parts of the house are used to detect, track, and identify people by matching the captured color images with the registered templates in the appearance database. People are detected and tracked by multisensor fusion (Kinects and color cameras) using a Kalman filter that can handle duplicate or partial measurements. People are identified by multimodal fusion (face, body appearance, and silhouette) using a track-based majority voting. Moreover, the appearance-based human detection, tracking, and identification modules can cooperate seamlessly and benefit from each other. Experimental results show the effectiveness of the human tracking across multiple sensors and human identification considering the information of multi-view faces, full-body clothes, and silhouettes. The proposed home surveillance system can be applied to domestic applications in digital home security and intelligent healthcare

OCTAVIA CAMPS, Member. IEEE

with memben of the ai & industry, menUy dweioped technologies for a new suprrsanie aimraR One of the teehoalogieal a m considered lor thb aired is the us0 of video-em and imsge-pmeessing equipmwt to aid the pilot io detecting other aireran io the sky. The detection lffhniques should pmvide high detection probability for obstacles that rao vary fmm subpixel to a few pixels in size, while maintaining a low false alarm pmbability in the p-0 ~ of noise and severe background clmu. Furthermore, the detection algolithms mu @ be able to report such obstacles in a timely fashion, imposing severe constraints on their execution time Approache are drrlibed here to detect airbome obstacles 00 collision coulse and e-ng trajectorie in video images captured fmm an airborne air era ^ ~n both eases the approaches comist of m image-pmdog stage to identify pOSSible obstacle followed by a traelijng stage to distinguish behreen true obstades and image duner, based on their behavior. For coilision course objed detection, the image-p-ieg stage morphoiogjesi Biter Lo remove large-sined duller. To lomove tho remaWng small-sired clutter, differences in the behador of image translation and rrpaodan of the amponding leatuRs is urod io the traclriog stage. For emssing object detection, tho imag-pmeessing sfage uses low-stop Blter and image differondng to separate stationary baekgmund clutter. The remaining clutter is removed in the tracLing stage by assuming that the genuine object hss B large signal strength, as well s a si+raot and consistent motion over a number of frames. The amsing objprt delfftion algorithm was implemented on a pipeiined arehilecture fmm Datacube and mm io mal time. Both algorithms have been sueeensIuliy terted on flight tests conducted by NASA. Manuscript meived September 20, 2001; revised August 15, 2002; released for publication September 26, 2002

Traffic flow estimation and vehicle‐type classification using vision‐based spatial–temporal profile analysis

Vision‐based traffic surveillance plays an important role in traffic management. However, outdoor illuminations, the cast shadows and vehicle variations often create problems for video analysis and processing. Thus, the authors propose a real‐time cost‐effective traffic monitoring system that can reliably perform traffic flow estimation and vehicle classification at the same time. First, the foreground is extracted using a pixel‐wise weighting list that models the dynamic background. Shadows are discriminated utilising colour and edge invariants. Second, the foreground on a specified check‐line is then collected over time to form a spatial–temporal profile image. Third, the traffic flow is estimated by counting the number of connected components in the profile image. Finally, the vehicle type is classified according to the size of the foreground mask region. In addition, several traffic measures, including traffic velocity, flow, occupancy and density, are estimated based on the analysis of the segmentation. The availability and reliability of these traffic measures provides critical information for public transportation monitoring and intelligent traffic control. Since the proposed method only process a small area close to the check‐line to collect the spatial–temporal profile for analysis, the complete system is much more efficient than existing visual traffic flow estimation methods

Detection of Obstacles in the Flight Path of an Aircraft

The National Aeronautics and Space Administration (NASA), along with members of the aircraft industry, recently developed technologies for a new supersonic aircraft. One of the technological areas considered for this aircraft is the use of video cameras and image processing equipment to aid the pilot in detecting other aircraft in the sky. The detection techniques should provide high detection probability for obstacles that can vary from sub-pixel to a few pixels in size, while maintaining a low false alarm probability in the presence of noise and severe background clutter. Furthermore, the detection algorithms must be able to report such obstacles in a timely fashion, imposing severe constraints on their execution time. This paper describes approaches to detect airborne obstacles on collision course and crossing trajectories in video images captured from an airborne aircraft. In both cases the approaches consist of an image processing stage to identify possible obstacles followed by a tracking stage to distinguish between true obstacles and image clutter, based on their behavior. The crossing target detection algorithm was also implemented on a pipelined architecture from DataCube and runs in real time. Both algorithms have been successfully tested on flight tests conducted by NASA