SoccerDB: A Large-Scale Database for Comprehensive Video Understanding

Soccer videos can serve as a perfect research object for video understanding because soccer games are played under well-defined rules while complex and intriguing enough for researchers to study. In this paper, we propose a new soccer video database named SoccerDB, comprising 171,191 video segments from 346 high-quality soccer games. The database contains 702,096 bounding boxes, 37,709 essential event labels with time boundary and 17,115 highlight annotations for object detection, action recognition, temporal action localization, and highlight detection tasks. To our knowledge, it is the largest database for comprehensive sports video understanding on various aspects. We further survey a collection of strong baselines on SoccerDB, which have demonstrated state-of-the-art performances on independent tasks. Our evaluation suggests that we can benefit significantly when jointly considering the inner correlations among those tasks. We believe the release of SoccerDB will tremendously advance researches around comprehensive video understanding. {\itshape Our dataset and code published on https://github.com/newsdata/SoccerDB.}Comment: accepted by MM2020 sports worksho

Greykite: Deploying Flexible Forecasting at Scale at LinkedIn

Forecasts help businesses allocate resources and achieve objectives. At LinkedIn, product owners use forecasts to set business targets, track outlook, and monitor health. Engineers use forecasts to efficiently provision hardware. Developing a forecasting solution to meet these needs requires accurate and interpretable forecasts on diverse time series with sub-hourly to quarterly frequencies. We present Greykite, an open-source Python library for forecasting that has been deployed on over twenty use cases at LinkedIn. Its flagship algorithm, Silverkite, provides interpretable, fast, and highly flexible univariate forecasts that capture effects such as time-varying growth and seasonality, autocorrelation, holidays, and regressors. The library enables self-serve accuracy and trust by facilitating data exploration, model configuration, execution, and interpretation. Our benchmark results show excellent out-of-the-box speed and accuracy on datasets from a variety of domains. Over the past two years, Greykite forecasts have been trusted by Finance, Engineering, and Product teams for resource planning and allocation, target setting and progress tracking, anomaly detection and root cause analysis. We expect Greykite to be useful to forecast practitioners with similar applications who need accurate, interpretable forecasts that capture complex dynamics common to time series related to human activity.Comment: In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD '22), August 14-18, 2022, Washington, DC, USA. ACM, New York, NY, USA, 11 page

Two-stage video-based convolutional neural networks for adult spinal deformity classification

IntroductionAssessment of human gait posture can be clinically effective in diagnosing human gait deformities early in life. Currently, two methods—static and dynamic—are used to diagnose adult spinal deformity (ASD) and other spinal disorders. Full-spine lateral standing radiographs are used in the standard static method. However, this is a static assessment of joints in the standing position and does not include information on joint changes when the patient walks. Careful observation of long-distance walking can provide a dynamic assessment that reveals an uncompensated posture; however, this increases the workload of medical practitioners. A three-dimensional (3D) motion system is proposed for the dynamic method. Although the motion system successfully detected dynamic posture changes, access to the facilities was limited. Therefore, a diagnostic approach that is facility-independent, has low practice flow, and does not involve patient contact is required.MethodsWe focused on a video-based method to classify patients with spinal disorders either as ASD, or other forms of ASD. To achieve this goal, we present a video-based two-stage machine-learning method. In the first stage, deep learning methods are used to locate the patient and extract the area where the patient is located. In the second stage, a 3D CNN (convolutional neural network) device is used to capture spatial and temporal information (dynamic motion) from the extracted frames. Disease classification is performed by discerning posture and gait from the extracted frames. Model performance was assessed using the mean accuracy, F1 score, and area under the receiver operating characteristic curve (AUROC), with five-fold cross-validation. We also compared the final results with professional observations.ResultsOur experiments were conducted using a gait video dataset comprising 81 patients. The experimental results indicated that our method is effective for classifying ASD and other spinal disorders. The proposed method achieved a mean accuracy of 0.7553, an F1 score of 0.7063, and an AUROC score of 0.7864. Additionally, ablation experiments indicated the importance of the first stage (detection stage) and transfer learning of our proposed method.DiscussionThe observations from the two doctors were compared using the proposed method. The mean accuracies observed by the two doctors were 0.4815 and 0.5247, with AUROC scores of 0.5185 and 0.5463, respectively. We proved that the proposed method can achieve accurate and reliable medical testing results compared with doctors' observations using videos of 1 s duration. All our code, models, and results are available at https://github.com/ChenKaiXuSan/Walk_Video_PyTorch. The proposed framework provides a potential video-based method for improving the clinical diagnosis for ASD and non-ASD. This framework might, in turn, benefit both patients and clinicians to treat the disease quickly and directly and further reduce facility dependency and data-driven systems

Smart Information Reconstruction Via Time-Space-Spectrum Continuum For Cloud Removal In Satellite Images

Cloud contamination is a big obstacle when processing satellite images retrieved from visible and infrared spectral ranges for application. Although computational techniques including interpolation and substitution have been applied to recover missing information caused by cloud contamination, these algorithms are subject to many limitations. In this paper, a novel smart information reconstruction (SMIR) method is proposed, in order to reconstruct cloud contaminated pixel values from the time-space-spectrum continuum with the aid of a machine learning tool, namely extreme learning machine (ELM). For the purpose of demonstration, the performance of SMIR is evaluated by reconstructing the missing remote sensing reflectance values derived from the Moderate Resolution Imaging Spectroradiometer (MODIS) on board the Terra satellite over Lake Nicaragua, where is a very cloudy area year round. For comparison, the traditional backpropagation neural network algorithms will also be implemented to reconstruct the missing values. Experimental results show that the ELM outperforms the BP algorithms by an enhanced machine learning capacity with simulated memory effect embedded in MODIS due to linking the complex time-space-spectrum continuum between cloud-free and cloudy pixels. The ELM-based SMIR practice presents a correlation coefficient of 0.88 with root mean squared error of 7.4 E-04 sr-1 between simulated and observed reflectance values. Finding suggests that the SMIR method is effective to reconstruct all the missing information providing visually logical and quantitatively assured images for further image processing and interpretation in environmental applications

Spectral Information Adaptation And Synthesis Scheme For Merging Cross-Mission Ocean Color Reflectance Observations From Modis And Viirs

Obtaining a full clear view of coastal bays, estuaries, lakes, and inland waters is challenging with single satellite sensor observations due to cloud impacts. Cross-mission sensors provide the synergistic opportunity to improve spatial and temporal coverage by merging their observations; however, discrepancies originating from the instrumental, algorithmic, and temporal differences should be eliminated before merging. This paper presents the Spectral Information Adaptation and Synthesis Scheme (SIASS) for generating cross-mission consistent ocean color reflectance by merging 2012-2015 observations from Moderate Resolution Imaging Spectroradiometer and Visible Infrared Imaging Radiometer Suite over Lake Nicaragua in Central America, where the cloud impact is salient. The SIASS is able to not only eliminate incompatibilities for matchup bands but also reconstruct spectral information for mismatched bands among sensors. Statistics indicate that the average monthly coverage of a merged ocean color reflectance product over Lake Nicaragua is nearly twice that of any single-sensor observation. Results show that SIASS significantly improves consistency among cross-mission sensors by mitigating prominent discrepancies. In addition, reconstructed spectral information for those mismatched bands help preserve more spectral characteristics needed to better monitor and understand the dynamic aquatic environment. The final implementation of SIASS to map the chlorophyll-aconcentration demonstrates the efficacy of SIASS in bias correction and consistency improvement. In general, SIASS can be applied to remove cross-mission discrepancies among sensors to improve the overall consistency