Automatic thumbnail selection for soccer videos using machine learning

Thumbnail selection is a very important aspect of online sport video presentation, as thumbnails capture the essence of important events, engage viewers, and make video clips attractive to watch. Traditional solutions in the soccer domain for presenting highlight clips of important events such as goals, substitutions, and cards rely on the manual or static selection of thumbnails. However, such approaches can result in the selection of sub-optimal video frames as snapshots, which degrades the overall quality of the video clip as perceived by viewers, and consequently decreases viewership, not to mention that manual processes are expensive and time consuming. In this paper, we present an automatic thumbnail selection system for soccer videos which uses machine learning to deliver representative thumbnails with high relevance to video content and high visual quality in near real-time. Our proposed system combines a software framework which integrates logo detection, close-up shot detection, face detection, and image quality analysis into a modular and customizable pipeline, and a subjective evaluation framework for the evaluation of results. We evaluate our proposed pipeline quantitatively using various soccer datasets, in terms of complexity, runtime, and adherence to a pre-defined rule-set, as well as qualitatively through a user study, in terms of the perception of output thumbnails by end-users. Our results show that an automatic end-to-end system for the selection of thumbnails based on contextual relevance and visual quality can yield attractive highlight clips, and can be used in conjunction with existing soccer broadcast pipelines which require real-time operation

Development and Flight Results from the C3D2 Imager Payload on AlSat Nano

An experimental CubeSat camera system using 3 separate CMOS imagers was flown in 2014 on UKube-1. In response to an announcement opportunity in December 2014, we proposed an upgrade to our C3D imager payload, which was accepted to fly on AlSat Nano. Launched in September 2016 the system has been operational for over 1 year and has returned both images and housekeeping data, including detailed temperature and radiation dosimetry measurements. Through these in-orbit demonstrations on CubeSans, the image sensors and payload have attained TRL9, and these are now being used in other flight opportunities. In this paper we describe the C3D imager payload, which comprises 3 independent CMOS image sensors used in different camera systems; two wide field cameras are specifically optimised with one to observe the Earth from the 650 km orbit, and the other with its focus set to 40 cm to observe a deployable boom from the CubeSat. The experiment controller also contained thermometry and two RADFET dosimeters, one located on the payload, with the other deployed at a different point on the spacecraft. In this paper we will describe the experiment design and operational performance, and review the in-orbit data obtained during the operations covering over 17 months in-orbit, in addition to discussing lessons learned from the flight experience. We also discuss further developments of the payload concept which we are currently working on toward future flight opportunities

An Algorithm for Real-Time Blind Image Quality Comparison and Assessment

This research aims at providing means to image comparison from different image processing algorithms for performance assessment purposes. Reconstruction of images corrupted by blur and noise requires specialized filtering techniques. Due to the immense effect of these corruptive parameters, it is often impossible to evaluate the quality of a reconstructed image produced by one technique versus another. The algorithm presented here is capable of performing this comparison analytically and quantitatively at a low computational cost (real-time) and high efficiency. The parameters used for comparison are the degree of blurriness, information content, and the amount of various types of noise associated with the reconstructed image. Based on a heuristic analysis of these parameters the algorithm assesses the reconstructed image and quantify the quality of the image by characterizing important aspects of visual quality. Extensive effort has been set forth to obtain real-world noise and blur conditions so that the various test cases presented here could justify the validity of this approach well. The tests performed on the database of images produced valid results for the algorithms consistently. This paper presents the description and validation (along with test results) of the proposed algorithm for blind image quality assessment.DOI:http://dx.doi.org/10.11591/ijece.v2i1.112 

SAVASA project @ TRECVID 2012: interactive surveillance event detection

In this paper we describe our participation in the interactive surveillance event detection task at TRECVid 2012. The system we developed was comprised of individual classifiers brought together behind a simple video search interface that enabled users to select relevant segments based on down~sampled animated gifs. Two types of user -- `experts' and `end users' -- performed the evaluations. Due to time constraints we focussed on three events -- ObjectPut, PersonRuns and Pointing -- and two of the five available cameras (1 and 3). Results from the interactive runs as well as discussion of the performance of the underlying retrospective classifiers are presented

Evaluation of changes in image appearance with changes in displayed image size

This research focused on the quantification of changes in image appearance when images are displayed at different image sizes on LCD devices. The final results provided in calibrated Just Noticeable Differences (JNDs) on relevant perceptual scales, allowing the prediction of sharpness and contrast appearance with changes in the displayed image size. A series of psychophysical experiments were conducted to enable appearance predictions. Firstly, a rank order experiment was carried out to identify the image attributes that were most affected by changes in displayed image size. Two digital cameras, exhibiting very different reproduction qualities, were employed to capture the same scenes, for the investigation of the effect of the original image quality on image appearance changes. A wide range of scenes with different scene properties was used as a test-set for the investigation of image appearance changes with scene type. The outcomes indicated that sharpness and contrast were the most important attributes for the majority of scene types and original image qualities. Appearance matching experiments were further conducted to quantify changes in perceived sharpness and contrast with respect to changes in the displayed image size. For the creation of sharpness matching stimuli, a set of frequency domain filters were designed to provide equal intervals in image quality, by taking into account the system’s Spatial Frequency Response (SFR) and the observation distance. For the creation of contrast matching stimuli, a series of spatial domain S-shaped filters were designed to provide equal intervals in image contrast, by gamma adjustments. Five displayed image sizes were investigated. Observers were always asked to match the appearance of the smaller version of each stimulus to its larger reference. Lastly, rating experiments were conducted to validate the derived JNDs in perceptual quality for both sharpness and contrast stimuli. Data obtained by these experiments finally converted into JND scales for each individual image attribute. Linear functions were fitted to the final data, which allowed the prediction of image appearance of images viewed at larger sizes than these investigated in this research

Contour Enhancement Algorithm for Improving Visual Perception of Deutan and Protan Dichromats

A variety of recoloring methods has been proposed in the literature to remedy the problem of confusing red-green colors faced by dichromat people (as well by other color-blinded people). The common strategy to mitigate this problem is to remap colors to other colors. But, it is clear this does not guarantee neither the necessary contrast to distinguish the elements of an image, nor the naturalness of colors learnt from past experience of each individual. In other words, the individual’s perceptual learning may not hold under color remapping. With this in mind, we introduce the first algorithm primarily focused on the enhancement of object contours in still images, instead of recoloring the pixels of the regions bounded by such contours. This is particularly adequate to increase contrast in images where we find adjacent regions that are color-indistinguishable from the dichromacy’s point of view.info:eu-repo/semantics/publishedVersio

Fully-automated patient-level malaria assessment on field-prepared thin blood film microscopy images, including Supplementary Information

Malaria is a life-threatening disease affecting millions. Microscopy-based assessment of thin blood films is a standard method to (i) determine malaria species and (ii) quantitate high-parasitemia infections. Full automation of malaria microscopy by machine learning (ML) is a challenging task because field-prepared slides vary widely in quality and presentation, and artifacts often heavily outnumber relatively rare parasites. In this work, we describe a complete, fully-automated framework for thin film malaria analysis that applies ML methods, including convolutional neural nets (CNNs), trained on a large and diverse dataset of field-prepared thin blood films. Quantitation and species identification results are close to sufficiently accurate for the concrete needs of drug resistance monitoring and clinical use-cases on field-prepared samples. We focus our methods and our performance metrics on the field use-case requirements. We discuss key issues and important metrics for the application of ML methods to malaria microscopy.Comment: 16 pages, 13 figure