Video expert assessment of high quality video for Video Assistant Referee (VAR) : A comparative study

The International Football Association Board decided to introduce Video Assistant Referee (VAR) in 2018. This led to the need to develop methods for quality control of the VAR-systems. This article focuses on the important aspect to evaluate the video quality. Video Quality assessment has matured in the sense that there are standardized, commercial products and established open-source solutions to measure it with objective methods. Previous research has primarily focused on the end-user quality assessment. How to assess the video in the contribution phase of the chain is less studied. The novelties of this study are two-fold: 1) The user study is specifically targeting video experts i.e., to assess the perceived quality of video professionals working with video production. 2) Six video quality models have been independently benchmarked against the user data and evaluated to show which of the models could provide the best predictions of perceived quality. The independent evaluation is important to get unbiased results as shown by the Video Quality Experts Group. An experiment was performed involving 25 video experts in which they rated the perceived quality. The video formats tested were High-Definition TV both progressive and interlaced as well as a quarters size format that was scaled down half the size in both width and height. The videos were encoded with both H.264 and Motion JPEG for the full size but only H.264 for the quarter size. Bitrates ranged from 80 Mbit/s down to 10 Mbit/s. We could see that for H.264 that the quality was overall very good but dropped somewhat for 10 Mbit/s. For Motion JPEG the quality dropped over the whole range. For the interlaced format the degradation that was based on a simple deinterlacing method did receive overall low ratings. For the quarter size three different scaling algorithms were evaluated. Lanczos performed the best and Bilinear the worst. The performance of six different video quality models were evaluated for 1080p and 1080i. The Video Quality Metric for Variable Frame Delay had the best performance for both formats, followed by Video Multimethod Assessment Fusion method and the Video Quality Metric General model. This work was funded by Fédération Internationale de Football Association (FIFA) and Sweden´s Innovation Agency (VINNOVA, dnr. 2021-02107) through the Celtic-Next project IMMINENCE (C2020/2-2), which is hereby gratefully acknowledged.</p

Video quality of video professionals for Video Assisted Referee (VAR)

Changes in the footballing world’s approach to technology and innovation contributed to the decision by the International Football Association Board to introduce Video Assistant Referees (VAR). The change meant that under strict protocols referees could use video replays to review decisions in the event of a "clear and obvious error" or a "serious missed incident". This led to the need by Fédération Internationale de Football Association (FIFA) to develop methods for quality control of the VAR-systems, which was done in collaboration with RISE Research Institutes of Sweden AB. One of the important aspects is the video quality. The novelty of this study is that it has performed a user study specifically targeting video experts i.e., to measure the perceived quality of video professionals working with video production as their main occupation. An experiment was performed involving 25 video experts. In addition, six video quality models have been benchmarked against the user data and evaluated to show which of the models could provide the best predictions of perceived quality for this application. Video Quality Metric for variable frame delay (VQM_VFD) had the best performance for both formats, followed by Video Multimethod Assessment Fusion (VMAF) and VQM General model.This work was mainly funded by Fédération Internationale de Football Association (FIFA) and partly supported by the Sweden´s Innovation Agency (VINNOVA, dnr. 2021-02107) through the Celtic-Next project IMMINENCE (C2020/2-2) as well as RISE internal funding. </p

Supplementary information files for Comparison of player perceptions to mechanical measurements of third generation synthetic turf football surfaces

Supplementary files for article Comparison of player perceptions to mechanical measurements of third generation synthetic turf football surfacesMechanical testing of synthetic turf football surfaces is considered essential to ensure player performance and safety. However, it remains unknown how well the mechanical outputs reflect player perceptions of these surfaces. The first objective of this study was to investigate the agreement between the outputs from the Rotational Traction Tester and the Advanced Artificial Athlete with player perceptions across a range of controlled third generation turf football surfaces. The second objective was to identify the modifications to the Rotational Traction Tester and the Advanced Artificial Athlete configurations and output variables that give the strongest agreement with player perceptions. An indoor test area containing ten third generation turf surfaces with controlled hardness and traction properties was constructed. Each surface was tested using the Advanced Artificial Athlete and Rotational Traction Tester in their current configuration and in several modified configurations aimed at better replicating the player–surface interaction. Using a trained panel paired comparisons technique, 18 University footballers (11 males and 7 females) identified differences in the surfaces based on four sensory attributes Movement Speed, Slip, Leg Shock and Give. Results indicated strong agreement (correlation coefficients between 0.7 and 1.0) across several Rotational Traction Tester and Advanced Artificial Athlete testing configurations and output variables with player perceptions. It is recommended that the current Rotational Traction Tester is improved through added instrumentation to allow surface stiffness to be evaluated (the rate of generation of traction resistance). It is further recommended that the Advanced Artificial Athlete adopts a new algorithm to improve the accuracy of the surface’s Vertical Deformation and Energy Restitution, and the number of drops is reduced from three to one.</p

Comparison of player perceptions to mechanical measurements of third generation synthetic turf football surfaces

Mechanical testing of synthetic turf football surfaces is considered essential to ensure player performance and safety. However, it remains unknown how well the mechanical outputs reflect player perceptions of these surfaces. The first objective of this study was to investigate the agreement between the outputs from the Rotational Traction Tester and the Advanced Artificial Athlete with player perceptions across a range of controlled third generation turf football surfaces. The second objective was to identify the modifications to the Rotational Traction Tester and the Advanced Artificial Athlete configurations and output variables that give the strongest agreement with player perceptions. An indoor test area containing ten third generation turf surfaces with controlled hardness and traction properties was constructed. Each surface was tested using the Advanced Artificial Athlete and Rotational Traction Tester in their current configuration and in several modified configurations aimed at better replicating the player–surface interaction. Using a trained panel paired comparisons technique, 18 University footballers (11 males and 7 females) identified differences in the surfaces based on four sensory attributes Movement Speed, Slip, Leg Shock and Give. Results indicated strong agreement (correlation coefficients between 0.7 and 1.0) across several Rotational Traction Tester and Advanced Artificial Athlete testing configurations and output variables with player perceptions. It is recommended that the current Rotational Traction Tester is improved through added instrumentation to allow surface stiffness to be evaluated (the rate of generation of traction resistance). It is further recommended that the Advanced Artificial Athlete adopts a new algorithm to improve the accuracy of the surface’s Vertical Deformation and Energy Restitution, and the number of drops is reduced from three to one

Development and evaluation of a sensory panel for collecting reliable player perceptions of third‑generation synthetic turf football surfaces

Perceptions of synthetic surfaces used in football can vary considerably between players, and obtaining reliable feedback is challenging. The aim of this study was to develop a suitable process and evaluate the merits of establishing a sensory panel to assess the subjective attributes of third generation synthetic turf surfaces (3G turf) used in football. Focus groups with 12 male and 13 female footballers were conducted on an outdoor 3G turf pitch to develop a common language to describe sensory feedback related to player–surface interactions. Post-session analysis revealed two main themes related to player–surface interactions: hardness and grip. These themes were broken down further into five sensory attributes (Movement Speed, Slip, Movement Confidence, Leg Shock and Give) which were investigated further in an indoor test area containing ten 3G turf surfaces with controlled surface properties. A panel consisting of 18 University footballers (11 male and 7 female) undertook a screening and training session to refine the language associated with the sensory attributes and become familiar with the testing protocol. During a final evaluation session, players were asked to discriminate between surfaces using the paired comparison method for each of the sensory attributes. Player consistency remained similar between the screening and evaluation sessions whilst the panel’s ability to discriminate between surfaces improved during the evaluation session. Sensory training can therefore be a useful approach to aid players in differentiating between surfaces and lead to a greater understanding of athlete perceptions of surface attributes. </p