Pseudo-Dolly-In Video Generation Combining 3D Modeling and Image Reconstruction

This paper proposes a pseudo-dolly-in video generation method that reproduces motion parallax by applying image reconstruction processing to multi-view videos. Since dolly-in video is taken by moving a camera forward to reproduce motion parallax, we can present a sense of immersion. However, at a sporting event in a large-scale space, moving a camera is difficult. Our research generates dolly-in video from multi-view images captured by fixed cameras. By applying the Image-Based Modeling technique, dolly-in video can be generated. Unfortunately, the video quality is often damaged by the 3D estimation error. On the other hand, Bullet-Time realizes high-quality video observation. However, moving the virtual-viewpoint from the capturing positions is difficult. To solve these problems, we propose a method to generate a pseudo-dolly-in image by installing 3D estimation and image reconstruction techniques into Bullet-Time and show its effectiveness by applying it to multi-view videos captured at an actual soccer stadium. In the experiment, we compared the proposed method with digital zoom images and with the dolly-in video generated from the Image-Based Modeling and Rendering method.Published in: 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct) Date of Conference: 9-13 Oct. 2017 Conference Location: Nantes, Franc

Smoothly Switching Method of Asynchronous Multi-View Videos Using Frame Interpolation

This paper proposes a method that generates viewpoint smooth switching by reducing the flickering artifact observed at bullet-times generated from asynchronous multi-view videos using frame interpolation processing. When we asynchronously capture multi-view videos of an object moving at high velocity, deviations occur in the observed position at the bullet-times. We apply a frame interpolation technique to reduce the problem. By selecting suitable interpolated images that produce the smallest movement of the subject\u27s observed position, we smoothly generate viewpoint switched bullet-time video.Published in: 2017 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON) Date of Conference: 7-9 June 2017 Conference Location: Copenhagen, Denmar

Virtual camera synthesis for soccer game replays

International audienceIn this paper, we present a set of tools developed during the creation of a platform that allows the automatic generation of virtual views in a live soccer game production. Observing the scene through a multi-camera system, a 3D approximation of the players is computed and used for the synthesis of virtual views. The system is suitable both for static scenes, to create bullet time effects, and for video applications, where the virtual camera moves as the game plays

3D-TV Production from Conventional Cameras for Sports Broadcast

3DTV production of live sports events presents a challenging problem involving conflicting requirements of main- taining broadcast stereo picture quality with practical problems in developing robust systems for cost effective deployment. In this paper we propose an alternative approach to stereo production in sports events using the conventional monocular broadcast cameras for 3D reconstruction of the event and subsequent stereo rendering. This approach has the potential advantage over stereo camera rigs of recovering full scene depth, allowing inter-ocular distance and convergence to be adapted according to the requirements of the target display and enabling stereo coverage from both existing and ‘virtual’ camera positions without additional cameras. A prototype system is presented with results of sports TV production trials for rendering of stereo and free-viewpoint video sequences of soccer and rugby

Disparity map generation based on trapezoidal camera architecture for multiview video

Visual content acquisition is a strategic functional block of any visual system. Despite its wide possibilities, the arrangement of cameras for the acquisition of good quality visual content for use in multi-view video remains a huge challenge. This paper presents the mathematical description of trapezoidal camera architecture and relationships which facilitate the determination of camera position for visual content acquisition in multi-view video, and depth map generation. The strong point of Trapezoidal Camera Architecture is that it allows for adaptive camera topology by which points within the scene, especially the occluded ones can be optically and geometrically viewed from several different viewpoints either on the edge of the trapezoid or inside it. The concept of maximum independent set, trapezoid characteristics, and the fact that the positions of cameras (with the exception of few) differ in their vertical coordinate description could very well be used to address the issue of occlusion which continues to be a major problem in computer vision with regards to the generation of depth map

Enhanced Augmented Reality Framework for Sports Entertainment Applications

Augmented Reality (AR) superimposes virtual information on real-world data, such as displaying useful information on videos/images of a scene. This dissertation presents an Enhanced AR (EAR) framework for displaying useful information on images of a sports game. The challenge in such applications is robust object detection and recognition. This is even more challenging when there is strong sunlight. We address the phenomenon where a captured image is degraded by strong sunlight. The developed framework consists of an image enhancement technique to improve the accuracy of subsequent player and face detection. The image enhancement is followed by player detection, face detection, recognition of players, and display of personal information of players. First, an algorithm based on Multi-Scale Retinex (MSR) is proposed for image enhancement. For the tasks of player and face detection, we use adaptive boosting algorithm with Haar-like features for both feature selection and classification. The player face recognition algorithm uses adaptive boosting with the LDA for feature selection and nearest neighbor classifier for classification. The framework can be deployed in any sports where a viewer captures images. Display of players-specific information enhances the end-user experience. Detailed experiments are performed on 2096 diverse images captured using a digital camera and smartphone. The images contain players in different poses, expressions, and illuminations. Player face recognition module requires players faces to be frontal or up to ?350 of pose variation. The work demonstrates the great potential of computer vision based approaches for future development of AR applications.COMSATS Institute of Information Technolog

The design process for XR experiences

The act of telling stories has been a core part of the human experience since the days of cavemen standing around a fire to tell the story of the day's hunt (Balter). As new technologies are being developed, such as virtual reality headsets and cellphones with augmented reality capabilities, the designer's process has been challenged. We are increasingly living in a world where human experiences are separating from physical reality and moving toward an extended reality or XR. “Extended Reality” (XR) is the umbrella term used to describe VR (Virtual Reality), AR (Augmented Reality), and MR (Mixed Reality) as well as all future realities [any new experiences that might be created outside of the already existing realms] such technology, might bring. XR covers the full spectrum of real and virtual environments'' (Scribani). The challenge Designers are facing is how to effectively tell engaging stories using new and increasingly prevalent technologies within the XR envelope. Many experiences utilizing XR tend to focus on the technology and tools rather than the story. XR is the new “Wild West” in storytelling. There are no hard rules and no defined creative processes for crafting a successful experience within the technology. This thesis asks the question, can the theatrical design process help to create more successful XR experiences? Success being more social interaction and long-lasting engagement. Can a focus on human-centric stories make the experiences more engaging? In this thesis, I will be investigating three XR experiences and their varying design processes. The evaluation of this research will be qualitative rather than quantitative. The success or failure of these experiential XR designs will be determined through the lens of a collaborative theatrical designer.Theatre and Danc

Virtual camera synthesis for soccer game replays

n/

Using a virtual reality cricket simulator to explore the effects of pressure, competition anxiety on batting performance in cricket

Virtual reality (VR) has created opportunities to innovatively re-imagine the way we examine the relations between pressure, competition anxiety and performance. This study aimed to determine the efficacy of VR as a means of measuring the effects of competition anxiety when pressure manipulations are applied while participants bat in a cricket batting VR simulator. The twenty-eight male participants who took part in two experiments were divided into a high (14, mean age: 22.94, SD: 5.4) and a low skill group (14; mean age: 23.55, SD: 9.9). The aim of the first experiment was to validate the VR simulator as a tool that could capture differences in batting performance between a high and low skilled group. The results showed that high skill participants not only scored significantly higher run rates than low skill participants, but they outperformed the low skill group in all performance measures including higher incidences of correct foot placements that reflect better anticipatory responses. Having established the VR batting simulator as being a reliable tool for capturing batting dynamics, experiment 2 aimed to explore the effects of a pressure manipulation on competition anxiety and batting performance. All measures of competition anxiety were significantly greater for both groups in the high-pressure condition compared to the two low-pressure conditions (p &lt; 0.001). The magnitude of this effect was greater in the low skill group for cognitive (0.59) and somatic (0.794) anxiety. Despite anxiety levels significantly increasing in the high-pressure condition, no significant negative changes to batting performance were found for either group, with both groups actually demonstrating performance improvements. Overall, the findings show how a cricket batting virtual reality simulator can be used as a tool to measure the effects of pressure on competition anxiety and batting performance in tasks involving dynamic skill execution.</p