Towards Structured Analysis of Broadcast Badminton Videos

Sports video data is recorded for nearly every major tournament but remains archived and inaccessible to large scale data mining and analytics. It can only be viewed sequentially or manually tagged with higher-level labels which is time consuming and prone to errors. In this work, we propose an end-to-end framework for automatic attributes tagging and analysis of sport videos. We use commonly available broadcast videos of matches and, unlike previous approaches, does not rely on special camera setups or additional sensors. Our focus is on Badminton as the sport of interest. We propose a method to analyze a large corpus of badminton broadcast videos by segmenting the points played, tracking and recognizing the players in each point and annotating their respective badminton strokes. We evaluate the performance on 10 Olympic matches with 20 players and achieved 95.44% point segmentation accuracy, 97.38% player detection score ([email protected]), 97.98% player identification accuracy, and stroke segmentation edit scores of 80.48%. We further show that the automatically annotated videos alone could enable the gameplay analysis and inference by computing understandable metrics such as player's reaction time, speed, and footwork around the court, etc.Comment: 9 page

Automatic Mobile Video Remixing and Collaborative Watching Systems

In the thesis, the implications of combining collaboration with automation for remix creation are analyzed. We first present a sensor-enhanced Automatic Video Remixing System (AVRS), which intelligently processes mobile videos in combination with mobile device sensor information. The sensor-enhanced AVRS system involves certain architectural choices, which meet the key system requirements (leverage user generated content, use sensor information, reduce end user burden), and user experience requirements. Architecture adaptations are required to improve certain key performance parameters. In addition, certain operating parameters need to be constrained, for real world deployment feasibility. Subsequently, sensor-less cloud based AVRS and low footprint sensorless AVRS approaches are presented. The three approaches exemplify the importance of operating parameter tradeoffs for system design. The approaches cover a wide spectrum, ranging from a multimodal multi-user client-server system (sensor-enhanced AVRS) to a mobile application which can automatically generate a multi-camera remix experience from a single video. Next, we present the findings from the four user studies involving 77 users related to automatic mobile video remixing. The goal was to validate selected system design goals, provide insights for additional features and identify the challenges and bottlenecks. Topics studied include the role of automation, the value of a video remix as an event memorabilia, the requirements for different types of events and the perceived user value from creating multi-camera remix from a single video. System design implications derived from the user studies are presented. Subsequently, sport summarization, which is a specific form of remix creation is analyzed. In particular, the role of content capture method is analyzed with two complementary approaches. The first approach performs saliency detection in casually captured mobile videos; in contrast, the second one creates multi-camera summaries from role based captured content. Furthermore, a method for interactive customization of summary is presented. Next, the discussion is extended to include the role of users’ situational context and the consumed content in facilitating collaborative watching experience. Mobile based collaborative watching architectures are described, which facilitate a common shared context between the participants. The concept of movable multimedia is introduced to highlight the multidevice environment of current day users. The thesis presents results which have been derived from end-to-end system prototypes tested in real world conditions and corroborated with extensive user impact evaluation

Context-aware Services for Mobile Devices: From Architecture Design to Empirical Inference

Currently, mobile devices are aware of user position, which can be provided to mobile apps for the development of tailored services known as Location-Based Services. Further advances on current Location-based Services (LBS), i.e. using any other information from the user such as gender, music preferences etc, may lead to transition from a Location-Based environment to a fully developed ContextAware environment.The current trend towards Context-aware Services (CAS) is reflected in academic research since more than twenty years as well as in the progress in Software Development Kits (SDKs) of the main mobile operating systems, where CAS frameworks are currently being used. However, there is no community agreement for modelling context CAS and little is known about the architecture of these context management frameworks of the mobile operating systems.Based on previous research in the area of CAS, I establish and analyse a reasoning architecture, the Context Engine (CE), that enables the main steps of designing and implementing context-aware services. The chief utility of CAS is their ability to formulate and encapsulate information, obtain user context through context acquisition tools and distribute it to third-party applications that build personalised services based on the provided information. The CE has the responsibility of selecting the optimal context acquisition tool to solve a concrete problem which is discussed in this dissertation.Furthermore, this thesis contributes to the development of context inference tools by studying two particular cases. The first case aims at inferring user (semantic) location information based on mobile phone usage data. This first case has been carried out in collaboration with Microsoft Finland, which provides a similar context inference solution to mobile developers through their Software Development Kit (SDK). The second case aims at inferring user information based on social network information, i.e. infer user information based on his or her connections. Both studies yield positive results and have the potential to be extended to obtain better context acquisition tools and, therefore, better user context