Real-time Online Video Detection with Temporal Smoothing Transformers

Streaming video recognition reasons about objects and their actions in every frame of a video. A good streaming recognition model captures both long-term dynamics and short-term changes of video. Unfortunately, in most existing methods, the computational complexity grows linearly or quadratically with the length of the considered dynamics. This issue is particularly pronounced in transformer-based architectures. To address this issue, we reformulate the cross-attention in a video transformer through the lens of kernel and apply two kinds of temporal smoothing kernel: A box kernel or a Laplace kernel. The resulting streaming attention reuses much of the computation from frame to frame, and only requires a constant time update each frame. Based on this idea, we build TeSTra, a Temporal Smoothing Transformer, that takes in arbitrarily long inputs with constant caching and computing overhead. Specifically, it runs $6\times$ faster than equivalent sliding-window based transformers with 2,048 frames in a streaming setting. Furthermore, thanks to the increased temporal span, TeSTra achieves state-of-the-art results on THUMOS'14 and EPIC-Kitchen-100, two standard online action detection and action anticipation datasets. A real-time version of TeSTra outperforms all but one prior approaches on the THUMOS'14 dataset.Comment: ECCV 2022; Code available at https://github.com/zhaoyue-zephyrus/TeSTr

Blockchain for video streaming : opportunities, challenges and open issues

Blockchain, Quality of Experience (QoE), and Video Streaming have all received much attention from both academia and industry so far, although they have not been jointly addressed for prospective applications yet. While the industry has already adopted blockchain-based video streaming platforms, other stakeholders, e.g., academia, government, regulators, and service providers, could contribute more to develop protocols, technologies, and standards to help grow this niche technology and support its implementation in media streaming applications. This paper reviews the current technologies, industrial advancements, and critically identifies the current research activities and future research opportunities

Analysis of adaptive video streaming for users with demand heterogeneity

The revolution of smart phones and other electronic gadgets have increased the attention given to video streaming phenomena. The demand of video quality has also augmented as the number of users keeps on growing. This has toughened the distribution of bandwidth. It is becoming more challenging for service providers to cope with varied demand of video quality. In real life, users change frequently their demand of video quality. To mitigate these problems, an adaptive streaming approach was proposed to solve the problem related to users’ heterogeneous demands. A linear programing approach based on demand and supply was used to analyse and sustain user with low bandwidth. Enhancement Fractional Participative Scheme (EFPS) based on bandwidth contribution was explored. Using JSVM9.14 software, the simulated result shows that the optimized algorithms and enhancement structure improve the performance

Real-Time Neural Video Recovery and Enhancement on Mobile Devices

As mobile devices become increasingly popular for video streaming, it's crucial to optimize the streaming experience for these devices. Although deep learning-based video enhancement techniques are gaining attention, most of them cannot support real-time enhancement on mobile devices. Additionally, many of these techniques are focused solely on super-resolution and cannot handle partial or complete loss or corruption of video frames, which is common on the Internet and wireless networks. To overcome these challenges, we present a novel approach in this paper. Our approach consists of (i) a novel video frame recovery scheme, (ii) a new super-resolution algorithm, and (iii) a receiver enhancement-aware video bit rate adaptation algorithm. We have implemented our approach on an iPhone 12, and it can support 30 frames per second (FPS). We have evaluated our approach in various networks such as WiFi, 3G, 4G, and 5G networks. Our evaluation shows that our approach enables real-time enhancement and results in a significant increase in video QoE (Quality of Experience) of 24\% - 82\% in our video streaming system

No-reference video quality estimation based on machine learning for passive gaming video streaming applications

Recent years have seen increasing growth and popularity of gaming services, both interactive and passive. While interactive gaming video streaming applications have received much attention, passive gaming video streaming, in-spite of its huge success and growth in recent years, has seen much less interest from the research community. For the continued growth of such services in the future, it is imperative that the end user gaming quality of experience (QoE) is estimated so that it can be controlled and maximized to ensure user acceptance. Previous quality assessment studies have shown not so satisfactory performance of existing No-reference (NR) video quality assessment (VQA) metrics. Also, due to the inherent nature and different requirements of gaming video streaming applications, as well as the fact that gaming videos are perceived differently from non-gaming content (as they are usually computer generated and contain artificial/synthetic content), there is a need for application specific light-weight, no-reference gaming video quality prediction models. In this paper, we present two NR machine learning based quality estimation models for gaming video streaming, NR-GVSQI and NR-GVSQE, using NR features such as bitrate, resolution, blockiness, etc. We evaluate their performance on different gaming video datasets and show that the proposed models outperform the current state-of-the-art no-reference metrics, while also reaching a prediction accuracy comparable to the best known full reference metric

Secured Technique of AMOV and ESOV in the Clouds

The available hardware and technology for consumers and service providers today allow for advanced multimedia services over IP-based networks. Hence,the popularity of video and audio streaming services such as Video-on-Demand (VoD),The user demand for videos over the mobile devices through wireless links this wireless links capacity cannot be corporate with the traffic demand. As delay between traffic demand and link capacity, with link conditions, low ouput quality of service and sending data on this media result in buffering time . in this paper we propose a new secure mobile video streaming framework AMoV (adaptive mobile video streaming) and ESoV(efficient social video sharing) are the terms which are currently gaining the attention of variety of computer users and researchers. While enjoying the multimedia services like videos and images, the basic quandary faced by any individual is the progressive downloading or the buffering of the videos. As the researches are focusing on various technologies in said issue, very least focus is given on to the security issues present in these technologies. The basic idea behind this paper is to study and to survey the literature and to propose the security aspects in related field

Affective video and problem solving within a Web-environment

Currently there is a growing interest in Web-based multimedia learning environments, particularly those making use of asynchronous streaming video. This interest motivates renewed attention to properties of video for educational purposes. A typical property of video is its emotion-evoking potential. Research by Isen, Daubman, and Nowicki (1987), Kaufmann and Vosburg (1997) and by Vosburg (1998) on video-evoked positive or negative mood states inspired a research project on the didactical functionality of emotion-evoking video materials in relationship to (educational) problem solving tasks within a Web-based environment. The results show that the video materials that were used in the experiment induced the expected positive or negative mood. Differential effects of positive or negative mood for problem solving tasks, however, were not observed. This outcome is discussed in the context of the findings of the above-mentioned authors

A Hybrid of Adaptation and Dynamic Routing based on SDN for Improving QoE in HTTP Adaptive VBR Video Streaming

Recently, HTTP Adaptive Streaming HAS has received significant attention from both industry and academia based on its ability to enhancing media streaming services over the Internet. Recent research solutions that have tried to improve HAS by adaptation at the client side only may not be completely effective without interacting with routing decisions in the upper layers. In this paper, we address the aforementioned issue by proposing a dynamic bandwidth allocation and management architecture for streaming video flows to improve users satisfaction. We also introduce an initial cross layer hybrid method that combines quality adaptation of variable bitrate video streaming over the HTTP protocol at the client side and SDN based dynamical routing. This scheme is enabled by the Software Defined Networking architecture that is now being considered as an emerging paradigm that disassociates the forwarding process from the routing process. SDN brings flexibility and the ability to flexibly change routing solutions, in turn resulting in dynamically improving the services provided in the application layer. Our experimental results show that the proposed solution offers significantly higher overall bitrates as well as smoother viewing experience than existing methods.Comment: 14 pages, 17 figures, IJCSNS International Journal of Computer Science and Network Security, http://paper.ijcsns.org/07_book/201907/20190708.pd

Accessing and engaging with video streams for educational purposes: experiences, issues and concerns

Video streaming has the potential to offer tutors a more flexible and accessible means of incorporating moving images into learning resources for their students than conventional video. Consideration is given to this assertion by drawing upon the experiences of staff and evidence from students at the University of Southampton in the use of a video, Back Care for Health Professionals, before and after it was streamed. The resulting case study highlights various issues and concerns, both logistical and pedagogic. These include ease of access, the form and frequency of guidance with respect to technical matters, the use of multiple channels of communication to convey key messages about the availability and value of the video, and the provision of demonstrations or 'tasters'. In other words, what some might regard as the 'softer' aspects of technological developments should receive at least as much attention as the 'harder'