109 research outputs found
Interactive Video Search
With an increasing amount of video data in our daily life, the need for content-based search in videos increases as well. Though a lot of research has been spent on video retrieval tools and methods which allow for automatic search in videos through content-based queries, still the performance of automatic video retrieval is far from optimal. In this tutorial we discussed (i) proposed solutions for improved video content navigation, (ii) typical interaction of content-based querying features, and (iii) advanced video content visualization methods. Moreover, we discussed interactive video search systems and ways to evaluate their performance
Video browsing interfaces and applications: a review
We present a comprehensive review of the state of the art in video browsing and retrieval systems, with special emphasis on interfaces and applications. There has been a significant increase in activity (e.g., storage, retrieval, and sharing) employing video data in the past decade, both for personal and professional use. The ever-growing amount of video content available for human consumption and the inherent characteristics of video data—which, if presented in its raw format, is rather unwieldy and costly—have become driving forces for the development of more effective solutions to present video contents and allow rich user interaction. As a result, there are many contemporary research efforts toward developing better video browsing solutions, which we summarize. We review more than 40 different video browsing and retrieval interfaces and classify them into three groups: applications that use video-player-like interaction, video retrieval applications, and browsing solutions based on video surrogates. For each category, we present a summary of existing work, highlight the technical aspects of each solution, and compare them against each other
IMuR 2022 Introduction to the 2nd Workshop on Interactive Multimedia Retrieval
The retrieval of multimedia content remains a difficult problem where a high accuracy or specificity can often only be achieved interactively, with a user working closely and iteratively with a retrieval system. While there exist several venues for the exchange of insights in the area of information retrieval in general and multimedia retrieval specifically, there is little discussion on such interactive retrieval approaches. The Workshop on Interactive Multimedia Retrieval offers such a venue. Held for the 2nd time in 2022, it attracted a diverse set of contributions, six of which were accepted for presentation. The following provides a brief overview of the workshop itself as well as the contributions of 2022
Open Challenges of Interactive Video Search and Evaluation
During the last 10 years of Video Browser Showdown (VBS), there were many different approaches tested for known-item search and ad-hoc search tasks. Undoubtedly, teams incorporating state-of-the-art models from the machine learning domain had an advantage over teams focusing just on interactive interfaces. On the other hand, VBS results indicate that effective means of interaction with a search system is still necessary to accomplish challenging search tasks. In this tutorial, we summarize successful deep models tested at the Video Browser Showdown as well as interfaces designed on top of corresponding distance/similarity spaces. Our broad experience with competition organization and evaluation will be presented as well, focusing on promising findings and also challenging problems from the most recent iterations of the Video Browser Showdown
Content-Adaptive Variable Framerate Encoding Scheme for Green Live Streaming
Adaptive live video streaming applications use a fixed predefined
configuration for the bitrate ladder with constant framerate and encoding
presets in a session. However, selecting optimized framerates and presets for
every bitrate ladder representation can enhance perceptual quality, improve
computational resource allocation, and thus, the streaming energy efficiency.
In particular, low framerates for low-bitrate representations reduce
compression artifacts and decrease encoding energy consumption. In addition, an
optimized preset may lead to improved compression efficiency. To this light,
this paper proposes a Content-adaptive Variable Framerate (CVFR) encoding
scheme, which offers two modes of operation: ecological (ECO) and high-quality
(HQ). CVFR-ECO optimizes for the highest encoding energy savings by predicting
the optimized framerate for each representation in the bitrate ladder. CVFR-HQ
takes it further by predicting each representation's optimized
framerate-encoding preset pair using low-complexity discrete cosine transform
energy-based spatial and temporal features for compression efficiency and
sustainable storage. We demonstrate the advantage of CVFR using the x264
open-source video encoder. The results show that CVFR-ECO yields an average
PSNR and VMAF increase of 0.02 dB and 2.50 points, respectively, for the same
bitrate, compared to the fastest preset highest framerate encoding. CVFR-ECO
also yields an average encoding and storage energy consumption reduction of
34.54% and 76.24%, considering a just noticeable difference (JND) of six VMAF
points. In comparison, CVFR-HQ yields an average increase in PSNR and VMAF of
2.43 dB and 10.14 points, respectively, for the same bitrate. Finally, CVFR-HQ
resulted in an average reduction in storage energy consumption of 83.18%,
considering a JND of six VMAF points
Relevance-Based Compression of Cataract Surgery Videos
In the last decade, the need for storing videos from cataract surgery has
increased significantly. Hospitals continue to improve their imaging and
recording devices (e.g., microscopes and cameras used in microscopic surgery,
such as ophthalmology) to enhance their post-surgical processing efficiency.
The video recordings enable a lot of user-cases after the actual surgery, for
example, teaching, documentation, and forensics. However, videos recorded from
operations are typically stored in the internal archive without any
domain-specific compression, leading to a massive storage space consumption. In
this work, we propose a relevance-based compression scheme for videos from
cataract surgery, which is based on content specifics of particular cataract
surgery phases. We evaluate our compression scheme with three state-of-the-art
video codecs, namely H.264/AVC, H.265/HEVC, and AV1, and ask medical experts to
evaluate the visual quality of encoded videos. Our results show significant
savings, in particular up to 95.94% when using H.264/AVC, up to 98.71% when
using H.265/HEVC, and up to 98.82% when using AV1.Comment: 11 pages, 5 figures, 3 table
- …