Selection of Temporal Aligned Video Frames for Video Stitching Application

Multi-view image/video stitching algorithm is an extensive research area in computer vision and image based rendering. Most researches focus on stitching the images from different views with assumption that those images have been already aligned in temporal domain. However it is not the case in real application. If the images from different views are not aligned in temporal domain, or in another words, not time synchronized, the corresponding feature points or regions will not be located correctly among different views, which will result in ghost objects appearing in the final stitching/rendering result. In this paper, we present an epipolar geometry consistency scoring scheme to guide temporal aligned video frame pair selection for multi-view video stitching application. Essentially, the proposed scheme allows us to determine whether a given pair of video frames is temporally aligned well for video stitching. Experimental results confirm that better video stitching results can be obtained with the proposed scheme in place.published_or_final_versio

MediaSync: Handbook on Multimedia Synchronization

This book provides an approachable overview of the most recent advances in the fascinating field of media synchronization (mediasync), gathering contributions from the most representative and influential experts. Understanding the challenges of this field in the current multi-sensory, multi-device, and multi-protocol world is not an easy task. The book revisits the foundations of mediasync, including theoretical frameworks and models, highlights ongoing research efforts, like hybrid broadband broadcast (HBB) delivery and users' perception modeling (i.e., Quality of Experience or QoE), and paves the way for the future (e.g., towards the deployment of multi-sensory and ultra-realistic experiences). Although many advances around mediasync have been devised and deployed, this area of research is getting renewed attention to overcome remaining challenges in the next-generation (heterogeneous and ubiquitous) media ecosystem. Given the significant advances in this research area, its current relevance and the multiple disciplines it involves, the availability of a reference book on mediasync becomes necessary. This book fills the gap in this context. In particular, it addresses key aspects and reviews the most relevant contributions within the mediasync research space, from different perspectives. Mediasync: Handbook on Multimedia Synchronization is the perfect companion for scholars and practitioners that want to acquire strong knowledge about this research area, and also approach the challenges behind ensuring the best mediated experiences, by providing the adequate synchronization between the media elements that constitute these experiences

Dynamic adaptation of streamed real-time E-learning videos over the internet

Even though the e-learning is becoming increasingly popular in the academic environment, the quality of synchronous e-learning video is still substandard and significant work needs to be done to improve it. The improvements have to be brought about taking into considerations both: the network requirements and the psycho- physical aspects of the human visual system. One of the problems of the synchronous e-learning video is that the head-and-shoulder video of the instructor is mostly transmitted. This video presentation can be made more interesting by transmitting shots from different angles and zooms. Unfortunately, the transmission of such multi-shot videos will increase packet delay, jitter and other artifacts caused by frequent changes of the scenes. To some extent these problems may be reduced by controlled reduction of the quality of video so as to minimise uncontrolled corruption of the stream. Hence, there is a need for controlled streaming of a multi-shot e-learning video in response to the changing availability of the bandwidth, while utilising the available bandwidth to the maximum. The quality of transmitted video can be improved by removing the redundant background data and utilising the available bandwidth for sending high-resolution foreground information. While a number of schemes exist to identify and remove the background from the foreground, very few studies exist on the identification and separation of the two based on the understanding of the human visual system. Research has been carried out to define foreground and background in the context of e-learning video on the basis of human psychology. The results have been utilised to propose methods for improving the transmission of e-learning videos. In order to transmit the video sequence efficiently this research proposes the use of Feed- Forward Controllers that dynamically characterise the ongoing scene and adjust the streaming of video based on the availability of the bandwidth. In order to satisfy a number of receivers connected by varied bandwidth links in a heterogeneous environment, the use of Multi-Layer Feed-Forward Controller has been researched. This controller dynamically characterises the complexity (number of Macroblocks per frame) of the ongoing video sequence and combines it with the knowledge of availability of the bandwidth to various receivers to divide the video sequence into layers in an optimal way before transmitting it into network. The Single-layer Feed-Forward Controller inputs the complexity (Spatial Information and Temporal Information) of the on-going video sequence along with the availability of bandwidth to a receiver and adjusts the resolution and frame rate of individual scenes to transmit the sequence optimised to give the most acceptable perceptual quality within the bandwidth constraints. The performance of the Feed-Forward Controllers have been evaluated under simulated conditions and have been found to effectively regulate the streaming of real-time e-learning videos in order to provide perceptually improved video quality within the constraints of the available bandwidth

Spatialising Photographic Archives

Recent computer research allows previously unforeseen analysis of photographic archives. For multiple shots of a scene, new digital techniques permit (1) the recovery of the location and angle, in 3D, of the camera that took each shot and (2) the reconstruction, in 3D, of much of the geometry of the original location. While the underlying algorithms of this approach are open source, the only available front-end to them is closed and proprietary. In this project, we will develop an open-source platform for experimentation with archives not previously thought of as 3D data-sets. As a constructive proof, we will analyze and re-interpret a key historical event as documented by a photojournalist, tying the spatial and visual components of the resulting data-set to contemporaneous text sources (news articles, etc) to form a uniquely hybrid form of scholarship. We will express these results in forms available to the computer science, visual arts, and photo-journalistic communities

Enabling reliable and power efficient real-time multimedia delivery over wireless sensor networks

There is an increasing need to run real-time multimedia applications, e.g. battle field and border surveillance, over Wireless Sensor Networks (WSNs). In WSNs, packet delivery exhibits high packet loss rate due to congestion, wireless channel high bit error rate, route failure, signal attenuation, etc... Flooding conventional packets over all sensors redundantly provides reliable delivery. However, flooding real-time multimedia packets is energy inefficient for power limited sensors and causes severe contentions affecting reliable delivery. We propose the Flooding Zone Initialization Protocol (FZIP) to enhance reliability and reduce power consumption of real-time multimedia flooding in WSNs. FZIP is a setup protocol which constrains flooding within a small subset of intermediate nodes called Flooding Zone (FZ). Also, we propose the Flooding Zone Control Protocol (FZCP) which monitors the session quality and dynamically changes the FZ size to adapt to current network state, thus providing a tradeoff of good quality and less power consumption

Advances in Sensors, Big Data and Machine Learning in Intelligent Animal Farming

Animal production (e.g., milk, meat, and eggs) provides valuable protein production for human beings and animals. However, animal production is facing several challenges worldwide such as environmental impacts and animal welfare/health concerns. In animal farming operations, accurate and efficient monitoring of animal information and behavior can help analyze the health and welfare status of animals and identify sick or abnormal individuals at an early stage to reduce economic losses and protect animal welfare. In recent years, there has been growing interest in animal welfare. At present, sensors, big data, machine learning, and artificial intelligence are used to improve management efficiency, reduce production costs, and enhance animal welfare. Although these technologies still have challenges and limitations, the application and exploration of these technologies in animal farms will greatly promote the intelligent management of farms. Therefore, this Special Issue will collect original papers with novel contributions based on technologies such as sensors, big data, machine learning, and artificial intelligence to study animal behavior monitoring and recognition, environmental monitoring, health evaluation, etc., to promote intelligent and accurate animal farm management

Decentralization in messaging applications with support for contactless interaction

Peer-to-peer communication has increasingly been gaining prevalence in people’s daily lives, with its widespread adoption being catalysed by technological advances. Although there have been strides for the inclusion of disabled individuals to ease communication between peers, people who suffer arm/hand impairments have little to no support in regular mainstream applications to efficiently communicate with other individuals. Additionally, as centralized systems have come into scrutiny regarding privacy and security, the development of alternative, decentralized solutions have increased, a movement pioneered by Bitcoin that culminated in the blockchain technology and its variants. Aiming towards expanding inclusivity in the messaging applications panorama, this project showcases an alternative on contactless human-computer interaction with support for disabled individuals with focus on the decentralized backend counterpart. Users of the application partake in a decentralized network based on a distributed hash table that is designed for secure communication (granted by a custom cryptographic messaging protocol) and exchange of data between peers. Such system is both resilient to tampering attacks and central points of failure (akin to blockchains), as well as having no long-term restrictions regarding scalability prospects, something that is a recurring issue in blockchain-based platforms. The conducted experiments showcase a level of performance similar to mainstream centralized approaches, outperforming blockchain-based decentralized applications on the delay between sending and receiving messages.A comunicação ponto-a-ponto tem cada vez mais ganhado prevalência na vida contemporânea de pessoas, tendo a sua adoção sido catalisada pelos avanços tecnológicos. Embora tenham havido desenvolvimentos relativamente à inclusão de indivíduos com deficiência para facilitar a comunicação entre pessoas, as que sofrem imparidades no braço/mão têm um suporte escasso em aplicações convencionais para comunicar de forma eficiente com outros sujeitos. Adicionalmente, à medida que sistemas centralizados têm atraído ceticismo relativamente à sua privacidade e segurança, o desenvolvimento de soluções descentralizadas e alternativas têm aumentado, um movimento iniciado pela Bitcoin que culminou na tecnologia de blockchain e as suas variantes. Tendo como objectivo expandir a inclusão no panorama de aplicações de messaging, este projeto pretende demonstrar uma alternativa na interação humano-computador sem contacto direto físico e com suporte para indivíduos com deficiência, com foco no componente backend decentralizado. Utilizadores da aplicação são inseridos num sistema decentralizado baseado numa hash table distribuída que foi desenhado para comunicação segura (providenciado por um protocolo de messaging criptográfico customizado) e para troca de dados entre utilizadores. Tal sistema é tanto resiliente a ataques de adulteração de dados como também a pontos centrais de falha (presente em blockains), não tendo adicionalmente restrições ao nível de escabilidade a longo-prazo, algo que é um problem recorrente em plataformas baseadas em blockchain. As avaliações e experiências realizadas neste projeto demonstram um nível de performance semelhante a abordagens centralizadas convencionais, tendo uma melhor prestação que aplicações descentralizadas baseadas em blockchain no que toca à diferença no tempo entre enviar e receber mensagens