MediaSync: Handbook on Multimedia Synchronization

This book provides an approachable overview of the most recent advances in the fascinating field of media synchronization (mediasync), gathering contributions from the most representative and influential experts. Understanding the challenges of this field in the current multi-sensory, multi-device, and multi-protocol world is not an easy task. The book revisits the foundations of mediasync, including theoretical frameworks and models, highlights ongoing research efforts, like hybrid broadband broadcast (HBB) delivery and users' perception modeling (i.e., Quality of Experience or QoE), and paves the way for the future (e.g., towards the deployment of multi-sensory and ultra-realistic experiences). Although many advances around mediasync have been devised and deployed, this area of research is getting renewed attention to overcome remaining challenges in the next-generation (heterogeneous and ubiquitous) media ecosystem. Given the significant advances in this research area, its current relevance and the multiple disciplines it involves, the availability of a reference book on mediasync becomes necessary. This book fills the gap in this context. In particular, it addresses key aspects and reviews the most relevant contributions within the mediasync research space, from different perspectives. Mediasync: Handbook on Multimedia Synchronization is the perfect companion for scholars and practitioners that want to acquire strong knowledge about this research area, and also approach the challenges behind ensuring the best mediated experiences, by providing the adequate synchronization between the media elements that constitute these experiences

Social Computing: An Overview

A collection of technologies termed social computing is driving a dramatic evolution of the Web, matching the dot-com era in growth, excitement, and investment. All of these share high degree of community formation, user level content creation, and computing, and a variety of other characteristics. We provide an overview of social computing and identify salient characteristics. We argue that social computing holds tremendous disruptive potential in the business world and can significantly impact society, and outline possible changes in organized human action that could be brought about. Social computing can also have deleterious effects associated with it, including security issues. We suggest that social computing should be a priority for researchers and business leaders and illustrate the fundamental shifts in communication, computing, collaboration, and commerce brought about by this trend

Identification, synchronisation and composition of user-generated videos

Cotutela Universitat Politècnica de Catalunya i Queen Mary University of LondonThe increasing availability of smartphones is facilitating people to capture videos of their experience when attending events such as concerts, sports competitions and public rallies. Smartphones are equipped with inertial sensors which could be beneficial for event understanding. The captured User-Generated Videos (UGVs) are made available on media sharing websites. Searching and mining of UGVs of the same event are challenging due to inconsistent tags or incorrect timestamps. A UGV recorded from a fixed location contains monotonic content and unintentional camera motions, which may make it less interesting to playback. In this thesis, we propose the following identification, synchronisation and video composition frameworks for UGVs. We propose a framework for the automatic identification and synchronisation of unedited multi-camera UGVs within a database. The proposed framework analyses the sound to match and cluster UGVs that capture the same spatio-temporal event, and estimate their relative time-shift to temporally align them. We design a novel descriptor derived from the pairwise matching of audio chroma features of UGVs. The descriptor facilitates the definition of a classification threshold for automatic query-by-example event identification. We contribute a database of 263 multi-camera UGVs of 48 real-world events. We evaluate the proposed framework on this database and compare it with state-of-the-art methods. Experimental results show the effectiveness of the proposed approach in the presence of audio degradations (channel noise, ambient noise, reverberations). Moreover, we present an automatic audio and visual-based camera selection framework for composing uninterrupted recording from synchronised multi-camera UGVs of the same event. We design an automatic audio-based cut-point selection method that provides a common reference for audio and video segmentation. To filter low quality video segments, spatial and spatio-temporal assessments are computed. The framework combines segments of UGVs using a rank-based camera selection strategy by considering visual quality scores and view diversity. The proposed framework is validated on a dataset of 13 events (93~UGVs) through subjective tests and compared with state-of-the-art methods. Suitable cut-point selection, specific visual quality assessments and rank-based camera selection contribute to the superiority of the proposed framework over the existing methods. Finally, we contribute a method for Camera Motion Detection using Gyroscope for UGVs captured from smartphones and design a gyro-based quality score for video composition. The gyroscope measures the angular velocity of the smartphone that can be use for camera motion analysis. We evaluate the proposed camera motion detection method on a dataset of 24 multi-modal UGVs captured by us, and compare it with existing visual and inertial sensor-based methods. By designing a gyro-based score to quantify the goodness of the multi-camera UGVs, we develop a gyro-based video composition framework. A gyro-based score substitutes the spatial and spatio-temporal scores and reduces the computational complexity. We contribute a multi-modal dataset of 3 events (12~UGVs), which is used to validate the proposed gyro-based video composition framework.El incremento de la disponibilidad de teléfonos inteligentes o smartphones posibilita a la gente capturar videos de sus experiencias cuando asisten a eventos así como como conciertos, competiciones deportivas o mítines públicos. Los Videos Generados por Usuarios (UGVs) pueden estar disponibles en sitios web públicos especializados en compartir archivos. La búsqueda y la minería de datos de los UGVs del mismo evento son un reto debido a que los etiquetajes son incoherentes o las marcas de tiempo erróneas. Por otra parte, un UGV grabado desde una ubicación fija, contiene información monótona y movimientos de cámara no intencionados haciendo menos interesante su reproducción. En esta tesis, se propone una identificación, sincronización y composición de tramas de vídeo para UGVs. Se ha propuesto un sistema para la identificación y sincronización automática de UGVs no editados provenientes de diferentes cámaras dentro de una base de datos. El sistema propuesto analiza el sonido con el fin de hacerlo coincidir e integrar UGVs que capturan el mismo evento en el espacio y en el tiempo, estimando sus respectivos desfases temporales y alinearlos en el tiempo. Se ha diseñado un nuevo descriptor a partir de la coincidencia por parejas de características de la croma del audio de los UGVs. Este descriptor facilita la determinación de una clasificación por umbral para una identificación de eventos automática basada en búsqueda mediante ejemplo (en inglés, query by example). Se ha contribuido con una base de datos de 263 multi-cámaras UGVs de un total de 48 eventos reales. Se ha evaluado la trama propuesta en esta base de datos y se ha comparado con los métodos elaborados en el estado del arte. Los resultados experimentales muestran la efectividad del enfoque propuesto con la presencia alteraciones en el audio. Además, se ha presentado una selección automática de tramas en base a la reproducción de video y audio componiendo una grabación ininterrumpida de multi-cámaras UGVs sincronizadas en el mismo evento. También se ha diseñado un método de selección de puntos de corte automático basado en audio que proporciona una referencia común para la segmentación de audio y video. Con el fin de filtrar segmentos de videos de baja calidad, se han calculado algunas medidas espaciales y espacio-temporales. El sistema combina segmentos de UGVs empleando una estrategia de selección de cámaras basadas en la evaluación a través de un ranking considerando puntuaciones de calidad visuales y diversidad de visión. El sistema propuesto se ha validado con un conjunto de datos de 13 eventos (93 UGVs) a través de pruebas subjetivas y se han comparado con los métodos elaborados en el estado del arte. La selección de puntos de corte adecuados, evaluaciones de calidad visual específicas y la selección de cámara basada en ranking contribuyen en la mejoría de calidad del sistema propuesto respecto a otros métodos existentes. Finalmente, se ha realizado un método para la Detección de Movimiento de Cámara usando giróscopos para las UGVs capturadas desde smartphones y se ha diseñado un método de puntuación de calidad basada en el giro. El método de detección de movimiento de la cámara con una base de datos de 24 UGVs multi-modales y se ha comparado con los métodos actuales basados en visión y sistemas inerciales. A través del diseño de puntuación para cuantificar con el giróscopo cuán bien funcionan los sistemas de UGVs con multi-cámara, se ha desarrollado un sistema de composición de video basada en el movimiento del giroscopio. Este sistema basado en la puntuación a través del giróscopo sustituye a los sistemas de puntuaciones basados en parámetros espacio-temporales reduciendo la complejidad computacional. Además, se ha contribuido con un conjunto de datos de 3 eventos (12 UGVs), que se han empleado para validar los sistemas de composición de video basados en giróscopo.Postprint (published version

A case study in social media mashup concept validation

Sosiaalinen media korostaa käyttäjien välistä vuorovaikutusta ja käyttäjien luoman sisällön mahdollistamista ja jakamista. Sosiaaliseen mediaan kuuluvat myös sosiaaliset verkostot, jotka yhdistävät ihmisiä. Mashupit taas ovat palveluja, jotka yhdistävät tietoa eri lähteistä ja näyttävät tämän tiedon uudessa kontekstissa. Tässä työssä käsitellään sosiaalista verkostoa nimeltä AmiCarta. AmiCarta keskittyy paikkatiedon hyödyntämiseen sosiaalisessa verkostossa. Työn tarkoituksena on validoida palvelukonsepti käyttäjillä. Työssä analysoidaan validointiin käytettyjä metodeja sekä käyttäjätestien tuloksia. Käyttäjätestit koostuvat kukin kolmesta osasta. Käyttäjille annetaan prototyyppi tuotteen käyttöliittymästä, jonka he käyvät läpi. Prototyyppi on luotu paperiprototyypeillä ja toimii wizard-of-oz metodilla. Protoyypin tarkastelun jälkeen käyttäjiä haastatellaan puolistrukturoidulla haastattelulla ja esitetään kyselylomake. Käytetyt metodit tuottivat pääasiassa kvalitatiivista tietoa. Prototyypin käytöllä oli haittavaikutuksena se, että se tuotti pääasiassa tietoa olemassa olevista ominaisuuksista, eikä juurikaan auttanut uusien ominaisuuksien ideoinnissa. Kvalitatiiviset tulokset antoivat ideoita siitä, mihin suuntaan konseptia tulisi viedä. Kyselylomakkeen tuottamat kvantitaaviset tulokset taas olivat käytännöllisiä ominaisuuksien vertailussa. Tutkimuksen perusteella konseptin perusajatus muuttui karttapalvelusta, johon on lisätty erilaisia ominaisuuksia palveluun, joka perustuu kommunikaation helpottamiseen ja tukemiseen paikkatiedolla.In social media user-generated content and sharing are emphasized. As a subcategory for social media there are social networks that connect people. Mashups are applications that combine data from existing sources and display the data in a new context. This study considers the case of AmiCarta, a social network service that focuses on location information. The study focuses on validating the concept and evaluating the used methods for validation as well as the obtained results. The concept is validated with user tests. Each user test was conducted in three parts. In the first part, the user was demonstrated a mock-up prototype of the mobile user interface for the product that was created using paper prototyping and theprototype responds to the user in a wizard-of-oz implementation. The second phase consisted of a semi-structured interview and the third phase of a questionnaire. The methods of the study produced mainly qualitative information. As a drawback, the use of a mock-up prototype mainly generated information about the existing features of the concept. Qualitative results gave insight on how the concept could be further developed. The quantitative results of the questionnaire were useful in ranking the features. As a result of the study, the focus of the service was moved from a map application that has different features to an application that enables communication between users and supports this communication with the users' location information

Socially-Aware Multimedia Authoring

Bulterman, D.C.A. [Promotor]Cesar, P.S. [Copromotor

Head Up Games : on the design, creation and evaluation of interactive outdoor games for children

This thesis proposes a new genre of outdoor games for children, namely Head Up Games. The concept was inspired by the observation that existing pervasive outdoor games for children were mostly played head down, as the predominantly screen-based interaction of existing games required constant attention of the children. First, the vision of Head Up Games is described and illustrated with several design cases (Chapter 2). In contrast to the head down games, Head Up Games aim to encourage and support rich social interaction and physical activity, play behaviors that are similar to play behaviors seen in traditional outdoor games (such as tag and hide-and-seek). The design process of Head Up Games poses several challenges. In User Centered Design it is commonly accepted to start the development of a new product using low-fi mock-ups, e.g., paper prototypes, and evaluate these with end-users. In the case of Head Up Games this proved to be difficult, as the emerging game experience is significantly altered when using paper prototypes. Therefore, a study was carried out that used high-fi prototypes, i.e. working, interactive, prototypes, from a very early stage in the design process (Chapter 3). This way, the effect of interactions on the game experience can be addressed earlier and better in the design process. Furthemore, having access to technology early in the design process, allows designers to better explore the design space. However, designers often do not possess adequate skills to quickly prototype interactive products, particularly products that need to be evaluated in an outdoor context. Such a development is often costly and time-consuming. Therefore, the RaPIDO platform was developed (Chapter 4). The platform not only includes the appropriate hardware for creating outdoor games, but is also bundled with software libraries, to allow designers not specifically trained in software engineering to adopt the platform easily. RaPIDO was evaluated using a case study methodology with two Industrial Design master students. The evaluation not only focused on the usability of the platform, but, more importantly, how the use of the platform affected the design process. The main conclusion of the study was that the designers indeed were able to rapidly create mobile games, and that the hardware offered was suitable for creating outdoor games. Furthermore, issues were identified with regard to writing the game software, e.g., managing the complexity of the software. Finally, for evaluating Head Up Games with children two methods were applied: the Outdoor Play Observation Scheme (OPOS) was used to quantify the intended play behavior. Furthermore, GroupSorter was developed to provide a framework to interview a group of children simultaneously, resulting in qualitative comments. Both OPOS and GroupSorter were applied for evaluating three Head Up Games, which are described in Chapter 5

The digitally 'Hand Made' object

This article will outline the author’s investigations of types of computer interfaces in practical three-dimensional design practice. The paper contains a description of two main projects in glass and ceramic tableware design, using a Microscribe G2L digitising arm as an interface to record three-dimensional spatial\ud design input.\ud \ud The article will provide critical reflections on the results of the investigations and will argue that new approaches in digital design interfaces could have relevance in developing design methods which incorporate more physical ‘human’ expressions in a three-dimensional design practice. The research builds on concepts indentified in traditional craft practice as foundations for constructing new types of creative practices based on the use of digital technologies, as outlined by McCullough (1996)

Self-adaptation via concurrent multi-action evaluation for unknown context

Context-aware computing has been attracting growing attention in recent years. Generally, there are several ways for a context-aware system to select a course of action for a particular change of context. One way is for the system developers to encompass all possible context changes in the domain knowledge. Other methods include system inferences and adaptive learning whereby the system executes one action and evaluates the outcome and self-adapts/self-learns based on that. However, in situations where a system encounters unknown contexts, the iterative approach would become unfeasible when the size of the action space increases. Providing efficient solutions to this problem has been the main goal of this research project. Based on the developed abstract model, the designed methodology replaces the single action implementation and evaluation by multiple actions implemented and evaluated concurrently. This parallel evaluation of actions speeds up significantly the evolution time taken to select the best action suited to unknown context compared to the iterative approach. The designed and implemented framework efficiently carries out concurrent multi-action evaluation when an unknown context is encountered and finds the best course of action. Two concrete implementations of the framework were carried out demonstrating the usability and adaptability of the framework across multiple domains. The first implementation was in the domain of database performance tuning. The concrete implementation of the framework demonstrated the ability of concurrent multi-action evaluation technique to performance tune a database when performance is regressed for an unknown reason. The second implementation demonstrated the ability of the framework to correctly determine the threshold price to be used in a name-your-own-price channel when an unknown context is encountered. In conclusion the research introduced a new paradigm of a self-adaptation technique for context-aware application. Among the existing body of work, the concurrent multi-action evaluation is classified under the abstract concept of experiment-based self-adaptation techniques