User-centred video abstraction

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University LondonThe rapid growth of digital video content in recent years has imposed the need for the development of technologies with the capability to produce condensed but semantically rich versions of the input video stream in an effective manner. Consequently, the topic of Video Summarisation is becoming increasingly popular in multimedia community and numerous video abstraction approaches have been proposed accordingly. These recommended techniques can be divided into two major categories of automatic and semi-automatic in accordance with the required level of human intervention in summarisation process. The fully-automated methods mainly adopt the low-level visual, aural and textual features alongside the mathematical and statistical algorithms in furtherance to extract the most significant segments of original video. However, the effectiveness of this type of techniques is restricted by a number of factors such as domain-dependency, computational expenses and the inability to understand the semantics of videos from low-level features. The second category of techniques however, attempts to alleviate the quality of summaries by involving humans in the abstraction process to bridge the semantic gap. Nonetheless, a single user’s subjectivity and other external contributing factors such as distraction will potentially deteriorate the performance of this group of approaches. Accordingly, in this thesis we have focused on the development of three user-centred effective video summarisation techniques that could be applied to different video categories and generate satisfactory results. According to our first proposed approach, a novel mechanism for a user-centred video summarisation has been presented for the scenarios in which multiple actors are employed in the video summarisation process in order to minimise the negative effects of sole user adoption. Based on our recommended algorithm, the video frames were initially scored by a group of video annotators ‘on the fly’. This was followed by averaging these assigned scores in order to generate a singular saliency score for each video frame and, finally, the highest scored video frames alongside the corresponding audio and textual contents were extracted to be included into the final summary. The effectiveness of our approach has been assessed by comparing the video summaries generated based on our approach against the results obtained from three existing automatic summarisation tools that adopt different modalities for abstraction purposes. The experimental results indicated that our proposed method is capable of delivering remarkable outcomes in terms of Overall Satisfaction and Precision with an acceptable Recall rate, indicating the usefulness of involving user input in the video summarisation process. In an attempt to provide a better user experience, we have proposed our personalised video summarisation method with an ability to customise the generated summaries in accordance with the viewers’ preferences. Accordingly, the end-user’s priority levels towards different video scenes were captured and utilised for updating the average scores previously assigned by the video annotators. Finally, our earlier proposed summarisation method was adopted to extract the most significant audio-visual content of the video. Experimental results indicated the capability of this approach to deliver superior outcomes compared with our previously proposed method and the three other automatic summarisation tools. Finally, we have attempted to reduce the required level of audience involvement for personalisation purposes by proposing a new method for producing personalised video summaries. Accordingly, SIFT visual features were adopted to identify the video scenes’ semantic categories. Fusing this retrieved data with pre-built users’ profiles, personalised video abstracts can be created. Experimental results showed the effectiveness of this method in delivering superior outcomes comparing to our previously recommended algorithm and the three other automatic summarisation techniques

Usabilidade na Web e usabilidade na televisão interactiva

Tese de mestrado. Tecnologia Multimédia. 2005. Faculdade de Engenharia. Universidade do Porto, Departamento de Ciências da Comunicação, Artes e Tecnologias da Informação. Universidade Lusófona de Humanidades e Tecnologia

Computergestützte Inhaltsanalyse von digitalen Videoarchiven

Der Übergang von analogen zu digitalen Videos hat in den letzten Jahren zu großen Veränderungen innerhalb der Filmarchive geführt. Insbesondere durch die Digitalisierung der Filme ergeben sich neue Möglichkeiten für die Archive. Eine Abnutzung oder Alterung der Filmrollen ist ausgeschlossen, so dass die Qualität unverändert erhalten bleibt. Zudem wird ein netzbasierter und somit deutlich einfacherer Zugriff auf die Videos in den Archiven möglich. Zusätzliche Dienste stehen den Archivaren und Anwendern zur Verfügung, die erweiterte Suchmöglichkeiten bereitstellen und die Navigation bei der Wiedergabe erleichtern. Die Suche innerhalb der Videoarchive erfolgt mit Hilfe von Metadaten, die weitere Informationen über die Videos zur Verfügung stellen. Ein großer Teil der Metadaten wird manuell von Archivaren eingegeben, was mit einem großen Zeitaufwand und hohen Kosten verbunden ist. Durch die computergestützte Analyse eines digitalen Videos ist es möglich, den Aufwand bei der Erzeugung von Metadaten für Videoarchive zu reduzieren. Im ersten Teil dieser Dissertation werden neue Verfahren vorgestellt, um wichtige semantische Inhalte der Videos zu erkennen. Insbesondere werden neu entwickelte Algorithmen zur Erkennung von Schnitten, der Analyse der Kamerabewegung, der Segmentierung und Klassifikation von Objekten, der Texterkennung und der Gesichtserkennung vorgestellt. Die automatisch ermittelten semantischen Informationen sind sehr wertvoll, da sie die Arbeit mit digitalen Videoarchiven erleichtern. Die Informationen unterstützen nicht nur die Suche in den Archiven, sondern führen auch zur Entwicklung neuer Anwendungen, die im zweiten Teil der Dissertation vorgestellt werden. Beispielsweise können computergenerierte Zusammenfassungen von Videos erzeugt oder Videos automatisch an die Eigenschaften eines Abspielgerätes angepasst werden. Ein weiterer Schwerpunkt dieser Dissertation liegt in der Analyse historischer Filme. Vier europäische Filmarchive haben eine große Anzahl historischer Videodokumentationen zur Verfügung gestellt, welche Anfang bis Mitte des letzten Jahrhunderts gedreht und in den letzten Jahren digitalisiert wurden. Durch die Lagerung und Abnutzung der Filmrollen über mehrere Jahrzehnte sind viele Videos stark verrauscht und enthalten deutlich sichtbare Bildfehler. Die Bildqualität der historischen Schwarz-Weiß-Filme unterscheidet sich signifikant von der Qualität aktueller Videos, so dass eine verlässliche Analyse mit bestehenden Verfahren häufig nicht möglich ist. Im Rahmen dieser Dissertation werden neue Algorithmen vorgestellt, um eine zuverlässige Erkennung von semantischen Inhalten auch in historischen Videos zu ermöglichen

A component framework for personalized multimedia applications

Eine praktikable Unterstützung für eine dynamische Erstellung von personalisierten Multimedia-Präsentationen bieten bisher weder industrielle Lösungen noch Forschungsansätze. Mit dem Software-technischen Ansatz des MM4U-Frameworks („MultiMedia For You“) wird erstmals eine generische und zugleich praktikable Unterstützung für den dynamischen Erstellungsprozess bereitgestellt. Das Ziel des MM4U-Frameworks ist es den Anwendungsentwicklern eine umfangreiche und anwendungsunabhängige Unterstützung zur Erstellung von personalisierten Multimedia-Inhalten anzubieten und damit den Entwicklungsprozess solcher Anwendungen erheblich zu erleichtern. Um das Ziel eines Software-Frameworks zur generischen Unterstützung der Entwicklung von personalisierten Multimedia-Anwendungen zu erreichen, stellt sich die Frage nach einer geeigneten Software-technischen Unterstützung zur Entwicklung eines solchen Frameworks. Seit der Einführung von objektorientierten Frameworks, ist heute die Entwicklung immer noch aufwendig und schwierig. Um die Entwicklungsrisiken zu reduzieren, sind geeignete Vorgehensmodelle und Entwicklungsmethoden erstellt worden. Mit der Komponenten-Technologie sind auch so genannte Komponenten-Frameworks entstanden. Im Gegensatz zu objekt-orientierten Frameworks fehlt derzeit jedoch ein geeignetes Vorgehensmodell für Komponenten-Frameworks. Um den Entwicklungsprozess von Komponenten-Frameworks zu verbessern ist mit ProMoCF („Process Model for Component Frameworks“) ein neuartiger Ansatz entwickelt worden. Hierbei handelt es sich um ein leichtgewichtiges Vorgehensmodell und eine Entwicklungsmethodik für Komponenten-Frameworks. Das Vorgehensmodell wurde unter gegenseitigem Nutzen mit der Entwicklung des MM4U-Frameworks erstellt. Das MM4U-Framework stellt keine Neuerfindung der Adaption von Multimedia-Inhalten dar, sondern zielt auf die Vereinigung und Einbettung existierender Forschungsansätze und Lösungen im Umfeld der Multimedia-Personalisierung. Mit so einem Framework an der Hand können Anwendungsentwickler erstmals effizient und einfach eine dynamische Erstellung ihrer personalisierten Multimedia-Inhalte realisieren

A decisão editorial em televisão : o caso do telejornal da RTP

Tese de Doutoramento em Ciências da Comunicação na especialidade de Sociologia da Comunicação apresentada à Universidade AbertaApesar da evolução dos media digitais possibilitar uma partilha de informação mais personalizada, são as Televisões, sobretudo as generalistas, que continuam a deter a primazia na busca de informação de actualidade por parte dos grandes públicos (Buckley, 2008; Colvin, 2007). A informação televisiva antes de se tornar pública, passa por várias fases de concepção (newsmaking) – procura, selecção, redacção e tratamento final – cada uma delas passível de possibilitar contornos simbólicos ao produto noticioso, entendido quer na sua porção mínima – unidade jornalística (notícia) – quer no seu formato global – o jornal. Esta dissertação de Doutoramento centra-se essencialmente na fase da selecção, onde se integra o processo de gatekeeping, um fenómeno de filtragem controlado por sistemas objectivados de regras ou por gatekeepers – neste último caso, há um indivíduo, ou um grupo, que tem o «poder de decidir se deixa passar a informação ou se a bloqueia» (Lewin, 1947:145). Contudo, e porque o processo de decisão editorial é muito mais do que selecionar os itens noticiosos que entram ou são eliminados do processo, analisámos toda a noticiabilidade tornada pública, quer a nível dos conteúdos, como dos aspectos formais que também contribuem para a valoração da informação emitida. Assim, levantámos a nossa questão de pesquisa: “Como é que os editores do Telejornal da RTP percepcionam os constrangimentos valorativos inerentes ao processo de selecção que fazem da factualidade e de que forma são utilizados na construção dos alinhamentos noticiosos?” Para o efeito, adoptámos a observação participante naturalística da tomada de decisão editorial do Telejornal, apoiada por entrevistas etnográficas aos decisores e aos jornalistas e ainda entrevistas semi-estruturadas, apenas aos decisores. Procedemos também à análise de conteúdo do total de notícias (1744) de 62 jornais, emitidos de Segunda a Sexta-feira, entre Janeiro e Março de 2009. Para o efeito, criámos um conjunto de categorias de registo e de unidades de análise para avaliar o ‘sentido’ da informação transmitida. Em termos conclusivos, deparámo-nos com vários factores que exercem ‘forças’ positivas (oportunidades) e/ou negativas (constrangimentos) no processo de decisão noticioso. Este processo está longe de ser linear, e conta eminentemente com influências do sistema social, e da organização. Com esta investigação demonstrámos a pertinência das teorias integracionistas do gatekeeping, que não isolam um único factor de influência, num processo dinâmico como é a tomada de decisão editorial.Despite the evolution of digital media affords a personalized sharing of information, the Television, mainly the generalist type, still have, nowadays, the primacy in searching current information by the majority of public (Buckley, 2008; Colvin, 2007). Television news, before being a public affair, passes through many conception phases (news making process) - search, selection, writing and final edition - each one with capability to generate symbolic "senses" to the news, understood in its minimum portion - journalistic item - and in its global format - the journal. This doctoral thesis is focused, essentially, in the selection phase, where gatekeeping happens. This is a filtering phenomenon, controlled by objective systems of rules or by gatekeepers - an individual or a group who has the power to decide if information can pass or if it is blocked (Lewin, 1947:145). However, the editorial decision making is much more complex than a mere select or elimination of news items that come to the newsroom gates. Our analysis focused on contents and on shapes of news, because both are enable to give significance to the information. We raised the following research question: "How do the editors of the Telejornal (RTP) perceive the underlying constraints of the selection process they use to decide about factuality, and in which way those constraints are used in the construction of the news alignment?”. We adopted a naturalistic observation of editorial decision-making, participating in the editorial briefings, supported by ethnographic interviews to the decision makers and the journalists, and also semi-structured interviews only to the decision makers. We also performed a content analysis for all the news emitted, through a set of registry categories and units of analysis that allows to understand the social construction of the reality through the transmitted information. The segmentation of issues and the thematic areas for too scarce times and its subsequent priorization, based on the position in which they are placed in the sequential structure of news, the types of narratives shapes, the approaches angles, the news values and sources privileged, the available markets, the competitors’ offers, the social system rules, etc., are some of the many factors that transform an occurrence into a “news happening”. Concluding, we faced several factors that exert positive 'forces' (opportunities) and / or negative ones (constraints) in the decision-making process. This process is far from being linear, and it is highly influenced by the social system, and by the organization. This investigation demonstrated the relevance of the integrationist theories of gatekeeping, which do not isolate a single factor of influence in a dynamic process as the editorial decision-making

Automatic Construction Of Personalized

In this paper, we study the automatic construction of personalized TV News programs, where we want to build a program with predefined duration and maximum content value for a specific user. We combine video indexing techniques to parse TV News recordings into stories, and information filtering techniques to select stories which are most adequate given the user profile. We formalize the selection process as an optimization problem, and we study how to take into account duration in the selection of stories. Experiments show that a simple heuristic can provide high quality selection with little computation. We also describe two prototypes, which implement two different mechanisms for the construction of user profiles: . explicit specification, using a category-based model, . implicit specification, using a keyword-based model