1,698 research outputs found

    Creating walk-through images from a video sequence of a dynamic scene

    Get PDF
    A comprehensive scheme for creating walk-through images from a video sequence by generalizing the idea of tour into the picture (TIP) was discussed. The proposed scheme was designed to incorporate a new modeling scheme on a vanishing circle identified in the video and an automatic background detection from the video. This scheme let users experience the feel of navigating into a video sequence with their own interpretation and imagination about a given scene. The proposed scheme covers several types of video films of dynamic scenes such as sports coverage, cartoon animation and movie films in which object continuously change shapes and locations

    Анімований Тивопис Π² Π½ΠΎΠ²ΠΎΠΌΡƒ ΠΌΠ΅Π΄Ρ–ΠΉΠ½ΠΎΠΌΡƒ мистСцтві: Ρ‚Π΅Ρ…Π½ΠΎΠ»ΠΎΠ³Ρ–Ρ‡Π½Ρ– Ρ‚Π° ΠΊΠΎΠΌΡƒΠ½Ρ–ΠΊΠ°Ρ‚ΠΈΠ²Π½Ρ– особливості

    Get PDF
    Purpose of article. The purpose of the research is to investigate the animated painting as the kind of New Media Art. Methodology. The methodology of the research is based on using communicativistics and the concept theory as the basic methodological approaches in the analysis of the contemporary audiovisual art. Scientific novelty. The scientific novelty of the work consists in considering animated painting as an outstanding artistic kind that significantly influences the development of the contemporary audiovisual culture. Conclusions. The collocation Β«animated paintingΒ» most fully corresponds to the newest kind of the New Media Art that represents painting in dynamics, which, however, is not an animation in its classical sense. The main technological features of the animated painting are reproducibility, transgressivity, diffusion in exhibiting. The most important communicative features of animated painting are the use of well-known cultural codes, as well as their parody; the use of non-narrative tools of influence on the audience. Thus, there could be observed the hybrid type of the technological embodiment and communicative specificity of the animated painting that substantially transforms the contemporary audiovisual art. The further investigation of the animated painting could shed more light on the problem of changing classic artistic tradition that takes place in the contemporary audiovisual art.ЦСль Ρ€Π°Π±ΠΎΡ‚Ρ‹ – ΠΈΡΡΠ»Π΅Π΄ΠΎΠ²Π°Ρ‚ΡŒ Π°Π½ΠΈΠΌΠΈΡ€ΠΎΠ²Π°Π½Π½ΡƒΡŽ Тивопись ΠΊΠ°ΠΊ явлСниС Π½ΠΎΠ²ΠΎΠ³ΠΎ ΠΌΠ΅Π΄ΠΈΠΉΠ½ΠΎΠ³ΠΎ искусства. ΠœΠ΅Ρ‚ΠΎΠ΄ΠΎΠ»ΠΎΠ³ΠΈΡ исслСдования базируСтся Π½Π° ΠΏΡ€ΠΈΠΌΠ΅Π½Π΅Π½ΠΈΠΈ коммуникативистики ΠΈ Ρ‚Π΅ΠΎΡ€ΠΈΠΈ ΠΊΠΎΠ½Ρ†Π΅ΠΏΡ‚Π° ΠΊΠ°ΠΊ Π²Π΅Π΄ΡƒΡ‰ΠΈΡ… мСтодологичСских ΠΏΠΎΠ΄Ρ…ΠΎΠ΄ΠΎΠ² Π² ΠΈΠ·ΡƒΡ‡Π΅Π½ΠΈΠΈ ΠΏΡ€ΠΎΠΈΠ·Π²Π΅Π΄Π΅Π½ΠΈΠΉ соврСмСнного Π°ΡƒΠ΄ΠΈΠΎΠ²ΠΈΠ·ΡƒΠ°Π»ΡŒΠ½ΠΎΠ³ΠΎ искусства. Научная Π½ΠΎΠ²ΠΈΠ·Π½Π° Ρ€Π°Π±ΠΎΡ‚Ρ‹ состоит Π² ΠΈΠ·ΡƒΡ‡Π΅Π½ΠΈΠΈ Π°Π½ΠΈΠΌΠΈΡ€ΠΎΠ²Π°Π½Π½ΠΎΠΉ Тивописи ΠΊΠ°ΠΊ Π²Ρ‹Π΄Π°ΡŽΡ‰Π΅Π³ΠΎΡΡ худоТСствСнного явлСния, Π·Π°ΠΌΠ΅Ρ‚Π½ΠΎ Π²Π»ΠΈΡΡŽΡ‰Π΅Π³ΠΎ Π½Π° Ρ€Π°Π·Π²ΠΈΡ‚ΠΈΠ΅ соврСмСнной Π°ΡƒΠ΄ΠΈΠΎΠ²ΠΈΠ·ΡƒΠ°Π»ΡŒΠ½ΠΎΠΉ ΠΊΡƒΠ»ΡŒΡ‚ΡƒΡ€Ρ‹. Π’Ρ‹Π²ΠΎΠ΄Ρ‹. БловосочСтаниС «анимированная Тивопись» наиболСС ΠΏΠΎΠ»Π½ΠΎ соотвСтствуСт Π½ΠΎΠ²Π΅ΠΉΡˆΠ΅ΠΌΡƒ сСгмСнту Π½ΠΎΠ²ΠΎΠ³ΠΎ ΠΌΠ΅Π΄ΠΈΠΉΠ½ΠΎΠ³ΠΎ искусства, ΠΏΡ€Π΅Π΄ΡΡ‚Π°Π²Π»ΡΡŽΡ‰Π΅ΠΌΡƒ Тивопись Π² Π΄ΠΈΠ½Π°ΠΌΠΈΠΊΠ΅, Ρ‡Ρ‚ΠΎ, ΠΎΠ΄Π½Π°ΠΊΠΎ, Π½Π΅ являСтся Π°Π½ΠΈΠΌΠ°Ρ†ΠΈΠ΅ΠΉ Π² Π΅Π΅ классичСском ΠΏΠΎΠ½ΠΈΠΌΠ°Π½ΠΈΠΈ.Β ΠžΡΠ½ΠΎΠ²Π½Ρ‹ΠΌΠΈ тСхнологичСскими особСнностями Π°Π½ΠΈΠΌΠΈΡ€ΠΎΠ²Π°Π½Π½ΠΎΠΉ Тивописи ΡΠ²Π»ΡΡŽΡ‚ΡΡ Π²ΠΎΡΠΏΡ€ΠΎΠΈΠ·Π²ΠΎΠ΄ΠΈΠΌΠΎΡΡ‚ΡŒ, Ρ‚Ρ€Π°Π½ΡΠ³Ρ€Π΅ΡΡΠΈΠ²Π½ΠΎΡΡ‚ΡŒ, Π΄ΠΈΡ„Ρ„ΡƒΠ·Π½ΠΎΡΡ‚ΡŒ Π² экспонировании. НаиболСС Π²Π°ΠΆΠ½Ρ‹Π΅ ΠΊΠΎΠΌΠΌΡƒΠ½ΠΈΠΊΠ°Ρ‚ΠΈΠ²Π½Ρ‹Π΅ особСнности Π°Π½ΠΈΠΌΠΈΡ€ΠΎΠ²Π°Π½Π½ΠΎΠΉ Тивописи β€” задСйствованиС ΡˆΠΈΡ€ΠΎΠΊΠΎ извСстных ΠΊΡƒΠ»ΡŒΡ‚ΡƒΡ€Π½Ρ‹Ρ… ΠΊΠΎΠ΄ΠΎΠ², Π° Ρ‚Π°ΠΊΠΆΠ΅ ΠΈΡ… ΠΏΠ°Ρ€ΠΎΠ΄ΠΈΡ€ΠΎΠ²Π°Π½ΠΈΠ΅; использованиС Π½Π΅Π½Π°Ρ€Ρ€Π°Ρ‚ΠΈΠ²Π½Ρ‹Ρ… инструмСнтов влияния Π½Π° Π°ΡƒΠ΄ΠΈΡ‚ΠΎΡ€ΠΈΡŽ. Π’Π°ΠΊΠΈΠΌ ΠΎΠ±Ρ€Π°Π·ΠΎΠΌ, Π½Π°Π±Π»ΡŽΠ΄Π°Π΅Ρ‚ΡΡ Π³ΠΈΠ±Ρ€ΠΈΠ΄Π½Ρ‹ΠΉ Ρ…Π°Ρ€Π°ΠΊΡ‚Π΅Ρ€ тСхнологичСского воплощСния ΠΈ ΠΊΠΎΠΌΠΌΡƒΠ½ΠΈΠΊΠ°Ρ‚ΠΈΠ²Π½ΠΎΠΉ спСцифики Π°Π½ΠΈΠΌΠΈΡ€ΠΎΠ²Π°Π½Π½ΠΎΠΉ Тивописи, Ρ‡Ρ‚ΠΎ сущСствСнно измСняСт соврСмСнноС Π°ΡƒΠ΄ΠΈΠΎΠ²ΠΈΠ·ΡƒΠ°Π»ΡŒΠ½ΠΎΠ΅ искусство. ΠŸΠΎΡΠ»Π΅Π΄ΡƒΡŽΡ‰Π΅Π΅ ΠΈΠ·ΡƒΡ‡Π΅Π½ΠΈΠ΅ Π°Π½ΠΈΠΌΠΈΡ€ΠΎΠ²Π°Π½Π½ΠΎΠΉ Тивописи ΠΌΠΎΠΆΠ΅Ρ‚ ΠΏΡ€ΠΎΠ»ΠΈΡ‚ΡŒ большС свСта Π½Π° ΠΏΡ€ΠΎΠ±Π»Π΅ΠΌΡƒ трансформации классичСских худоТСствСнных Ρ‚Ρ€Π°Π΄ΠΈΡ†ΠΈΠΉ, ΠΊΠΎΡ‚ΠΎΡ€Ρ‹Π΅ ΠΈΠΌΠ΅ΡŽΡ‚ мСсто Π² соврСмСнном Π°ΡƒΠ΄ΠΈΠΎΠ²ΠΈΠ·ΡƒΠ°Π»ΡŒΠ½ΠΎΠΌ искусствС.ΠœΠ΅Ρ‚Π° Ρ€ΠΎΠ±ΠΎΡ‚ΠΈ – дослідити Π°Π½Ρ–ΠΌΠΎΠ²Π°Π½ΠΈΠΉ Тивопис як явищС Π½ΠΎΠ²ΠΎΠ³ΠΎ ΠΌΠ΅Π΄Ρ–ΠΉΠ½ΠΎΠ³ΠΎ мистСцтва. ΠœΠ΅Ρ‚ΠΎΠ΄ΠΎΠ»ΠΎΠ³Ρ–Ρ дослідТСння Π±Π°Π·ΡƒΡ”Ρ‚ΡŒΡΡ Π½Π° застосуванні комунікативістики Ρ‚Π° Ρ‚Π΅ΠΎΡ€Ρ–Ρ— ΠΊΠΎΠ½Ρ†Π΅ΠΏΡ‚Ρƒ як ΠΏΡ€ΠΎΠ²Ρ–Π΄Π½ΠΈΡ… ΠΌΠ΅Ρ‚ΠΎΠ΄ΠΎΠ»ΠΎΠ³Ρ–Ρ‡Π½ΠΈΡ… ΠΏΡ–Π΄Ρ…ΠΎΠ΄Ρ–Π² Ρƒ Π²ΠΈΠ²Ρ‡Π΅Π½Π½Ρ– Ρ‚Π²ΠΎΡ€Ρ–Π² сучасного Π°ΡƒΠ΄Ρ–ΠΎΠ²Ρ–Π·ΡƒΠ°Π»ΡŒΠ½ΠΎΠ³ΠΎ мистСцтва. Наукова Π½ΠΎΠ²ΠΈΠ·Π½Π° Ρ€ΠΎΠ±ΠΎΡ‚ΠΈ полягає Ρƒ Π²ΠΈΠ²Ρ‡Π΅Π½Π½Ρ– Π°Π½Ρ–ΠΌΠΎΠ²Π°Π½ΠΎΠ³ΠΎ Тивопису як Π²ΠΈΠ·Π½Π°Ρ‡Π½ΠΎΠ³ΠΎ ΠΌΠΈΡΡ‚Π΅Ρ†ΡŒΠΊΠΎΠ³ΠΎ явища, Ρ‰ΠΎ ΠΏΠΎΠΌΡ–Ρ‚Π½ΠΎ Π²ΠΏΠ»ΠΈΠ²Π°Ρ” Π½Π° Ρ€ΠΎΠ·Π²ΠΈΡ‚ΠΎΠΊ сучасної Π°ΡƒΠ΄Ρ–ΠΎΠ²Ρ–Π·ΡƒΠ°Π»ΡŒΠ½ΠΎΡ— ΠΊΡƒΠ»ΡŒΡ‚ΡƒΡ€ΠΈ. Висновки. БловосполучСння Β«Π°Π½Ρ–ΠΌΠΎΠ²Π°Π½ΠΈΠΉ Тивопис» Π½Π°ΠΉΠ±Ρ–Π»ΡŒΡˆ ΠΏΠΎΠ²Π½ΠΎ Π²Ρ–Π΄ΠΏΠΎΠ²Ρ–Π΄Π°Ρ” Π½ΠΎΠ²Ρ–Ρ‚Π½ΡŒΠΎΠΌΡƒ сСгмСнту Π½ΠΎΠ²ΠΎΠ³ΠΎ ΠΌΠ΅Π΄Ρ–ΠΉΠ½ΠΎΠ³ΠΎ мистСцтва Ρ‚Π° Ρ€Π΅ΠΏΡ€Π΅Π·Π΅Π½Ρ‚ΡƒΡ” Тивопис Π² Π΄ΠΈΠ½Π°ΠΌΡ–Ρ†Ρ–, Ρ‰ΠΎ, Π²Ρ‚Ρ–ΠΌ, Π½Π΅ΠΌΠΎΠΆΠ½Π° Π½Π°Π·Π²Π°Ρ‚ΠΈ Π°Π½Ρ–ΠΌΠ°Ρ†Ρ–ΡŽ Π² Ρ—Ρ— класичному Ρ€ΠΎΠ·ΡƒΠΌΡ–Π½Π½Ρ–. Основними Ρ‚Π΅Ρ…Π½ΠΎΠ»ΠΎΠ³Ρ–Ρ‡Π½ΠΈΠΌΠΈ особливостями Π°Π½Ρ–ΠΌΠΎΠ²Π°Π½ΠΎΠ³ΠΎ Тивопису Ρ” Ρ‚Π΅Ρ…Π½Ρ–Ρ‡Π½Π° Π²Ρ–Π΄Ρ‚Π²ΠΎΡ€ΡŽΠ²Π°Π½Ρ–ΡΡ‚ΡŒ, Ρ‚Ρ€Π°Π½ΡΠ³Ρ€Π΅ΡΠΈΠ²Π½Ρ–ΡΡ‚ΡŒ, Π΄ΠΈΡ„ΡƒΠ·Π½Ρ–ΡΡ‚ΡŒ Π² Скспонуванні. ΠΠ°ΠΉΠ²Π°ΠΆΠ»ΠΈΠ²Ρ–ΡˆΡ– ΠΊΠΎΠΌΡƒΠ½Ρ–ΠΊΠ°Ρ‚ΠΈΠ²Π½Ρ– особливості Π°Π½Ρ–ΠΌΠΎΠ²Π°Π½ΠΎΠ³ΠΎ Тивопису – залучання ΡˆΠΈΡ€ΠΎΠΊΠΎ Π²Ρ–Π΄ΠΎΠΌΠΈΡ… ΠΊΡƒΠ»ΡŒΡ‚ΡƒΡ€Π½ΠΈΡ… ΠΊΠΎΠ΄Ρ–Π² Ρ‚Π° Ρ—Ρ… ΠΏΠ°Ρ€ΠΎΠ΄Ρ–ΡŽΠ²Π°Π½Π½Ρ; використання Π½Π΅Π½Π°Ρ€Π°Ρ‚ΠΈΠ²Π½ΠΈΡ… інструмСнтів Π²ΠΏΠ»ΠΈΠ²Ρƒ Π½Π° Π°ΡƒΠ΄ΠΈΡ‚ΠΎΡ€Ρ–ΡŽ. Π’Π°ΠΊΠΈΠΌ Ρ‡ΠΈΠ½ΠΎΠΌ, ΡΠΏΠΎΡΡ‚Π΅Ρ€Ρ–Π³Π°Ρ”Ρ‚ΡŒΡΡ Π³Ρ–Π±Ρ€ΠΈΠ΄Π½ΠΈΠΉ Ρ…Π°Ρ€Π°ΠΊΡ‚Π΅Ρ€ Ρ‚Π΅Ρ…Π½ΠΎΠ»ΠΎΠ³Ρ–Ρ‡Π½ΠΎΠ³ΠΎ втілСння Ρ‚Π° ΠΊΠΎΠΌΡƒΠ½Ρ–ΠΊΠ°Ρ‚ΠΈΠ²Π½ΠΎΡ— спСцифіки Π°Π½Ρ–ΠΌΠΎΠ²Π°Π½ΠΎΠ³ΠΎ Тивопису, Ρ‰ΠΎ суттєво Π·ΠΌΡ–Π½ΡŽΡ” сучаснС Π°ΡƒΠ΄Ρ–ΠΎΠ²Ρ–Π·ΡƒΠ°Π»ΡŒΠ½Π΅ мистСцтво. ΠŸΠΎΠ΄Π°Π»ΡŒΡˆΡ– дослідТСння Π°Π½Ρ–ΠΌΠΎΠ²Π°Π½ΠΎΠ³ΠΎ Тивопису ΠΌΠΎΠΆΡƒΡ‚ΡŒ ΠΏΡ€ΠΎΠ»ΠΈΡ‚ΠΈ Π±Ρ–Π»ΡŒΡˆΠ΅ світла Π½Π° ΠΏΡ€ΠΎΠ±Π»Π΅ΠΌΡƒ трансформації класичних Ρ…ΡƒΠ΄ΠΎΠΆΠ½Ρ–Ρ… Ρ‚Ρ€Π°Π΄ΠΈΡ†Ρ–ΠΉ, Ρ‰ΠΎ ΠΌΠ°ΡŽΡ‚ΡŒ місцС Π² сучасному Π°ΡƒΠ΄Ρ–ΠΎΠ²Ρ–Π·ΡƒΠ°Π»ΡŒΠ½ΠΎΠΌΡƒ мистСцтві

    Efficiently Modeling 3D Scenes from a Single Image

    Full text link

    Enhancing the museum experience with a sustainable solution based on contextual information obtained from an on-line analysis of users’ behaviour

    Get PDF
    Human computer interaction has evolved in the last years in order to enhance users’ experiences and provide more intuitive and usable systems. A major leap through in this scenario is obtained by embedding, in the physical environment, sensors capable of detecting and processing users’ context (position, pose, gaze, ...). Feeded by the so collected information flows, user interface paradigms may shift from stereotyped gestures on physical devices, to more direct and intuitive ones that reduce the semantic gap between the action and the corresponding system reaction or even anticipate the user’s needs, thus limiting the overall learning effort and increasing user satisfaction. In order to make this process effective, the context of the user (i.e. where s/he is, what is s/he doing, who s/he is, what are her/his preferences and also actual perception and needs) must be properly understood. While collecting data on some aspects can be easy, interpreting them all in a meaningful way in order to improve the overall user experience is much harder. This is more evident when we consider informal learning environments like museums, i.e. places that are designed to elicit visitor response towards the artifacts on display and the cultural themes proposed. In such a situation, in fact, the system should adapt to the attention paid by the user choosing the appropriate content for the user’s purposes, presenting an intuitive interface to navigate it. My research goal is focused on collecting, in a simple,unobtrusive, and sustainable way, contextual information about the visitors with the purpose of creating more engaging and personalized experiences

    Merging static and dynamic visual media along an event timeline

    Get PDF
    Thesis (M.S.)--Massachusetts Institute of Technology, Program in Media Arts & Sciences, 1998.Includes bibliographical references (p. 63-65).Kyratso Karahalios.M.S

    Mapping the West: Nineteenth-Century American Landscape Photographs from the Boston Public Library

    Full text link
    This is the catalogue of the exhibition "Mapping the West" at Boston University Art Gallery

    Chinese drawing, architectural poetics : traditional painting as a semantic representation of modern architectural design

    Get PDF
    This thesis is partly an attempt to explore the potential of pre-modern Chinese painting, on its distinctive formats and schemes to achieve spatial depth and time duration, as a way to interpret and design architecture. By a survey on changing modes of Chinese traditional landscape and cityscape paintings in different scales, the poetic language of painting will be gradually explored. Beyond pictorial techniques, language is concerned with an ideological level of understanding and experience. Thus, it signposts a wider significance of architectural representation – as a verbal medium to express narrative and critic semantics besides visual effects. In this thesis, we will also see how traditional painting remains a base in the ideating process of several contemporary Chinese architects, so to avoid a mere uncritical imitation of international models. A subtle fusion of contemporaneity with cultural identity afforded by the presence of taken concepts from traditional painting, allows this architecture to increase its meaning and dimension. Lastly, understanding such processes of ideation can possibly provide us assistance in the intuitive formulation of ways to enrich Western architecture. Particularly, establishing poetic connections to our cultural traditions can be a useful strategy to prevent Western architecture's frequent falls into empty excesses of utilitarianism, iconicism or simple banality.Esta tesis en parte intenta explorar la capacidad de la pintura china pre-moderna en sus peculiares formatos y esquemas para lograr expresar la profundidad del espacio y la duraciΓ³n del tiempo, como una manera de interpretar y diseΓ±ar arquitectura contemporΓ‘nea. Mediante un estudio de la pintura tradicional de temΓ‘tica paisajΓ­stica y urbana, y a diferentes escalas, se analizarΓ‘ el lenguaje poΓ©tico de la pintura china. MΓ‘s allΓ‘ de las tΓ©cnicas pictΓ³ricas, este lenguaje se sitΓΊa en un nivel ideolΓ³gico de comprensiΓ³n y experiencia; expresa, por tanto, una gama de significados mΓ‘s amplia que la mera representaciΓ³n arquitectΓ³nica, actΓΊa como lo harΓ­a un medio verbal para expresar una semΓ‘ntica de tipo crΓ­tico y narrativo, ademΓ‘s de los consiguientes efectos visuales. En esta tesis, tambiΓ©n veremos cΓ³mo la pintura tradicional sigue siendo la base del proceso de creaciΓ³n de ideas de varios arquitectos chinos contemporΓ‘neos para evitar asΓ­ una mera imitaciΓ³n acrΓ­tica de modelos internacionales. Una fusiΓ³n sutil de la contemporaneidad con la identidad cultural proporcionada por la presencia de conceptos de la pintura tradicional permite a esta arquitectura ganar nuevas capas de significado y dimensiΓ³n. Por ΓΊltimo, comprender tales procesos de ideaciΓ³n puede brindarnos ayuda en la formulaciΓ³n intuitiva de formas de enriquecer la arquitectura occidental. En particular, establecer conexiones poΓ©ticas con nuestras tradiciones culturales puede ser una estrategia ΓΊtil para prevenir las frecuentes caΓ­das de la arquitectura occidental en los excesos vacΓ­os del utilitarismo, el iconicismo o la simple banalidad.Postprint (published version

    Spherical Image Processing for Immersive Visualisation and View Generation

    Get PDF
    This research presents the study of processing panoramic spherical images for immersive visualisation of real environments and generation of in-between views based on two views acquired. For visualisation based on one spherical image, the surrounding environment is modelled by a unit sphere mapped with the spherical image and the user is then allowed to navigate within the modelled scene. For visualisation based on two spherical images, a view generation algorithm is developed for modelling an indoor manmade environment and new views can be generated at an arbitrary position with respect to the existing two. This allows the scene to be modelled using multiple spherical images and the user to move smoothly from one sphere mapped image to another one by going through in-between sphere mapped images generated

    Automatic video segmentation employing object/camera modeling techniques

    Get PDF
    Practically established video compression and storage techniques still process video sequences as rectangular images without further semantic structure. However, humans watching a video sequence immediately recognize acting objects as semantic units. This semantic object separation is currently not reflected in the technical system, making it difficult to manipulate the video at the object level. The realization of object-based manipulation will introduce many new possibilities for working with videos like composing new scenes from pre-existing video objects or enabling user-interaction with the scene. Moreover, object-based video compression, as defined in the MPEG-4 standard, can provide high compression ratios because the foreground objects can be sent independently from the background. In the case that the scene background is static, the background views can even be combined into a large panoramic sprite image, from which the current camera view is extracted. This results in a higher compression ratio since the sprite image for each scene only has to be sent once. A prerequisite for employing object-based video processing is automatic (or at least user-assisted semi-automatic) segmentation of the input video into semantic units, the video objects. This segmentation is a difficult problem because the computer does not have the vast amount of pre-knowledge that humans subconsciously use for object detection. Thus, even the simple definition of the desired output of a segmentation system is difficult. The subject of this thesis is to provide algorithms for segmentation that are applicable to common video material and that are computationally efficient. The thesis is conceptually separated into three parts. In Part I, an automatic segmentation system for general video content is described in detail. Part II introduces object models as a tool to incorporate userdefined knowledge about the objects to be extracted into the segmentation process. Part III concentrates on the modeling of camera motion in order to relate the observed camera motion to real-world camera parameters. The segmentation system that is described in Part I is based on a background-subtraction technique. The pure background image that is required for this technique is synthesized from the input video itself. Sequences that contain rotational camera motion can also be processed since the camera motion is estimated and the input images are aligned into a panoramic scene-background. This approach is fully compatible to the MPEG-4 video-encoding framework, such that the segmentation system can be easily combined with an object-based MPEG-4 video codec. After an introduction to the theory of projective geometry in Chapter 2, which is required for the derivation of camera-motion models, the estimation of camera motion is discussed in Chapters 3 and 4. It is important that the camera-motion estimation is not influenced by foreground object motion. At the same time, the estimation should provide accurate motion parameters such that all input frames can be combined seamlessly into a background image. The core motion estimation is based on a feature-based approach where the motion parameters are determined with a robust-estimation algorithm (RANSAC) in order to distinguish the camera motion from simultaneously visible object motion. Our experiments showed that the robustness of the original RANSAC algorithm in practice does not reach the theoretically predicted performance. An analysis of the problem has revealed that this is caused by numerical instabilities that can be significantly reduced by a modification that we describe in Chapter 4. The synthetization of static-background images is discussed in Chapter 5. In particular, we present a new algorithm for the removal of the foreground objects from the background image such that a pure scene background remains. The proposed algorithm is optimized to synthesize the background even for difficult scenes in which the background is only visible for short periods of time. The problem is solved by clustering the image content for each region over time, such that each cluster comprises static content. Furthermore, it is exploited that the times, in which foreground objects appear in an image region, are similar to the corresponding times of neighboring image areas. The reconstructed background could be used directly as the sprite image in an MPEG-4 video coder. However, we have discovered that the counterintuitive approach of splitting the background into several independent parts can reduce the overall amount of data. In the case of general camera motion, the construction of a single sprite image is even impossible. In Chapter 6, a multi-sprite partitioning algorithm is presented, which separates the video sequence into a number of segments, for which independent sprites are synthesized. The partitioning is computed in such a way that the total area of the resulting sprites is minimized, while simultaneously satisfying additional constraints. These include a limited sprite-buffer size at the decoder, and the restriction that the image resolution in the sprite should never fall below the input-image resolution. The described multisprite approach is fully compatible to the MPEG-4 standard, but provides three advantages. First, any arbitrary rotational camera motion can be processed. Second, the coding-cost for transmitting the sprite images is lower, and finally, the quality of the decoded sprite images is better than in previously proposed sprite-generation algorithms. Segmentation masks for the foreground objects are computed with a change-detection algorithm that compares the pure background image with the input images. A special effect that occurs in the change detection is the problem of image misregistration. Since the change detection compares co-located image pixels in the camera-motion compensated images, a small error in the motion estimation can introduce segmentation errors because non-corresponding pixels are compared. We approach this problem in Chapter 7 by integrating risk-maps into the segmentation algorithm that identify pixels for which misregistration would probably result in errors. For these image areas, the change-detection algorithm is modified to disregard the difference values for the pixels marked in the risk-map. This modification significantly reduces the number of false object detections in fine-textured image areas. The algorithmic building-blocks described above can be combined into a segmentation system in various ways, depending on whether camera motion has to be considered or whether real-time execution is required. These different systems and example applications are discussed in Chapter 8. Part II of the thesis extends the described segmentation system to consider object models in the analysis. Object models allow the user to specify which objects should be extracted from the video. In Chapters 9 and 10, a graph-based object model is presented in which the features of the main object regions are summarized in the graph nodes, and the spatial relations between these regions are expressed with the graph edges. The segmentation algorithm is extended by an object-detection algorithm that searches the input image for the user-defined object model. We provide two objectdetection algorithms. The first one is specific for cartoon sequences and uses an efficient sub-graph matching algorithm, whereas the second processes natural video sequences. With the object-model extension, the segmentation system can be controlled to extract individual objects, even if the input sequence comprises many objects. Chapter 11 proposes an alternative approach to incorporate object models into a segmentation algorithm. The chapter describes a semi-automatic segmentation algorithm, in which the user coarsely marks the object and the computer refines this to the exact object boundary. Afterwards, the object is tracked automatically through the sequence. In this algorithm, the object model is defined as the texture along the object contour. This texture is extracted in the first frame and then used during the object tracking to localize the original object. The core of the algorithm uses a graph representation of the image and a newly developed algorithm for computing shortest circular-paths in planar graphs. The proposed algorithm is faster than the currently known algorithms for this problem, and it can also be applied to many alternative problems like shape matching. Part III of the thesis elaborates on different techniques to derive information about the physical 3-D world from the camera motion. In the segmentation system, we employ camera-motion estimation, but the obtained parameters have no direct physical meaning. Chapter 12 discusses an extension to the camera-motion estimation to factorize the motion parameters into physically meaningful parameters (rotation angles, focal-length) using camera autocalibration techniques. The speciality of the algorithm is that it can process camera motion that spans several sprites by employing the above multi-sprite technique. Consequently, the algorithm can be applied to arbitrary rotational camera motion. For the analysis of video sequences, it is often required to determine and follow the position of the objects. Clearly, the object position in image coordinates provides little information if the viewing direction of the camera is not known. Chapter 13 provides a new algorithm to deduce the transformation between the image coordinates and the real-world coordinates for the special application of sport-video analysis. In sport videos, the camera view can be derived from markings on the playing field. For this reason, we employ a model of the playing field that describes the arrangement of lines. After detecting significant lines in the input image, a combinatorial search is carried out to establish correspondences between lines in the input image and lines in the model. The algorithm requires no information about the specific color of the playing field and it is very robust to occlusions or poor lighting conditions. Moreover, the algorithm is generic in the sense that it can be applied to any type of sport by simply exchanging the model of the playing field. In Chapter 14, we again consider panoramic background images and particularly focus ib their visualization. Apart from the planar backgroundsprites discussed previously, a frequently-used visualization technique for panoramic images are projections onto a cylinder surface which is unwrapped into a rectangular image. However, the disadvantage of this approach is that the viewer has no good orientation in the panoramic image because he looks into all directions at the same time. In order to provide a more intuitive presentation of wide-angle views, we have developed a visualization technique specialized for the case of indoor environments. We present an algorithm to determine the 3-D shape of the room in which the image was captured, or, more generally, to compute a complete floor plan if several panoramic images captured in each of the rooms are provided. Based on the obtained 3-D geometry, a graphical model of the rooms is constructed, where the walls are displayed with textures that are extracted from the panoramic images. This representation enables to conduct virtual walk-throughs in the reconstructed room and therefore, provides a better orientation for the user. Summarizing, we can conclude that all segmentation techniques employ some definition of foreground objects. These definitions are either explicit, using object models like in Part II of this thesis, or they are implicitly defined like in the background synthetization in Part I. The results of this thesis show that implicit descriptions, which extract their definition from video content, work well when the sequence is long enough to extract this information reliably. However, high-level semantics are difficult to integrate into the segmentation approaches that are based on implicit models. Intead, those semantics should be added as postprocessing steps. On the other hand, explicit object models apply semantic pre-knowledge at early stages of the segmentation. Moreover, they can be applied to short video sequences or even still pictures since no background model has to be extracted from the video. The definition of a general object-modeling technique that is widely applicable and that also enables an accurate segmentation remains an important yet challenging problem for further research
    • …
    corecore