UR-60 Video-to-Video Synthesis With Semantically Segmented Video

Our project involves studying the usage of generative adversarial networks (GANs) to translate semantically segmented video to photo-realistic video in a process known as video-to-video synthesis. The model is able to learn a mapping from semantically segmented masks to real-life images which depict the corresponding semantic labels. To achieve this, we employ a conditional GAN-based learning method that produces output conditionally based on the source video to be translated. Our model is capable of synthesizing a translated video, given semantically labeled video, that resembles real video by accurately replicating low-frequency details from the source.Advisors(s): Dr. Mohammed AledhariTopic(s): Artificial IntelligenceCS 473

Fünf evidenzbasierte Heuristiken für den Einsatz von Video in der universitären Lehrerausbildung

This article provides a research synthesis on the use of video in pre-service teacher education. Common ideas and evidences concerning the use of video in pre-service teacher education are reviewed. Based on the state-of-the-art in using video, five research-based heuristics are derived. Research findings of a number of studies are further used to illustrate the specification of heuristics. Specifically, a set of rules of thumb about when, how, and why to use video is presented to clarify the strengths and limitations of video as a medium to support pre-service teacher learning. (DIPF/Orig.)Der Beitrag liefert eine Forschungssynthese zur Nutzung von Video in der universitären Lehrerausbildung. Die Forschung wird dahingehend zusammengefasst, welche Ideen derzeit verfolgt werden und welche Evidenzen zur Nutzung von Video vorliegen. Basierend auf dem Forschungsstand leiten die Autoren fünf forschungsbasierte Heuristiken zum Einsatz von Video ab. Die Forschungsergebnisse einer Reihe ausgewählter Studien werden genutzt, um die Heuristiken weiter zu spezifizieren. Es werden Erfahrungsregeln vorgestellt, wann, wie und warum Video in der universitären Lehrerbildung eingesetzt werden kann. Die Erfahrungsregeln sollen helfen, Stärken und Schwächen von Video als ein Medium zur Unterstützung des Lernens von Lehramtsstudierenden zu klären. (DIPF/Orig.

Text-based Editing of Talking-head Video

Editing talking-head video to change the speech content or to remove filler words is challenging. We propose a novel method to edit talking-head video based on its transcript to produce a realistic output video in which the dialogue of the speaker has been modified, while maintaining a seamless audio-visual flow (i.e. no jump cuts). Our method automatically annotates an input talking-head video with phonemes, visemes, 3D face pose and geometry, reflectance, expression and scene illumination per frame. To edit a video, the user has to only edit the transcript, and an optimization strategy then chooses segments of the input corpus as base material. The annotated parameters corresponding to the selected segments are seamlessly stitched together and used to produce an intermediate video representation in which the lower half of the face is rendered with a parametric face model. Finally, a recurrent video generation network transforms this representation to a photorealistic video that matches the edited transcript. We demonstrate a large variety of edits, such as the addition, removal, and alteration of words, as well as convincing language translation and full sentence synthesis