Enabling arbitrary rotation camera-motion using multi-sprites with minimum coding cost

Object-oriented coding in the MPEG-4 standard enables the separate processing of foreground objects and the scene background (sprite). Since the background sprite only has to be sent once, transmission bandwidth can be saved.We have found that the counter-intuitive approach of splitting the background into several independent parts can reduce the overall amount of data. Furthermore, we show that in the general case, the synthesis of a single background sprite is even impossible and that the scene background must be sent as multiple sprites instead. For this reason, we propose an algorithm that provides an optimal partitioning of a video sequence into independent background sprites (a multisprite), resulting in a significant reduction of the involved coding cost. Additionally, our sprite-generation algorithm ensures that the sprite resolution is kept high enough to preserve all details of the input sequence, which is a problem especially during camera zoom-in operations. Even though our sprite generation algorithm creates multiple sprites instead of only a single background sprite, it is fully compatible with the existing MPEG-4 standard. The algorithm has been evaluated with several test sequences, including the well-known Table-tennis and Stefan sequences. The total coding cost for the sprite VOP is reduced by a factor of about 2.6 or even higher, depending on the sequence

The physics of streamer discharge phenomena

In this review we describe a transient type of gas discharge which is commonly called a streamer discharge, as well as a few related phenomena in pulsed discharges. Streamers are propagating ionization fronts with self-organized field enhancement at their tips that can appear in gases at (or close to) atmospheric pressure. They are the precursors of other discharges like sparks and lightning, but they also occur in for example corona reactors or plasma jets which are used for a variety of plasma chemical purposes. When enough space is available, streamers can also form at much lower pressures, like in the case of sprite discharges high up in the atmosphere. We explain the structure and basic underlying physics of streamer discharges, and how they scale with gas density. We discuss the chemistry and applications of streamers, and describe their two main stages in detail: inception and propagation. We also look at some other topics, like interaction with flow and heat, related pulsed discharges, and electron runaway and high energy radiation. Finally, we discuss streamer simulations and diagnostics in quite some detail. This review is written with two purposes in mind: First, we describe recent results on the physics of streamer discharges, with a focus on the work performed in our groups. We also describe recent developments in diagnostics and simulations of streamers. Second, we provide background information on the above-mentioned aspects of streamers. This review can therefore be used as a tutorial by researchers starting to work in the field of streamer physics.Comment: 89 pages, 29 figure

I Am Error

I Am Error is a platform study of the Nintendo Family Computer (or Famicom), a videogame console first released in Japan in July 1983 and later exported to the rest of the world as the Nintendo Entertainment System (or NES). The book investigates the underlying computational architecture of the console and its effects on the creative works (e.g. videogames) produced for the platform. I Am Error advances the concept of platform as a shifting configuration of hardware and software that extends even beyond its ‘native’ material construction. The book provides a deep technical understanding of how the platform was programmed and engineered, from code to silicon, including the design decisions that shaped both the expressive capabilities of the machine and the perception of videogames in general. The book also considers the platform beyond the console proper, including cartridges, controllers, peripherals, packaging, marketing, licensing, and play environments. Likewise, it analyzes the NES’s extension and afterlife in emulation and hacking, birthing new genres of creative expression such as ROM hacks and tool-assisted speed runs. I Am Error considers videogames and their platforms to be important objects of cultural expression, alongside cinema, dance, painting, theater and other media. It joins the discussion taking place in similar burgeoning disciplines—code studies, game studies, computational theory—that engage digital media with critical rigor and descriptive depth. But platform studies is not simply a technical discussion—it also keeps a keen eye on the cultural, social, and economic forces that influence videogames. No platform exists in a vacuum: circuits, code, and console alike are shaped by the currents of history, politics, economics, and culture—just as those currents are shaped in kind

Animation of a High-Definition 2d Fighting Game Character

This thesis aims to identify the principles of good in-game character animation and examines different approaches to creating 2d animation. Traditional animation principles, such as timing and spacing, apply to in-game animation as they do to film animation. However, due to restrictions set by the technical and interactive elements of video games, additional challenges are faced in creating animations that both support gameplay and are visually engaging. Appealing character animation is particularly important in the fighting game genre. Early fighting games from the beginning of the 1990s used low resolution bitmap graphics with low frame counts for animation, but nowadays the standards for graphics and animation in games are high. In recent years, many game developers have shifted to using 3d graphics in place of 2d graphics, due to the more adaptive nature of 3d animation. However, there is still demand for the unique aesthetic of hand-drawn 2d, which cannot be fully replaced by 3d techniques. To make the creation process of 2d animation more efficient, supporting techniques such as tracing animation from 3d models have been employed. As the practical part of the thesis character animations suitable for a high-definition 2d fighting game were created. To assist in the creation of 2d animation, 3d animations were used as reference. While using 3d reference is beneficial especially for less experienced animators, it was discovered that relying too heavily on 3d may compromise the quality of the final 2d animation. The assistance of 3d tools cannot substitute good 2d fundamentals.Tämä opinnäytetyö pyrkii erittelemään hyvän pelihahmoanimaation periaatteita ja tarkastelee eri lähestymistapoja 2d-animaation luomiseen. Perinteisen animaation periaatteet, kuten ajoitus ja liikkeen välistys, pätevät pelianimaatiossa samalla tavalla kuin elokuva-animaatiossakin. Pelien tekniset rajoitukset ja interaktiivisuus asettavat kuiten-kin lisähaasteita animaatioiden toteuttamiseen tavalla, joka sekä tukee pelimekaniikkaa että on visuaalisesti kiinnostava. Vetoava hahmoanimaatio on erityisen tärkeää taistelupeligenressä. Varhaiset taistelupelit 1990–luvun alusta käyttivät matalaresoluutioista bittikarttagrafiikkaa ja niissä oli alhainen määrä animaatiokehyksiä, mutta nykyään pelien standardit grafiikan ja animaation suhteen ovat korkealla. Viime vuosina monet pelinkehittäjät ovat siirtyneet käyttämään 2d-grafiikan sijasta 3d-grafiikkaa, koska 3d-animaation tuottaminen on monella tavalla joustavampaa. Perinteiselle 2d-grafiikalle on kuitenkin edelleen kysyntää, sillä käsin piirretyn animaation ainutlaatuista ulkoasua ei voi täysin korvata 3d-tekniikoilla. 2d-animaation luomista on pyritty tehostamaan avustavilla tekniikoilla kuten animaation jäljentämisellä 3d-animaatiosta. Opinnäytetyön käytännön osuutena luotiin hahmoanimaatioita jotka soveltuvat teräväpiirtoiseen 2d-taistelupeliin. 2d-animoinnin apuna käytettiin 3d-animaatiota. Vaikka 3d-animaation käyttäminen mallina on hyödyllistä etenkin kokemattomalle animaattorille, liian tukeutumisen 3d-animaatioon todettiin saattavan heikentää lopullisen 2d-animaation laatua. 3d-työkalujen avustus ei korvaa hyvää 2d-perusteiden hallintaa

Novel methods for mapping genome organization and genome fragility in the 3D space of the nucleus

For over a century, scientists have been trying to understand how the DNA molecule that is so tightly packed in the micro space of the nucleus sustains critical cellular processes such as transcription, replication or the maintenance of genetic information. Despite the huge effort of researchers around the world, we know relatively little. This is in part due to the lack of methods that bring the right throughput and resolution to the study of the higher-order spatial arrangement of the genome. For example, the radial arrangement of the chromatin in mammalian cells remains largely unrevealed as well as the processes that lead to genome instability, which could potentially lead to the development of cancer. It is therefore crucial to develop tools that would allow us to precisely map both the location and frequency of DNA double-strand breaks (DSBs) along the genome, as well as to study them in the right 3D context of the chromatin. In order to fill this gap in, this thesis describes two methods which we have developed in order to map genome organization and genome fragility in the 3D space of the nucleus. In paper I, we developed GPSeq (Genome Loci Positioning by Sequencing) as a genome-wide technique for mapping radial arrangement of the genome in mammalian cells. We showed that GPSeq accurately generates maps of the radial organization of the human genome at 1 Mb and 100 kb resolutions, thus allowing us to reveal unique radial patterns of various genetic and epigenomic traits, gene expression, A and B subcompartments as well as radial arrangements of DSBs, cancer mutations or germline variants. In paper II, we developed BLISS (Breaks Labelling In Situ and Sequencing) as a genome- wide technique to quantitatively profile DSBs distribution in cells. We demonstrated that BLISS can be successfully applied to samples with either low number of cells or to tissue sections and yet accurately detect DSBs. We showed the sensitivity of BLISS by estimating off-target activity of two nucleases- Cas9 and Cpf1 in CRISPR system and demonstrated that Cpf1 is more specific when compared to Cas9

Versioning in Interactive Systems

Dealing with past states of an interactive system is often difficult, and users often resort to unwieldy methods such as saving and naming multiple copies. Versioning tools can help users save and manipulate different versions of a document, but traditional tools designed for coding are often unsuitable for interactive systems. Supporting versioning in interactive systems requires investigation of how users think about versions and how they want to access and manipulate past states. We first surveyed users to understand what a ‘version’ means to them in the context of digital interactive work, and the circumstances under which they create new versions or go back to previous ones. We then built a versioning tool that can store versions using a variety of explicit and implicit mechanisms and shows a graphical representation of the version tree to allow easy inspection and manipulation. To observe how users used versions in different work contexts, we tested our versioning tool in two interactive systems – a game level editor and a web analysis tool. We report several new findings about how users of interactive systems create versions and use them as undo alternatives, exploring options, and planning future work. Our results show that versioning can be a valuable component that improves the power and usability of interactive systems. The new understanding that we gained about versioning in interactive environments by developing and evaluating our custom version tool can help us design more effective versioning tools for interactive systems

Automatic video segmentation employing object/camera modeling techniques

Practically established video compression and storage techniques still process video sequences as rectangular images without further semantic structure. However, humans watching a video sequence immediately recognize acting objects as semantic units. This semantic object separation is currently not reflected in the technical system, making it difficult to manipulate the video at the object level. The realization of object-based manipulation will introduce many new possibilities for working with videos like composing new scenes from pre-existing video objects or enabling user-interaction with the scene. Moreover, object-based video compression, as defined in the MPEG-4 standard, can provide high compression ratios because the foreground objects can be sent independently from the background. In the case that the scene background is static, the background views can even be combined into a large panoramic sprite image, from which the current camera view is extracted. This results in a higher compression ratio since the sprite image for each scene only has to be sent once. A prerequisite for employing object-based video processing is automatic (or at least user-assisted semi-automatic) segmentation of the input video into semantic units, the video objects. This segmentation is a difficult problem because the computer does not have the vast amount of pre-knowledge that humans subconsciously use for object detection. Thus, even the simple definition of the desired output of a segmentation system is difficult. The subject of this thesis is to provide algorithms for segmentation that are applicable to common video material and that are computationally efficient. The thesis is conceptually separated into three parts. In Part I, an automatic segmentation system for general video content is described in detail. Part II introduces object models as a tool to incorporate userdefined knowledge about the objects to be extracted into the segmentation process. Part III concentrates on the modeling of camera motion in order to relate the observed camera motion to real-world camera parameters. The segmentation system that is described in Part I is based on a background-subtraction technique. The pure background image that is required for this technique is synthesized from the input video itself. Sequences that contain rotational camera motion can also be processed since the camera motion is estimated and the input images are aligned into a panoramic scene-background. This approach is fully compatible to the MPEG-4 video-encoding framework, such that the segmentation system can be easily combined with an object-based MPEG-4 video codec. After an introduction to the theory of projective geometry in Chapter 2, which is required for the derivation of camera-motion models, the estimation of camera motion is discussed in Chapters 3 and 4. It is important that the camera-motion estimation is not influenced by foreground object motion. At the same time, the estimation should provide accurate motion parameters such that all input frames can be combined seamlessly into a background image. The core motion estimation is based on a feature-based approach where the motion parameters are determined with a robust-estimation algorithm (RANSAC) in order to distinguish the camera motion from simultaneously visible object motion. Our experiments showed that the robustness of the original RANSAC algorithm in practice does not reach the theoretically predicted performance. An analysis of the problem has revealed that this is caused by numerical instabilities that can be significantly reduced by a modification that we describe in Chapter 4. The synthetization of static-background images is discussed in Chapter 5. In particular, we present a new algorithm for the removal of the foreground objects from the background image such that a pure scene background remains. The proposed algorithm is optimized to synthesize the background even for difficult scenes in which the background is only visible for short periods of time. The problem is solved by clustering the image content for each region over time, such that each cluster comprises static content. Furthermore, it is exploited that the times, in which foreground objects appear in an image region, are similar to the corresponding times of neighboring image areas. The reconstructed background could be used directly as the sprite image in an MPEG-4 video coder. However, we have discovered that the counterintuitive approach of splitting the background into several independent parts can reduce the overall amount of data. In the case of general camera motion, the construction of a single sprite image is even impossible. In Chapter 6, a multi-sprite partitioning algorithm is presented, which separates the video sequence into a number of segments, for which independent sprites are synthesized. The partitioning is computed in such a way that the total area of the resulting sprites is minimized, while simultaneously satisfying additional constraints. These include a limited sprite-buffer size at the decoder, and the restriction that the image resolution in the sprite should never fall below the input-image resolution. The described multisprite approach is fully compatible to the MPEG-4 standard, but provides three advantages. First, any arbitrary rotational camera motion can be processed. Second, the coding-cost for transmitting the sprite images is lower, and finally, the quality of the decoded sprite images is better than in previously proposed sprite-generation algorithms. Segmentation masks for the foreground objects are computed with a change-detection algorithm that compares the pure background image with the input images. A special effect that occurs in the change detection is the problem of image misregistration. Since the change detection compares co-located image pixels in the camera-motion compensated images, a small error in the motion estimation can introduce segmentation errors because non-corresponding pixels are compared. We approach this problem in Chapter 7 by integrating risk-maps into the segmentation algorithm that identify pixels for which misregistration would probably result in errors. For these image areas, the change-detection algorithm is modified to disregard the difference values for the pixels marked in the risk-map. This modification significantly reduces the number of false object detections in fine-textured image areas. The algorithmic building-blocks described above can be combined into a segmentation system in various ways, depending on whether camera motion has to be considered or whether real-time execution is required. These different systems and example applications are discussed in Chapter 8. Part II of the thesis extends the described segmentation system to consider object models in the analysis. Object models allow the user to specify which objects should be extracted from the video. In Chapters 9 and 10, a graph-based object model is presented in which the features of the main object regions are summarized in the graph nodes, and the spatial relations between these regions are expressed with the graph edges. The segmentation algorithm is extended by an object-detection algorithm that searches the input image for the user-defined object model. We provide two objectdetection algorithms. The first one is specific for cartoon sequences and uses an efficient sub-graph matching algorithm, whereas the second processes natural video sequences. With the object-model extension, the segmentation system can be controlled to extract individual objects, even if the input sequence comprises many objects. Chapter 11 proposes an alternative approach to incorporate object models into a segmentation algorithm. The chapter describes a semi-automatic segmentation algorithm, in which the user coarsely marks the object and the computer refines this to the exact object boundary. Afterwards, the object is tracked automatically through the sequence. In this algorithm, the object model is defined as the texture along the object contour. This texture is extracted in the first frame and then used during the object tracking to localize the original object. The core of the algorithm uses a graph representation of the image and a newly developed algorithm for computing shortest circular-paths in planar graphs. The proposed algorithm is faster than the currently known algorithms for this problem, and it can also be applied to many alternative problems like shape matching. Part III of the thesis elaborates on different techniques to derive information about the physical 3-D world from the camera motion. In the segmentation system, we employ camera-motion estimation, but the obtained parameters have no direct physical meaning. Chapter 12 discusses an extension to the camera-motion estimation to factorize the motion parameters into physically meaningful parameters (rotation angles, focal-length) using camera autocalibration techniques. The speciality of the algorithm is that it can process camera motion that spans several sprites by employing the above multi-sprite technique. Consequently, the algorithm can be applied to arbitrary rotational camera motion. For the analysis of video sequences, it is often required to determine and follow the position of the objects. Clearly, the object position in image coordinates provides little information if the viewing direction of the camera is not known. Chapter 13 provides a new algorithm to deduce the transformation between the image coordinates and the real-world coordinates for the special application of sport-video analysis. In sport videos, the camera view can be derived from markings on the playing field. For this reason, we employ a model of the playing field that describes the arrangement of lines. After detecting significant lines in the input image, a combinatorial search is carried out to establish correspondences between lines in the input image and lines in the model. The algorithm requires no information about the specific color of the playing field and it is very robust to occlusions or poor lighting conditions. Moreover, the algorithm is generic in the sense that it can be applied to any type of sport by simply exchanging the model of the playing field. In Chapter 14, we again consider panoramic background images and particularly focus ib their visualization. Apart from the planar backgroundsprites discussed previously, a frequently-used visualization technique for panoramic images are projections onto a cylinder surface which is unwrapped into a rectangular image. However, the disadvantage of this approach is that the viewer has no good orientation in the panoramic image because he looks into all directions at the same time. In order to provide a more intuitive presentation of wide-angle views, we have developed a visualization technique specialized for the case of indoor environments. We present an algorithm to determine the 3-D shape of the room in which the image was captured, or, more generally, to compute a complete floor plan if several panoramic images captured in each of the rooms are provided. Based on the obtained 3-D geometry, a graphical model of the rooms is constructed, where the walls are displayed with textures that are extracted from the panoramic images. This representation enables to conduct virtual walk-throughs in the reconstructed room and therefore, provides a better orientation for the user. Summarizing, we can conclude that all segmentation techniques employ some definition of foreground objects. These definitions are either explicit, using object models like in Part II of this thesis, or they are implicitly defined like in the background synthetization in Part I. The results of this thesis show that implicit descriptions, which extract their definition from video content, work well when the sequence is long enough to extract this information reliably. However, high-level semantics are difficult to integrate into the segmentation approaches that are based on implicit models. Intead, those semantics should be added as postprocessing steps. On the other hand, explicit object models apply semantic pre-knowledge at early stages of the segmentation. Moreover, they can be applied to short video sequences or even still pictures since no background model has to be extracted from the video. The definition of a general object-modeling technique that is widely applicable and that also enables an accurate segmentation remains an important yet challenging problem for further research

Role of Cis-regulatory Elements in Transcriptional Regulation: From Evolution to 4D Interactions

Transcriptional regulation is the principal mechanism in establishing cell-type specific gene activity by exploring an almost infinite space of different combinations of regulatory elements, transcription factors with high precision. Recent efforts have mapped thousands of candidate regulatory elements, of which a great portion is cell-type specific yet it is still unclear as to what fraction of these elements is functional, what genes these elements regulate, or how they are established in a cell-type specific manner. In this dissertation, I will discuss methods and approaches I developed to better understand the role of regulatory elements and transcription factors in gene expression regulation. First, by comparing the transcriptome and chromatin landscape between mouse and human innate immune cells I showed specific gene expression programs are regulated by highly conserved regulatory elements that contain a set of constrained sequence motifs, which can successfully classify gene-induction in both species. Next, using chromatin interactions I accurately defined functional enhancers and their target genes. This fine mapping dramatically improved the prediction of transcriptional changes. Finally, we built a supervised learning approach to detect the short DNA sequences motifs that regulate the activation of regulatory elements following LPS stimulation. This approach detected several transcription factors to be critical in remodeling the epigenetic landscape both across time and individuals. Overall this thesis addresses several important aspects of cis-regulatory elements in transcriptional regulation and started to derive principles and models of gene-expression regulation that address the fundamental question: “How do cis-regulatory elements drive cell-type-specific transcription?

Semantic Segmentation in 2D Videogames

This Master Thesis focuses on applying semantic segmentation, a computer vision technique, with the objective of improving the performance of deep-learning reinforcement models, and in particular, the performance over the original Super Mario Bros videogame. While humans can play a stage from a videogame like Super Mario Bros, and quickly identify from the elements in the screen what object is the character they are playing with, what are enemies and what elements are obstacles, this is not the case for neural networks, as they require a certain training to understand what is displayed in the screen. Using semantic segmentation, we can heavily simplify the frames from the videogame, and reduce visual information of elements in the screen to class and location, which is the most relevant information required to complete the game. In this work, a synthetic dataset generator that simulates frames from the Super Mario Bros videogame has been developed. This dataset has been used to train semantic segmentation deep-learning models which have been incorporated to a deep reinforcement learning algorithm with the objective of improving the performance of it. We have found that applying semantic segmentation as a frame processing method can actually help reinforcement learning models to train more efficiently and with better generalization. These results also suggest that there could be other computer vision techniques, like object detection or tracking, that could be found useful to help with the training of reinforcement learning algorithms, and they could be an interesting topic for future research