A survey of real-time crowd rendering

In this survey we review, classify and compare existing approaches for real-time crowd rendering. We first overview character animation techniques, as they are highly tied to crowd rendering performance, and then we analyze the state of the art in crowd rendering. We discuss different representations for level-of-detail (LoD) rendering of animated characters, including polygon-based, point-based, and image-based techniques, and review different criteria for runtime LoD selection. Besides LoD approaches, we review classic acceleration schemes, such as frustum culling and occlusion culling, and describe how they can be adapted to handle crowds of animated characters. We also discuss specific acceleration techniques for crowd rendering, such as primitive pseudo-instancing, palette skinning, and dynamic key-pose caching, which benefit from current graphics hardware. We also address other factors affecting performance and realism of crowds such as lighting, shadowing, clothing and variability. Finally we provide an exhaustive comparison of the most relevant approaches in the field.Peer ReviewedPostprint (author's final draft

Representation and coding of 3D video data

Livrable D4.1 du projet ANR PERSEECe rapport a été réalisé dans le cadre du projet ANR PERSEE (n° ANR-09-BLAN-0170). Exactement il correspond au livrable D4.1 du projet

Variable-resolution Compression of Vector Data

The compression of spatial data is a promising solution to reduce the space of data storage and to decrease the transmission time of spatial data over the Internet. This paper proposes a new method for variable-resolution compression of vector data. Three key steps are encompassed in the proposed method, namely, the simplification of vector data via the elimination of vertices, the compression of removed vertices, and the decoding of the compressed vector data. The proposed compression method was implemented and applied to compress vector data to investigate its performance in terms of the compression ratio, distortions of geometric shapes. The results show that the proposed method provides a feasible and efficient solution for the compression of vector data, is able to achieve good compression ratios and maintains the main shape characteristics of the spatial objects within the compressed vector dat

Aggregated Text Transformer for Scene Text Detection

This paper explores the multi-scale aggregation strategy for scene text detection in natural images. We present the Aggregated Text TRansformer(ATTR), which is designed to represent texts in scene images with a multi-scale self-attention mechanism. Starting from the image pyramid with multiple resolutions, the features are first extracted at different scales with shared weight and then fed into an encoder-decoder architecture of Transformer. The multi-scale image representations are robust and contain rich information on text contents of various sizes. The text Transformer aggregates these features to learn the interaction across different scales and improve text representation. The proposed method detects scene texts by representing each text instance as an individual binary mask, which is tolerant of curve texts and regions with dense instances. Extensive experiments on public scene text detection datasets demonstrate the effectiveness of the proposed framework

Recurrent Contour-based Instance Segmentation with Progressive Learning

Contour-based instance segmentation has been actively studied, thanks to its flexibility and elegance in processing visual objects within complex backgrounds. In this work, we propose a novel deep network architecture, i.e., PolySnake, for contour-based instance segmentation. Motivated by the classic Snake algorithm, the proposed PolySnake achieves superior and robust segmentation performance with an iterative and progressive contour refinement strategy. Technically, PolySnake introduces a recurrent update operator to estimate the object contour iteratively. It maintains a single estimate of the contour that is progressively deformed toward the object boundary. At each iteration, PolySnake builds a semantic-rich representation for the current contour and feeds it to the recurrent operator for further contour adjustment. Through the iterative refinements, the contour finally progressively converges to a stable status that tightly encloses the object instance. Moreover, with a compact design of the recurrent architecture, we ensure the running efficiency under multiple iterations. Extensive experiments are conducted to validate the merits of our method, and the results demonstrate that the proposed PolySnake outperforms the existing contour-based instance segmentation methods on several prevalent instance segmentation benchmarks. The codes and models are available at https://github.com/fh2019ustc/PolySnake

Image synthesis based on a model of human vision

Modern computer graphics systems are able to construct renderings of such high quality that viewers are deceived into regarding the images as coming from a photographic source. Large amounts of computing resources are expended in this rendering process, using complex mathematical models of lighting and shading. However, psychophysical experiments have revealed that viewers only regard certain informative regions within a presented image. Furthermore, it has been shown that these visually important regions contain low-level visual feature differences that attract the attention of the viewer. This thesis will present a new approach to image synthesis that exploits these experimental findings by modulating the spatial quality of image regions by their visual importance. Efficiency gains are therefore reaped, without sacrificing much of the perceived quality of the image. Two tasks must be undertaken to achieve this goal. Firstly, the design of an appropriate region-based model of visual importance, and secondly, the modification of progressive rendering techniques to effect an importance-based rendering approach. A rule-based fuzzy logic model is presented that computes, using spatial feature differences, the relative visual importance of regions in an image. This model improves upon previous work by incorporating threshold effects induced by global feature difference distributions and by using texture concentration measures. A modified approach to progressive ray-tracing is also presented. This new approach uses the visual importance model to guide the progressive refinement of an image. In addition, this concept of visual importance has been incorporated into supersampling, texture mapping and computer animation techniques. Experimental results are presented, illustrating the efficiency gains reaped from using this method of progressive rendering. This visual importance-based rendering approach is expected to have applications in the entertainment industry, where image fidelity may be sacrificed for efficiency purposes, as long as the overall visual impression of the scene is maintained. Different aspects of the approach should find many other applications in image compression, image retrieval, progressive data transmission and active robotic vision

Towards Robust Real-Time Scene Text Detection: From Semantic to Instance Representation Learning

Due to the flexible representation of arbitrary-shaped scene text and simple pipeline, bottom-up segmentation-based methods begin to be mainstream in real-time scene text detection. Despite great progress, these methods show deficiencies in robustness and still suffer from false positives and instance adhesion. Different from existing methods which integrate multiple-granularity features or multiple outputs, we resort to the perspective of representation learning in which auxiliary tasks are utilized to enable the encoder to jointly learn robust features with the main task of per-pixel classification during optimization. For semantic representation learning, we propose global-dense semantic contrast (GDSC), in which a vector is extracted for global semantic representation, then used to perform element-wise contrast with the dense grid features. To learn instance-aware representation, we propose to combine top-down modeling (TDM) with the bottom-up framework to provide implicit instance-level clues for the encoder. With the proposed GDSC and TDM, the encoder network learns stronger representation without introducing any parameters and computations during inference. Equipped with a very light decoder, the detector can achieve more robust real-time scene text detection. Experimental results on four public datasets show that the proposed method can outperform or be comparable to the state-of-the-art on both accuracy and speed. Specifically, the proposed method achieves 87.2% F-measure with 48.2 FPS on Total-Text and 89.6% F-measure with 36.9 FPS on MSRA-TD500 on a single GeForce RTX 2080 Ti GPU.Comment: Accepted by ACM MM 202

Task-based Adaptation of Graphical Content in Smart Visual Interfaces

To be effective visual representations must be adapted to their respective context of use, especially in so-called Smart Visual Interfaces striving to present specifically those information required for the task at hand. This thesis proposes a generic approach that facilitate the automatic generation of task-specific visual representations from suitable task descriptions. It is discussed how the approach is applied to four principal content types raster images, 2D vector and 3D graphics as well as data visualizations, and how existing display techniques can be integrated into the approach.Effektive visuelle Repräsentationen müssen an den jeweiligen Nutzungskontext angepasst sein, insbesondere in sog. Smart Visual Interfaces, welche anstreben, möglichst genau für die aktuelle Aufgabe benötigte Informationen anzubieten. Diese Arbeit entwirft einen generischen Ansatz zur automatischen Erzeugung aufgabenspezifischer Darstellungen anhand geeigneter Aufgabenbeschreibungen. Es wird gezeigt, wie dieser Ansatz auf vier grundlegende Inhaltstypen Rasterbilder, 2D-Vektor- und 3D-Grafik sowie Datenvisualisierungen anwendbar ist, und wie existierende Darstellungstechniken integrierbar sind