Lossless Compression of Point Cloud Sequences Using Sequence Optimized CNN Models

In this paper we propose a new paradigm for encoding the geometry of dense point cloud sequences, where a convolutional neural network (CNN), which estimates the encoding distributions, is optimized on several frames of the sequence to be compressed. We adopt lightweight CNN structures, we perform training as part of the encoding process and the CNN parameters are transmitted as part of the bitstream. The newly proposed encoding scheme operates on the octree representation for each point cloud, consecutively encoding each octree resolution level. At every octree resolution level, the voxel grid is traversed section-by-section (each section being perpendicular to a selected coordinate axis), and in each section, the occupancies of groups of two-by-two voxels are encoded at once in a single arithmetic coding operation. A context for the conditional encoding distribution is defined for each two-by-two group of voxels based on the information available about the occupancy of the neighboring voxels in the current and lower resolution layers of the octree. The CNN estimates the probability mass functions of the occupancy patterns of all the voxel groups from one section in four phases. In each new phase, the contexts are updated with the occupancies encoded in the previous phase, and each phase estimates the probabilities in parallel, providing a reasonable trade-off between the parallelism of the processing and the informativeness of the contexts. The CNN training time is comparable to the time spent in the remaining encoding steps, leading to competitive overall encoding times. The bitrates and encoding-decoding times compare favorably with those of recently published compression schemes.publishedVersionPeer reviewe

Point cloud data compression

The rapid growth in the popularity of Augmented Reality (AR), Virtual Reality (VR), and Mixed Reality (MR) experiences have resulted in an exponential surge of three-dimensional data. Point clouds have emerged as a commonly employed representation for capturing and visualizing three-dimensional data in these environments. Consequently, there has been a substantial research effort dedicated to developing efficient compression algorithms for point cloud data. This Master's thesis aims to investigate the current state-of-the-art lossless point cloud geometry compression techniques, explore some of these techniques in more detail and then propose improvements and/or extensions to enhance them and provide directions for future work on this topic

Compressão do sinal de cor de uma nuvem de pontos baseada em cortes de geometria

Trabalho de conclusão de curso (graduação)—Universidade de Brasília, Faculdade de Tecnologia, Departamento de Engenharia Elétrica, 2019.Point clouds, ou nuvens de pontos, são um método de representação de imagens tridimensionais cada vez mais difundido. As imagens em 3D são úteis não só na criação de ambientes imersivos com fins de entretenimento, mas também nas mais diversas apli cações industriais, acadêmicas e até mesmo forenses. Point clouds são adaptadas a muitas tarefas de visão computacional, tendo destaque entre elas aplicações em carros autônomos. De fato, dados representados por nuvens de pontos gerados por sensores como o LiDAR já são utilizados para navegação e segurança destes veículos. O volume de dados deste tipo de aplicação vem crescendo exponencialmente, e estas informações precisam ser transmiti das de maneira eficiente por canais cuja capacidade é naturalmente limitada. Isto cria um desafio: como representar point clouds de maneira eficiente? Este trabalho visa resolver uma parte deste problema. Aqui é proposto um esquema de codificação sem perdas do sinal de cor de imagens tridimensionais representadas em point clouds. O procedimento desenvolvido utiliza informações oriundas da geometria das nuvens para agrupar pontos parecidos. As sequências de pontos assim geradas apresentam redundância no sinal de cor. Esta redundância é explorada com a utilização de um codificador diferencial. O sinal gerado por esse passo é por sua vez alimentado a um codificador aritmético adapta tivo. Os algoritmos propostos apresentaram taxas entre 10 e 17 bpov para as nuvens de pontos estudadas. Os algoritmos desenvolvidos foram implementados em linguagem de programação Python com o auxílio de bibliotecas de código aberto.Point clouds are computer representations of three-dimensional objects. They are used in creating immersive environments in virtual reality; in the quality control of man ufacturing processes; and even in crime scene investigations. One growing application of this technology is the acquisition and processing of environment data for autonomous vehicles, where point cloud representations generated by LiDAR systems are useful for computational vision tasks such as route planning and collision avoidance. The increasing volume of this type of data being acquired and processed creates a problem: how can this information be represented efficiently? Inherent limitations in storage and transmission could hinder the development of novel applications. Thus, the compression of point cloud data is crucial to the spread of those new technologies. This work aims to further the efforts already made to tackle this challenge. Here we present a compression scheme for a point cloud’s color attributes. The scheme consists in processing the cloud’s geometry and segmenting it into groups of points we call filaments, which are arrays of voxels that are transmitted sequentially. Each filament’s color signal is fed to a differential encoder, the output of which is encoded using adaptive arithmetic compression. The proposed algorithms reached bitrates between 10 and 17 bpov for the example point clouds. All the processes were implemented using open-source tools and most of the work was done in Python

Visual and Geometric Data Compression for Immersive Technologies

The contributions of this thesis are new compression algorithms for light field images and point cloud geometry. Light field imaging attracted wide attention in the recent decade, partly due to emergence of relatively low-cost handheld light field cameras designed for commercial purposes whereas point clouds are used more and more frequently in immersive technologies, replacing other forms of 3D representation. We obtain successful coding performance by combining conventional image processing methods, entropy coding, learning-based disparity estimation and optimization of neural networks for context probability modeling. On the light field coding side, we develop a lossless light field coding method which uses learning-based disparity estimations to predict any view in a light field from a set of reference views. On the point cloud geometry compression side, we develop four different algorithms. The first two of these algorithms follow the so-called bounding volumes approach which initially represents a part of the point cloud in two depth maps where the remaining points of the cloud are contained in a bounding volume which can be derived using only the two depth maps that are losslessly transmitted. One of the two algorithms is a lossy coder that reconstructs some of the remaining points in several steps which involve conventional image processing and image coding techniques. The other one is a lossless coder which applies a novel context arithmetic coding approach involving gradual expansion of the reconstructed point cloud into neighboring voxels. The last two of the proposed point cloud compression algorithms use neural networks for context probability modeling for coding the octree representation of point clouds using arithmetic coding. One of these two algorithms is a learning-based intra-frame coder which requires an initial training stage on a set of training point clouds. The lastly presented algorithm is an inter-frame (sequence) encoder which incorporates the neural network training into the encoding stage, thus for each sequence of point clouds, a specific neural network model is optimized which is also transmitted as a header in the bitstream