Nouvelles méthodes de prédiction inter-images pour la compression d’images et de vidéos

Due to the large availability of video cameras and new social media practices, as well as the emergence of cloud services, images and videosconstitute today a significant amount of the total data that is transmitted over the internet. Video streaming applications account for more than 70% of the world internet bandwidth. Whereas billions of images are already stored in the cloud and millions are uploaded every day. The ever growing streaming and storage requirements of these media require the constant improvements of image and video coding tools. This thesis aims at exploring novel approaches for improving current inter-prediction methods. Such methods leverage redundancies between similar frames, and were originally developed in the context of video compression. In a first approach, novel global and local inter-prediction tools are associated to improve the efficiency of image sets compression schemes based on video codecs. By leveraging a global geometric and photometric compensation with a locally linear prediction, significant improvements can be obtained. A second approach is then proposed which introduces a region-based inter-prediction scheme. The proposed method is able to improve the coding performances compared to existing solutions by estimating and compensating geometric and photometric distortions on a semi-local level. This approach is then adapted and validated in the context of video compression. Bit-rate improvements are obtained, especially for sequences displaying complex real-world motions such as zooms and rotations. The last part of the thesis focuses on deep learning approaches for inter-prediction. Deep neural networks have shown striking results for a large number of computer vision tasks over the last years. Deep learning based methods proposed for frame interpolation applications are studied here in the context of video compression. Coding performance improvements over traditional motion estimation and compensation methods highlight the potential of these deep architectures.En raison de la grande disponibilité des dispositifs de capture vidéo et des nouvelles pratiques liées aux réseaux sociaux, ainsi qu’à l’émergence desservices en ligne, les images et les vidéos constituent aujourd’hui une partie importante de données transmises sur internet. Les applications de streaming vidéo représentent ainsi plus de 70% de la bande passante totale de l’internet. Des milliards d’images sont déjà stockées dans le cloud et des millions y sont téléchargés chaque jour. Les besoins toujours croissants en streaming et stockage nécessitent donc une amélioration constante des outils de compression d’image et de vidéo. Cette thèse vise à explorer des nouvelles approches pour améliorer les méthodes actuelles de prédiction inter-images. De telles méthodes tirent parti des redondances entre images similaires, et ont été développées à l’origine dans le contexte de la vidéo compression. Dans une première partie, de nouveaux outils de prédiction inter globaux et locaux sont associés pour améliorer l’efficacité des schémas de compression de bases de données d’image. En associant une compensation géométrique et photométrique globale avec une prédiction linéaire locale, des améliorations significatives peuvent être obtenues. Une seconde approche est ensuite proposée qui introduit un schéma deprédiction inter par régions. La méthode proposée est en mesure d’améliorer les performances de codage par rapport aux solutions existantes en estimant et en compensant les distorsions géométriques et photométriques à une échelle semi locale. Cette approche est ensuite adaptée et validée dans le cadre de la compression vidéo. Des améliorations en réduction de débit sont obtenues, en particulier pour les séquences présentant des mouvements complexes réels tels que des zooms et des rotations. La dernière partie de la thèse se concentre sur l’étude des méthodes d’apprentissage en profondeur dans le cadre de la prédiction inter. Ces dernières années, les réseaux de neurones profonds ont obtenu des résultats impressionnants pour un grand nombre de tâches de vision par ordinateur. Les méthodes basées sur l’apprentissage en profondeur proposéesà l’origine pour de l’interpolation d’images sont étudiées ici dans le contexte de la compression vidéo. Des améliorations en terme de performances de codage sont obtenues par rapport aux méthodes d’estimation et de compensation de mouvements traditionnelles. Ces résultats mettent en évidence le fort potentiel de ces architectures profondes dans le domaine de la compression vidéo

Representation Learning for Shape Decomposition, By Shape Decomposition

The ability to parse 3D objects into their constituent parts is essential for humans to understand and interact with the surrounding world. Imparting this skill in machines is important for various computer graphics, computer vision, and robotics tasks. Machines endowed with this skill can better interact with its surroundings, perform shape editing, texturing, recomposing, tracking, and animation. In this thesis, we ask two questions. First, how can machines decompose 3D shapes into their fundamental parts? Second, does the ability to decompose the 3D shape into these parts help learn useful 3D shape representations? In this thesis, we focus on parsing the shape into compact representations, such as parametric surface patches and Constructive Solid Geometry (CSG) primitives, which are also widely used representations in 3D modeling in computer graphics. Inspired by the advances in neural networks for 3D shape processing, we develop neural network approaches to tackle shape decomposition. First, we present CSGNet, a network architecture to parse shapes into CSG programs, which is trained using combination of supervised and reinforcement learning. Second, we present ParSeNet, a network architecture to decompose a shape into parametric surface patches (B-Spline) and geometric primitives (plane, cone, cylinder and sphere), trained on a large set of CAD models using supervised learning. The training of deep neural network architectures for 3D recognition and generation tasks requires a large amount of labeled datasets. We explore ways to alleviate this problem by relying on shape decomposition methods to guide the learning process. Towards that end, we first study the use of freely available metadata, albeit inconsistent, from shape repositories to learn 3D shape features. Later we show that learning to decompose a 3D shape into geometric primitives also helps in learning shape representations useful for semantic segmentation tasks. Finally, since most 3D shapes encountered in real life are textured, consisting of several fine-grained semantic parts, we propose a method to learn fine-grained representations for textured 3D shapes in a self-supervised manner by incorporating 3D geometric priors

Intelligent Sensors for Human Motion Analysis

The book, "Intelligent Sensors for Human Motion Analysis," contains 17 articles published in the Special Issue of the Sensors journal. These articles deal with many aspects related to the analysis of human movement. New techniques and methods for pose estimation, gait recognition, and fall detection have been proposed and verified. Some of them will trigger further research, and some may become the backbone of commercial systems

Machine Learning in Aerodynamic Shape Optimization

Machine learning (ML) has been increasingly used to aid aerodynamic shape optimization (ASO), thanks to the availability of aerodynamic data and continued developments in deep learning. We review the applications of ML in ASO to date and provide a perspective on the state-of-the-art and future directions. We first introduce conventional ASO and current challenges. Next, we introduce ML fundamentals and detail ML algorithms that have been successful in ASO. Then, we review ML applications to ASO addressing three aspects: compact geometric design space, fast aerodynamic analysis, and efficient optimization architecture. In addition to providing a comprehensive summary of the research, we comment on the practicality and effectiveness of the developed methods. We show how cutting-edge ML approaches can benefit ASO and address challenging demands, such as interactive design optimization. Practical large-scale design optimizations remain a challenge because of the high cost of ML training. Further research on coupling ML model construction with prior experience and knowledge, such as physics-informed ML, is recommended to solve large-scale ASO problems

State of the Art in Face Recognition

Notwithstanding the tremendous effort to solve the face recognition problem, it is not possible yet to design a face recognition system with a potential close to human performance. New computer vision and pattern recognition approaches need to be investigated. Even new knowledge and perspectives from different fields like, psychology and neuroscience must be incorporated into the current field of face recognition to design a robust face recognition system. Indeed, many more efforts are required to end up with a human like face recognition system. This book tries to make an effort to reduce the gap between the previous face recognition research state and the future state

Saliency-driven dynamic point cloud coding using projections onto images

Dissertação (mestrado)—Universidade de Brasília, Faculdade de Tecnologia, Departamento de Engenharia Elétrica, 2021.As regiões de interesse (ROI) têm sido utilizadas na codicação tradicional de imagens e vídeos para melhorar a qualidade do quadro em certas regiões, como rostos, em detrimento de outras áreas. No entanto, a ROI na compressão de nuvens de pontos não foi amplamente abordada, as- sim como a criação de mapas de saliência. Ambos os pontos são abordados neste trabalho. É difícil identicar diretamente atributos como rostos em nuvens de pontos esparsas e foi desenvolvido um método alternativo para o fazer. São utilizadas projeções ortográcas em planos 2D que são sub- metidas a algoritmos de visão computacional bem conhecidos. Uma vez identicada uma região de interesse, os seus pixels são retroprojetados nos voxels correspondentes. Ao repetir as projeções ao longo de muitas vistas, a informação de múltiplas projeções é agregada para formar um conjunto de voxels que se acredita conter a ROI ou serem os com maior valor de saliência. Como método não supervisionado, foi concebido um algoritmo para procurar as melhores vistas para projeções, utilizando informação de consistência temporal que é herdada de um quadro para outro. Foram utilizados algoritmos de detecção facial, tais como Viola-Jones, para determinar a ROI 2D e foram também utilizados algoritmos de criação de mapas de saliências bem estabelecidos para imagens bidimensionais. A m de utilizar a ROI para compressão, foi desenvolvida uma estratégia de codi- cação baseada num critério de distorção modicada que pode ser aplicado a muitos codicadores e é naturalmente aplicável ao codicador que utiliza a transformação hierárquica por região adap- tável (RAHT). Na sua essência, os bits (e a qualidade) são deslocados para a ROI a partir de áreas não-ROI, assumindo que as partes não-ROI são visualmente menos importantes e têm valores de saliência inferiores. Os resultados revelam uma grande melhoria subjetiva global ao melhorar con- sideravelmente o ROI à custa de uma pequena degradação das regiões de menor saliência.Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES).Regions of interest (ROI) have been used in traditional image and video coding to improve im- age quality in certain regions, like faces, at the expense of other areas. Nevertheless, ROI in point cloud compression have not been properly addressed, nor has the creation of saliency maps. Both points are addressed in this work. It is hard to directly identify features such as faces in unconnected point clouds and an alternative method to do so was developed. Orthographic projections in 2D planes which are subject to well established computer vision algorithms are used. Once an image region is identied, their pixels are back-projected onto the corresponding voxels. By repeating the projections over many orientations, the information of the many back projections is fused to form a collection of voxels believed to contain the ROI or to be the most salient. As an unsupervised method, it was devised an algorithm to search the projection orientations for the best views, which include temporal consistency information which is inherited from one frame to another. Face de- tection algorithms, such as Viola-Jones, were used to determine the 2D ROI and well established saliency map creation algorithms were also used in the 2D image case. In order to use ROI for com- pression, it was developed an encoding strategy based on a modied distortion criterion that can be applied to many coders and is naturally applicable to the region-adaptive hierarchical transform (RAHT) based coder, which is being adapted into compression standards. In essence, bits (and quality) are shifted towards the ROI from non-ROI areas, assuming non-ROI parts are visually less important and have lower salience values. Results reveal large overall subjective improvement by greatly improving the ROI at the expense of a small degradation of textured regions of lower salience

Recent Developments in Smart Healthcare

Medicine is undergoing a sector-wide transformation thanks to the advances in computing and networking technologies. Healthcare is changing from reactive and hospital-centered to preventive and personalized, from disease focused to well-being centered. In essence, the healthcare systems, as well as fundamental medicine research, are becoming smarter. We anticipate significant improvements in areas ranging from molecular genomics and proteomics to decision support for healthcare professionals through big data analytics, to support behavior changes through technology-enabled self-management, and social and motivational support. Furthermore, with smart technologies, healthcare delivery could also be made more efficient, higher quality, and lower cost. In this special issue, we received a total 45 submissions and accepted 19 outstanding papers that roughly span across several interesting topics on smart healthcare, including public health, health information technology (Health IT), and smart medicine