Depth of Field Segmentation for Near-Lossless Image Compression and 3D Reconstruction

Over the years, photometric 3d reconstruction gained increasing importance in several disciplines, especially in cultural heritage preservation. While increasing sizes of images and datasets enhanced the overall reconstruction results, requirements in storage got immense. Additionally, unsharp areas in the background have a negative influence on 3d reconstructions algorithms. Handling the sharp foreground differently from the background simultaneously helps to reduce storage size requirements and improves 3d reconstruction results. In this paper, we examine regions outside the Depth of Field (DoF) and eliminate their inaccurate information to 3d reconstructions. We extract DoF maps from the images and use them to handle the foreground and background with different compression backends making sure that the actual object is compressed losslessly. Our algorithm achieves compression rates between 1:8 and 1:30 depending on the artifact and DoF size and improves the 3d reconstruction

Visual Human Traits Recognition

In this thesis state-of-the-art machine vision and learning technologies are applied to train a computer to perform human-like recognition from experience and classify pedestrians from image and video data with respect to directional orientation, gender, age group, and body physique. In order to perform the classification task, distinctive visual traits are extracted and learned from the full human body. Compared to the many successful studies, which exclusively focus on faces, full body based classification has been rarely addressed before, partly because much harder challenges have to be dealt with, such as the high variety of body postures, clothing styles and irritating background. Nevertheless, if no faces are exposed, it is often the only option. To the best of the author's knowledge, this study is the first to address the problem of gender and age recognition from low resolution full body images of persons in arbitrary upright postures, without being constrained to only frontal views. In the process numerous configurations of image descriptors and classifiers are investigated, with regard to their suitability for visual person profiling. Features include histograms of oriented gradients (HOG) as shape descriptors and local RGB histograms as color descriptors. These image representations are learned and combined by multiple support vector machines (SVM) in a hierarchical classifier structure. Thus, the most discriminating visual traits could be identified and their effect evaluated, whereby the fusion of multiple cues representing shape and color resulted in an improvement of the classification performance. Best and robust results, 86 gender recognition accuracy, were achieved by the additional introduction of temporal integration strategies when classifying tracked people from video. Moreover, the evaluated practical approach is economically motivated and integrated into a software application that serves as prototype of a novel automated audience measurement system for adaptive digital signage, whose major design and use case is demonstrated and discussed. As an additional contribution, the large, manually performed annotations (utilized for the classifier training and testing), which refer to a compilation of online available image and video datasets (most originally designed for people detection and tracking tasks), are planned to be published. In dieser Arbeit werden moderne, anerkannte Methoden aus den Bereichen maschinelles Sehen und Lernen angewendet, um einen Computer zur menschlichen Wahrnehmung zu befähigen und ihm so das Klassifizieren von Fußgängern nach Laufrichtung, Geschlecht, Altersgruppe und Körperstatur zu ermöglichen. Zur Durchführung dieser Aufgabe werden charakteristische visuelle Merkmale vom gesamten menschlichen Körper extrahiert. Im Vergleich zu den zahlreichen wissenschaftlichen Studien, die sich ausschließlich auf Gesichter spezialisieren, wurde Ganzkörperklassifikation bisher relativ selten behandelt, weil es zusätzliche Herausforderungen mit sich bringt, z.B. durch die mögliche Vielfalt an Körperhaltungen, Kleidungsstilen und irritierendem Hintergrund. Nach bestem Wissen ist diese Studie die erste, die sich mit Geschlechts- und Altersklassifizierung befasst anhand von niedrigauflösenden Ganzkörperbildern von Personen in beliebiger, aufrechter Körperhaltung, ohne sich dabei nur auf Frontalansichten zu beschränken. Hierfür wurden zahlreiche Konfigurationen von Bilddeskriptoren und Klassifikatoren untersucht, wie z.B. Histogramme orientierter Gradienten um Formen zu beschreiben und RGB Histogramme um Farbe zu erfassen und das in Kombination mit hierarchisch angeordneten Support Vector Machines. So konnten aussagekräftige visuelle Merkmale identifiziert und deren Wirkung bewertet werden. Es zeigte sich, dass eine Verschmelzung von Form- und Farbmerkmalen sich positiv auf die Qualität der Klassifikation auswirkt. Beste Ergebnisse, 86 Genauigkeit der Geschlechterklassifikation, wurden erzielt durch den zusätzlichen Einsatz von zeitlicher Integration über den Beobachtungszeitraum von Personen in Videodaten. Der evaluierte Ansatz wird wirtschaftlich motiviert und in eine Software-Anwendung integriert, die als Prototyp eines neuartigen, automatisierten Zuschauermessungssystems dient. Aufbau und Wirkungsweise der Applikation werden demonstriert und diskutiert. Als zusätzlichen Beitrag, ist geplant die manuell erstellte und verwendete Datensatzannotation zu veröffentlichen

Visual Human Traits Recognition

In this thesis state-of-the-art machine vision and learning technologies are applied to train a computer to perform human-like recognition from experience and classify pedestrians from image and video data with respect to directional orientation, gender, age group, and body physique. In order to perform the classification task, distinctive visual traits are extracted and learned from the full human body. Compared to the many successful studies, which exclusively focus on faces, full body based classification has been rarely addressed before, partly because much harder challenges have to be dealt with, such as the high variety of body postures, clothing styles and irritating background. Nevertheless, if no faces are exposed, it is often the only option. To the best of the author's knowledge, this study is the first to address the problem of gender and age recognition from low resolution full body images of persons in arbitrary upright postures, without being constrained to only frontal views. In the process numerous configurations of image descriptors and classifiers are investigated, with regard to their suitability for visual person profiling. Features include histograms of oriented gradients (HOG) as shape descriptors and local RGB histograms as color descriptors. These image representations are learned and combined by multiple support vector machines (SVM) in a hierarchical classifier structure. Thus, the most discriminating visual traits could be identified and their effect evaluated, whereby the fusion of multiple cues representing shape and color resulted in an improvement of the classification performance. Best and robust results, 86 gender recognition accuracy, were achieved by the additional introduction of temporal integration strategies when classifying tracked people from video. Moreover, the evaluated practical approach is economically motivated and integrated into a software application that serves as prototype of a novel automated audience measurement system for adaptive digital signage, whose major design and use case is demonstrated and discussed. As an additional contribution, the large, manually performed annotations (utilized for the classifier training and testing), which refer to a compilation of online available image and video datasets (most originally designed for people detection and tracking tasks), are planned to be published. In dieser Arbeit werden moderne, anerkannte Methoden aus den Bereichen maschinelles Sehen und Lernen angewendet, um einen Computer zur menschlichen Wahrnehmung zu befähigen und ihm so das Klassifizieren von Fußgängern nach Laufrichtung, Geschlecht, Altersgruppe und Körperstatur zu ermöglichen. Zur Durchführung dieser Aufgabe werden charakteristische visuelle Merkmale vom gesamten menschlichen Körper extrahiert. Im Vergleich zu den zahlreichen wissenschaftlichen Studien, die sich ausschließlich auf Gesichter spezialisieren, wurde Ganzkörperklassifikation bisher relativ selten behandelt, weil es zusätzliche Herausforderungen mit sich bringt, z.B. durch die mögliche Vielfalt an Körperhaltungen, Kleidungsstilen und irritierendem Hintergrund. Nach bestem Wissen ist diese Studie die erste, die sich mit Geschlechts- und Altersklassifizierung befasst anhand von niedrigauflösenden Ganzkörperbildern von Personen in beliebiger, aufrechter Körperhaltung, ohne sich dabei nur auf Frontalansichten zu beschränken. Hierfür wurden zahlreiche Konfigurationen von Bilddeskriptoren und Klassifikatoren untersucht, wie z.B. Histogramme orientierter Gradienten um Formen zu beschreiben und RGB Histogramme um Farbe zu erfassen und das in Kombination mit hierarchisch angeordneten Support Vector Machines. So konnten aussagekräftige visuelle Merkmale identifiziert und deren Wirkung bewertet werden. Es zeigte sich, dass eine Verschmelzung von Form- und Farbmerkmalen sich positiv auf die Qualität der Klassifikation auswirkt. Beste Ergebnisse, 86 Genauigkeit der Geschlechterklassifikation, wurden erzielt durch den zusätzlichen Einsatz von zeitlicher Integration über den Beobachtungszeitraum von Personen in Videodaten. Der evaluierte Ansatz wird wirtschaftlich motiviert und in eine Software-Anwendung integriert, die als Prototyp eines neuartigen, automatisierten Zuschauermessungssystems dient. Aufbau und Wirkungsweise der Applikation werden demonstriert und diskutiert. Als zusätzlichen Beitrag, ist geplant die manuell erstellte und verwendete Datensatzannotation zu veröffentlichen

Real-time Full-body Visual Traits Recognition from Image Sequences

The automatic recognition of human visual traits from images is a challenging computer vision task. Visual traits describe for example gender and age, or other properties of a person that can be derived from visual appearance. Gathering anonymous knowledge about people from visual cues bears potential for many interesting applications, as for example in the area of human machine interfacing, targeted advertisement or video surveillance. Most related work investigates visual traits recognition from facial features of a person, with good recognition performance. Few systems have recently applied recognition on low resolution full-body images, which shows lower performance than the facial regions but already can deliver classification results even if no face is visible. Obviously full-body classification is more challenging, mainly due to large variations in body pose, clothing and occlusion. In our study we present an approach to human visual traits recognition, based on Histogram of oriented Gradients (HoG), colour features and Support Vector Machines (SVM). In this experimental study we focus on gender classification. Motivated by our application of real-time adaptive advertisement on public situated displays, and unlike previous works, we perform a thorough evaluation on much more comprehensive datasets that include hard cases like side- and back views. The extended annotations used in our evaluation will be published. We further show that a hierarchical classification scheme to disambiguate a person's directional orientation and additional colour features can increase recognition rates. Finally, we demonstrate that temporal integration of per-frame classification scores significantly improves the overall classification performance for tracked individuals and clearly outperforms current state-of-the-art accuracy for single images

Whole Genome Sequence Analysis of a Prototype Strain of the Novel Putative Rotavirus Species L

Rotaviruses infect humans and animals and are a main cause of diarrhea. They are non-enveloped viruses with a genome of 11 double-stranded RNA segments. Based on genome analysis and amino acid sequence identities of the capsid protein VP6, the rotavirus species A to J (RVA-RVJ) have been defined so far. In addition, rotaviruses putatively assigned to the novel rotavirus species K (RVK) and L (RVL) have been recently identified in common shrews (Sorex araneus), based on partial genome sequences. Here, the complete genome sequence of strain KS14/0241, a prototype strain of RVL, is presented. The deduced amino acid sequence for VP6 of this strain shows only up to 47% identity to that of RVA to RVJ reference strains. Phylogenetic analyses indicate a clustering separated from the established rotavirus species for all 11 genome segments of RVL, with the closest relationship to RVH and RVJ within the phylogenetic RVB-like clade. The non-coding genome segment termini of RVL showed conserved sequences at the 5′-end (positive-sense RNA strand), which are common to all rotaviruses, and those conserved among the RVB-like clade at the 3′-end. The results are consistent with a classification of the virus into a novel rotavirus species L

Effect of Polarization on RGB Imaging and Color Accuracy/Fidelity

The use of polarization while trying to keep the digital color reproduction accuracy at its finest is very challenging due to how polarization is interacting and affecting the light spectrum itself and due to the quality of the used polarization materials. Our study on RGB imaging and color reproduction’s fidelity with and without polarization shows that a cross circular polarization (on a camera lens and light source) will have a major impact on how a linear grayscale, whether it has a semi-glossy or matt finishing, would be reproduced in contrast to no polarization at all. A major loss in deep black shades in the case of a semi-glossy grayscale is unmistakable. In addition to a noticeable shift in both lightness and Chroma components regardless of the grayscale’s finishing but depending rather on the used color target for correction. DE00 could not paint the full picture about color fidelity despite its low conformant reported values. Whereas, a closer visual inspection of the color components separately (lightness and Chroma) reveals color reproduction problems caused by polarization

Genome analysis of the novel putative rotavirus species K

Rotaviruses are causative agents of diarrhea in humans and animals. Currently, the species rotavirus A-J (RVA-RVJ) and the putative species RVK and RVL are defined, mainly based on their genome sequence identities. RVK strains were first identified in 2019 in common shrews (Sorex aranaeus) in Germany; however, only short sequence fragments were available so far. Here, we analyzed the complete coding regions of strain RVK/shrew-wt/GER/KS14–0241/2013, which showed highest sequence identities with RVC. The amino acid sequence identity of VP6, which is used for rotavirus species definition, reached only 51% with other rotavirus reference strains thus confirming classification of RVK as a separate species. Phylogenetic analyses for the deduced amino acid sequences of all 11 virus proteins showed, that for most of them RVK and RVC formed a common branch within the RVA-like phylogenetic clade. Only the tree for the highly variable NSP4 showed a different branching; however, with very low bootstrap support. Comparison of partial nucleotide sequences of other RVK strains from common shrews of different regions in Germany indicated a high degree of sequence variability (61–97% identity) within the putative species. These RVK strains clustered separately from RVC genotype reference strains in phylogenetic trees indicating diversification of RVK independent from RVC. The results indicate that RVK represents a novel rotavirus species, which is most closely related to RVC

3DHOG for geometric similarity measurement and retrieval for digital cultural heritage archives

With projects such as CultLab3D, 3D Digital preservation of cultural heritage will become more affordable and with this, the number of 3D-models representing scanned artefacts will dramatically increase. However, once mass digitization is possible, the subsequent bottleneck to overcome is the annotation of cultural heritage artefacts with provenance data. Current annotation tools are mostly based on textual input, eventually being able to link an artefact to documents, pictures, videos and only some tools already support 3D models. Therefore, we envisage the need to aid curators by allowing for fast, web-based, semi-automatic, 3D-centered annotation of artefacts with metadata. In this paper we give an overview of various technologies we are currently developing to address this issue. On one hand we want to store 3D models with similarity descriptors which are applicable independently of different 3D model quality levels of the same artefact. The goal is to retrieve and suggest to the curator metadata of already annotated similar artefacts for a new artefact to be annotated, so he can eventually reuse and adapt it to the current case. In addition we describe our web-based, 3D-centered annotation tool with meta- and object repositories supporting various databases and ontologies such as CIDOC-CRM

Automated 3D Mass Digitization for the GLAM Sector

The European Cultural Heritage Strategy for the 21st century has led to an increased demand for fast, efficient and faithful 3D digitization technologies for cultural heritage artefacts. Yet, unlike the digital acquisition of cultural goods in 2D which is widely used and automated today, 3D digitization often still requires significant manual intervention, time and money. To overcome this, the authors have developed CultLab3D, the world’s first fully automatic 3D mass digitization technology for collections of three-dimensional objects. 3D scanning robots such as the CultArm3D-P are specifically designed to automate the entire 3D digitization process thus allowing to capture and archive objects on a large-scale and produce highly accurate photo-realistic representations