Causes of Catastrophic Forgetting in Class-Incremental Semantic Segmentation

Class-incremental learning for semantic segmentation (CiSS) is presently a highly researched field which aims at updating a semantic segmentation model by sequentially learning new semantic classes. A major challenge in CiSS is overcoming the effects of catastrophic forgetting, which describes the sudden drop of accuracy on previously learned classes after the model is trained on a new set of classes. Despite latest advances in mitigating catastrophic forgetting, the underlying causes of forgetting specifically in CiSS are not well understood. Therefore, in a set of experiments and representational analyses, we demonstrate that the semantic shift of the background class and a bias towards new classes are the major causes of forgetting in CiSS. Furthermore, we show that both causes mostly manifest themselves in deeper classification layers of the network, while the early layers of the model are not affected. Finally, we demonstrate how both causes are effectively mitigated utilizing the information contained in the background, with the help of knowledge distillation and an unbiased cross-entropy loss.Comment: currently under revie

Improving Replay-Based Continual Semantic Segmentation with Smart Data Selection

Continual learning for Semantic Segmentation (CSS) is a rapidly emerging field, in which the capabilities of the segmentation model are incrementally improved by learning new classes or new domains. A central challenge in Continual Learning is overcoming the effects of catastrophic forgetting, which refers to the sudden drop in accuracy on previously learned tasks after the model is trained on new classes or domains. In continual classification this challenge is often overcome by replaying a small selection of samples from previous tasks, however replay is rarely considered in CSS. Therefore, we investigate the influences of various replay strategies for semantic segmentation and evaluate them in class- and domain-incremental settings. Our findings suggest that in a class-incremental setting, it is critical to achieve a uniform distribution for the different classes in the buffer to avoid a bias towards newly learned classes. In the domain-incremental setting, it is most effective to select buffer samples by uniformly sampling from the distribution of learned feature representations or by choosing samples with median entropy. Finally, we observe that the effective sampling methods help to decrease the representation shift significantly in early layers, which is a major cause of forgetting in domain-incremental learning.Comment: Accepted at 2022 IEEE Conference on Intelligent Transportation Systems (ITSC 2022

Continual Learning for Class- and Domain-Incremental Semantic Segmentation

The field of continual deep learning is an emerging field and a lot of progress has been made. However, concurrently most of the approaches are only tested on the task of image classification, which is not relevant in the field of intelligent vehicles. Only recently approaches for class-incremental semantic segmentation were proposed. However, all of those approaches are based on some form of knowledge distillation. At the moment there are no investigations on replay-based approaches that are commonly used for object recognition in a continual setting. At the same time while unsupervised domain adaption for semantic segmentation gained a lot of traction, investigations regarding domain-incremental learning in an continual setting is not well-studied. Therefore, the goal of our work is to evaluate and adapt established solutions for continual object recognition to the task of semantic segmentation and to provide baseline methods and evaluation protocols for the task of continual semantic segmentation. We firstly introduce evaluation protocols for the class- and domain-incremental segmentation and analyze selected approaches. We show that the nature of the task of semantic segmentation changes which methods are most effective in mitigating forgetting compared to image classification. Especially, in class-incremental learning knowledge distillation proves to be a vital tool, whereas in domain-incremental learning replay methods are the most effective method

Dynamic landscapes and human dispersal patterns : tectonics, coastlines, and the reconstruction of human habitats

Studies of the impact of physical environment on human evolution usually focus on climate as the main external forcing agent of evolutionary and cultural change. In this paper we focus on changes in the physical character of the landscape driven by geophysical processes as an equally potent factor. Most of the landscapes where finds of early human fossils and artefacts are concentrated are ones that have been subjected to high levels of geological instability, either because of especially active tectonic processes associated with faulting and volcanic activity or because of proximity to coastlines subject to dramatic changes of geographical position and physical character by changes of relative sea level. These processes can have both beneficial effects, creating ecologically attractive conditions for human settlement, and deleterious or disruptive ones, creating barriers to movement, disruption of ecological conditions, or hazards to survival. Both positive and negative factors can have powerful selective effects on human behaviour and patterns of settlement and dispersal. We consider both these aspects of the interaction, develop a framework for the reconstruction and comparison of landscapes and landscape change at a variety of scales, and illustrate this with selected examples drawn from Africa and Arabia

Human Pose Estimation for Real-World Crowded Scenarios

Human pose estimation has recently made significant progress with the adoption of deep convolutional neural networks and many applications have attracted tremendous interest in recent years. However, many of these applications require pose estimation for human crowds, which still is a rarely addressed problem. For this purpose this work explores methods to optimize pose estimation for human crowds, focusing on challenges introduced with larger scale crowds like people in close proximity to each other, mutual occlusions, and partial visibility of people due to the environment. In order to address these challenges, multiple approaches are evaluated including: the explicit detection of occluded body parts, a data augmentation method to generate occlusions and the use of the synthetic generated dataset JTA [3]. In order to overcome the transfer gap of JTA originating from a low pose variety and less dense crowds, an extension dataset is created to ease the use for real-world applications

Controlled Crystallinity of TiO₂ Layers Grown by Atmospheric Pressure Spatial Atomic Layer Deposition and their Impact on Perovskite Solar Cell Efficiency

Atmospheric Pressure Spatial Atomic Layer Deposition (AP-SALD) is an upcoming deposition technique suitable for a variety of materials and combines the benefits of a regular atomic layer deposition with a significantly increased deposition rate at ambient conditions. In this work, amorphous and anatase TiO2 layers are fabricated by AP-SALD via systematic variation of process conditions such as temperature, reactant (H2O and O3), and posttreatment. The formed layers are characterized for their structural and optoelectronic properties and utilized as a hole-blocking layer in hybrid perovskite solar cells. It is found that TiO2 layers fabricated at elevated deposition temperatures possess strong anatase character but expose an unfavorable interface to the perovskite layer, promoting recombination and lowering the shunt resistance and open circuit voltage of the solar cells. Therefore, the interface is essential for efficient charge extraction, which can be significantly improved by controlling the process parameters.publishe

Demonstration of quantum-digital payments

Abstract Digital payments have replaced physical banknotes in many aspects of our daily lives. Similarly to banknotes, they should be easy to use, unique, tamper-resistant and untraceable, but additionally withstand digital attackers and data breaches. Current technology substitutes customers’ sensitive data by randomized tokens, and secures the payment’s uniqueness with a cryptographic function, called a cryptogram. However, computationally powerful attacks violate the security of these functions. Quantum technology comes with the potential to protect even against infinite computational power. Here, we show how quantum light can secure daily digital payments by generating inherently unforgeable quantum cryptograms. We implement the scheme over an urban optical fiber link, and show its robustness to noise and loss-dependent attacks. Unlike previously proposed protocols, our solution does not depend on long-term quantum storage or trusted agents and authenticated channels. It is practical with near-term technology and may herald an era of quantum-enabled security

Virucidal or Not Virucidal? That Is the Question—Predictability of Ionic Liquid’s Virucidal Potential in Biological Test Systems

For three decades now, ionic liquids (ILs), organic salts comprising only ions, have emerged as a new class of pharmaceuticals. Although recognition of the antimicrobial effects of ILs is growing rapidly, there is almost nothing known about their possible virucidal activities. This probably reflects the paucity of understanding virus inactivation. In this study, we performed a systematic analysis to determine the effect of specific structural motifs of ILs on three different biological test systems (viruses, bacteria and enzymes). Overall, the effects of 27 different ILs on two non-enveloped and one enveloped virus (P100, MS2 and Phi6), two Gram negative and one Gram positive bacteria (E. coli, P. syringae and L. monocytogenes) and one enzyme (Taq DNA polymerase) were investigated. Results show that while some ILs were virucidal, no clear structure activity relationships (SARs) could be identified for the non-enveloped viruses P100 and MS2. However, for the first time, a correlation has been demonstrated between the effects of ILs on enveloped viruses, bacteria and enzyme inhibition. These identified SARs serve as a sound starting point for further studies