24 research outputs found
Learning Representations on the Unit Sphere: Application to Online Continual Learning
We use the maximum a posteriori estimation principle for learning
representations distributed on the unit sphere. We derive loss functions for
the von Mises-Fisher distribution and the angular Gaussian distribution, both
designed for modeling symmetric directional data. A noteworthy feature of our
approach is that the learned representations are pushed toward fixed
directions, allowing for a learning strategy that is resilient to data drift.
This makes it suitable for online continual learning, which is the problem of
training neural networks on a continuous data stream, where multiple
classification tasks are presented sequentially so that data from past tasks
are no longer accessible, and data from the current task can be seen only once.
To address this challenging scenario, we propose a memory-based representation
learning technique equipped with our new loss functions. Our approach does not
require negative data or knowledge of task boundaries and performs well with
smaller batch sizes while being computationally efficient. We demonstrate with
extensive experiments that the proposed method outperforms the current
state-of-the-art methods on both standard evaluation scenarios and realistic
scenarios with blurry task boundaries. For reproducibility, we use the same
training pipeline for every compared method and share the code at
https://t.ly/SQTj.Comment: 16 pages, 4 figures, under revie
Contrastive Learning for Online Semi-Supervised General Continual Learning
We study Online Continual Learning with missing labels and propose SemiCon, a
new contrastive loss designed for partly labeled data. We demonstrate its
efficiency by devising a memory-based method trained on an unlabeled data
stream, where every data added to memory is labeled using an oracle. Our
approach outperforms existing semi-supervised methods when few labels are
available, and obtain similar results to state-of-the-art supervised methods
while using only 2.6% of labels on Split-CIFAR10 and 10% of labels on
Split-CIFAR100.Comment: Accepted at ICIP'2
Domain-Aware Augmentations for Unsupervised Online General Continual Learning
Continual Learning has been challenging, especially when dealing with
unsupervised scenarios such as Unsupervised Online General Continual Learning
(UOGCL), where the learning agent has no prior knowledge of class boundaries or
task change information. While previous research has focused on reducing
forgetting in supervised setups, recent studies have shown that self-supervised
learners are more resilient to forgetting. This paper proposes a novel approach
that enhances memory usage for contrastive learning in UOGCL by defining and
using stream-dependent data augmentations together with some implementation
tricks. Our proposed method is simple yet effective, achieves state-of-the-art
results compared to other unsupervised approaches in all considered setups, and
reduces the gap between supervised and unsupervised continual learning. Our
domain-aware augmentation procedure can be adapted to other replay-based
methods, making it a promising strategy for continual learning.Comment: Accepted to BMVC'2
New metrics for analyzing continual learners
Deep neural networks have shown remarkable performance when trained on
independent and identically distributed data from a fixed set of classes.
However, in real-world scenarios, it can be desirable to train models on a
continuous stream of data where multiple classification tasks are presented
sequentially. This scenario, known as Continual Learning (CL) poses challenges
to standard learning algorithms which struggle to maintain knowledge of old
tasks while learning new ones. This stability-plasticity dilemma remains
central to CL and multiple metrics have been proposed to adequately measure
stability and plasticity separately. However, none considers the increasing
difficulty of the classification task, which inherently results in performance
loss for any model. In that sense, we analyze some limitations of current
metrics and identify the presence of setup-induced forgetting. Therefore, we
propose new metrics that account for the task's increasing difficulty. Through
experiments on benchmark datasets, we demonstrate that our proposed metrics can
provide new insights into the stability-plasticity trade-off achieved by models
in the continual learning environment.Comment: 6 pages, presented at MIRU 202
Beam tests of the trigger and digital processing electronics for the electromagnetic calorimeter of the CMS experiment
A prototype of the trigger and digital processing electronics for the electromagnetic calorimeter of the CMS experiment, coupled to a prototype of the PbWO4 crystal calorimeter, was tested during summer 96 in the H4 beamline at the CERN SPS. A very successful operation was achieved for this system, which runs in synchronous and pipelined mode at the LHC clock frequency, and performs the basic trigger and data acquisition functions needed in the CMS electromagnetic calorimeter. The performance of the trigger front-end electronics is well within the established requirements: a highly efficient bunch crossing identification ( > 99.9%), a good trigger energy resolution ( s/E ~9%/sq( E)+2%) and a highly efficient electron cluster shape identification ( ~99%) have been achieved. The FERMI digitizing system based on a dynamic analog compressor and a sampling ADC showed a very good perform ance, in particular the energy resolution for 150 GeV electrons was 0.54%, equal to the resolution obtained with a conventional charge integration ADC system
Energy Resolution Performance of the CMS Electromagnetic Calorimeter
The energy resolution performance of the CMS lead tungstate crystal electromagnetic calorimeter is presented. Measurements were made with an electron beam using a fully equipped supermodule of the calorimeter barrel. Results are given both for electrons incident on the centre of crystals and for electrons distributed uniformly over the calorimeter surface. The electron energy is reconstructed in matrices of 3 times 3 or 5 times 5 crystals centred on the crystal containing the maximum energy. Corrections for variations in the shower containment are applied in the case of uniform incidence. The resolution measured is consistent with the design goals
Altimetry for the future: Building on 25 years of progress
In 2018 we celebrated 25 years of development of radar altimetry, and the progress achieved by this methodology in the fields of global and coastal oceanography, hydrology, geodesy and cryospheric sciences. Many symbolic major events have celebrated these developments, e.g., in Venice, Italy, the 15th (2006) and 20th (2012) years of progress and more recently, in 2018, in Ponta Delgada, Portugal, 25 Years of Progress in Radar Altimetry. On this latter occasion it was decided to collect contributions of scientists, engineers and managers involved in the worldwide altimetry community to depict the state of altimetry and propose recommendations for the altimetry of the future. This paper summarizes contributions and recommendations that were collected and provides guidance for future mission design, research activities, and sustainable operational radar altimetry data exploitation. Recommendations provided are fundamental for optimizing further scientific and operational advances of oceanographic observations by altimetry, including requirements for spatial and temporal resolution of altimetric measurements, their accuracy and continuity. There are also new challenges and new openings mentioned in the paper that are particularly crucial for observations at higher latitudes, for coastal oceanography, for cryospheric studies and for hydrology. The paper starts with a general introduction followed by a section on Earth System Science including Ocean Dynamics, Sea Level, the Coastal Ocean, Hydrology, the Cryosphere and Polar Oceans and the ââGreenâ Ocean, extending the frontier from biogeochemistry to marine ecology. Applications are described in a subsequent section, which covers Operational Oceanography, Weather, Hurricane Wave and Wind Forecasting, Climate projection. Instrumentsâ development and satellite missionsâ evolutions are described in a fourth section. A fifth section covers the key observations that altimeters provide and their potential complements, from other Earth observation measurements to in situ data. Section 6 identifies the data and methods and provides some accuracy and resolution requirements for the wet tropospheric correction, the orbit and other geodetic requirements, the Mean Sea Surface, Geoid and Mean Dynamic Topography, Calibration and Validation, data accuracy, data access and handling (including the DUACS system). Section 7 brings a transversal view on scales, integration, artificial intelligence, and capacity building (education and training). Section 8 reviews the programmatic issues followed by a conclusion
Altimetry for the future: building on 25 years of progress
In 2018 we celebrated 25âŻyears of development of radar altimetry, and the progress achieved by this methodology in the fields of global and coastal oceanography, hydrology, geodesy and cryospheric sciences. Many symbolic major events have celebrated these developments, e.g., in Venice, Italy, the 15th (2006) and 20th (2012) years of progress and more recently, in 2018, in Ponta Delgada, Portugal, 25 Years of Progress in Radar Altimetry. On this latter occasion it was decided to collect contributions of scientists, engineers and managers involved in the worldwide altimetry community to depict the state of altimetry and propose recommendations for the altimetry of the future. This paper summarizes contributions and recommendations that were collected and provides guidance for future mission design, research activities, and sustainable operational radar altimetry data exploitation. Recommendations provided are fundamental for optimizing further scientific and operational advances of oceanographic observations by altimetry, including requirements for spatial and temporal resolution of altimetric measurements, their accuracy and continuity. There are also new challenges and new openings mentioned in the paper that are particularly crucial for observations at higher latitudes, for coastal oceanography, for cryospheric studies and for hydrology.
The paper starts with a general introduction followed by a section on Earth System Science including Ocean Dynamics, Sea Level, the Coastal Ocean, Hydrology, the Cryosphere and Polar Oceans and the âGreenâ Ocean, extending the frontier from biogeochemistry to marine ecology. Applications are described in a subsequent section, which covers Operational Oceanography, Weather, Hurricane Wave and Wind Forecasting, Climate projection. Instrumentsâ development and satellite missionsâ evolutions are described in a fourth section. A fifth section covers the key observations that altimeters provide and their potential complements, from other Earth observation measurements to in situ data. Section 6 identifies the data and methods and provides some accuracy and resolution requirements for the wet tropospheric correction, the orbit and other geodetic requirements, the Mean Sea Surface, Geoid and Mean Dynamic Topography, Calibration and Validation, data accuracy, data access and handling (including the DUACS system). Section 7 brings a transversal view on scales, integration, artificial intelligence, and capacity building (education and training). Section 8 reviews the programmatic issues followed by a conclusion
Contrastive Learning for Online Semi-Supervised General Continual Learning
International audienceWe study Online Continual Learning with missing labels and propose SemiCon, a new contrastive loss designed for partly labeled data. We demonstrate its efficiency by devising a memory-based method trained on an unlabeled data stream, where every data added to memory is labeled using an oracle. Our approach outperforms existing semi-supervised methods when few labels are available, and obtain similar results to state-of-the-art supervised methods while using only 2.6% of labels on Split-CIFAR10 and 10% of labels on Split-CIFAR100