Concurrent Geometric Multicasting

We present MCFR, a multicasting concurrent face routing algorithm that uses geometric routing to deliver a message from source to multiple targets. We describe the algorithm's operation, prove it correct, estimate its performance bounds and evaluate its performance using simulation. Our estimate shows that MCFR is the first geometric multicast routing algorithm whose message delivery latency is independent of network size and only proportional to the distance between the source and the targets. Our simulation indicates that MCFR has significantly better reliability than existing algorithms

Balanced Softmax Cross-Entropy for Incremental Learning

Deep neural networks are prone to catastrophic forgetting when incrementally trained on new classes or new tasks as adaptation to the new data leads to a drastic decrease of the performance on the old classes and tasks. By using a small memory for rehearsal and knowledge distillation, recent methods have proven to be effective to mitigate catastrophic forgetting. However due to the limited size of the memory, large imbalance between the amount of data available for the old and new classes still remains which results in a deterioration of the overall accuracy of the model. To address this problem, we propose the use of the Balanced Softmax Cross-Entropy loss and show that it can be combined with exiting methods for incremental learning to improve their performances while also decreasing the computational cost of the training procedure in some cases. Complete experiments on the competitive ImageNet, subImageNet and CIFAR100 datasets show states-of-the-art results

Class-Incremental Learning using Diffusion Model for Distillation and Replay

Class-incremental learning aims to learn new classes in an incremental fashion without forgetting the previously learned ones. Several research works have shown how additional data can be used by incremental models to help mitigate catastrophic forgetting. In this work, following the recent breakthrough in text-to-image generative models and their wide distribution, we propose the use of a pretrained Stable Diffusion model as a source of additional data for class-incremental learning. Compared to competitive methods that rely on external, often unlabeled, datasets of real images, our approach can generate synthetic samples belonging to the same classes as the previously encountered images. This allows us to use those additional data samples not only in the distillation loss but also for replay in the classification loss. Experiments on the competitive benchmarks CIFAR100, ImageNet-Subset, and ImageNet demonstrate how this new approach can be used to further improve the performance of state-of-the-art methods for class-incremental learning on large scale datasets.Comment: Best paper award at 1st Workshop on Visual Continual Learning, ICCV 202

Improved Orbital Propagator Integrated with SGP4 and Machine Learning

The current industry standard orbital propagator, the Simplified General Perturbation Model-4 (SPG4), relies completely on physics-based orbital mechanics, can only provide accurate orbital predictions ~12 hours in advance. We developed a novel hybrid model, combining the SGP4 baseline with two machine learning estimators, autoencoder and random forest, in order to reduce the errors of the SGP4 propagator. The sources of errors in SGP4 propagators come from incomplete perturbation calculations and low-order of series expansions. The time-series nature of these error patterns are modeled by our machine learning estimators and then are used to make corrections to the SGP4 propagation, which result in more accurate orbit predictions. We tested our hybrid model on 3 satellite objects with the corresponding TLE (Two Line Element) data. The improvement on orbit prediction achieved 20-30% over the future 30 days period. The limitation of this hybrid approach is the requirement of at least 3 years of historical TLE data for the machine learning models, but could be overcome by creating synthetic orbital data from a similar space object. This hybrid model can be easily packaged into a software tool for space mission operation planning and facilitate mission autonomy

ExpeL: LLM Agents Are Experiential Learners

The recent surge in research interest in applying large language models (LLMs) to decision-making tasks has flourished by leveraging the extensive world knowledge embedded in LLMs. While there is a growing demand to tailor LLMs for custom decision-making tasks, finetuning them for specific tasks is resource-intensive and may diminish the model's generalization capabilities. Moreover, state-of-the-art language models like GPT-4 and Claude are primarily accessible through API calls, with their parametric weights remaining proprietary and unavailable to the public. This scenario emphasizes the growing need for new methodologies that allow learning from agent experiences without requiring parametric updates. To address these problems, we introduce the Experiential Learning (ExpeL) agent. Our agent autonomously gathers experiences and extracts knowledge using natural language from a collection of training tasks. At inference, the agent recalls its extracted insights and past experiences to make informed decisions. Our empirical results highlight the robust learning efficacy of the ExpeL agent, indicating a consistent enhancement in its performance as it accumulates experiences. We further explore the emerging capabilities and transfer learning potential of the ExpeL agent through qualitative observations and additional experiments

Cram\'er-Rao bound-informed training of neural networks for quantitative MRI

Neural networks are increasingly used to estimate parameters in quantitative MRI, in particular in magnetic resonance fingerprinting. Their advantages over the gold standard non-linear least square fitting are their superior speed and their immunity to the non-convexity of many fitting problems. We find, however, that in heterogeneous parameter spaces, i.e. in spaces in which the variance of the estimated parameters varies considerably, good performance is hard to achieve and requires arduous tweaking of the loss function, hyper parameters, and the distribution of the training data in parameter space. Here, we address these issues with a theoretically well-founded loss function: the Cram\'er-Rao bound (CRB) provides a theoretical lower bound for the variance of an unbiased estimator and we propose to normalize the squared error with respective CRB. With this normalization, we balance the contributions of hard-to-estimate and not-so-hard-to-estimate parameters and areas in parameter space, and avoid a dominance of the former in the overall training loss. Further, the CRB-based loss function equals one for a maximally-efficient unbiased estimator, which we consider the ideal estimator. Hence, the proposed CRB-based loss function provides an absolute evaluation metric. We compare a network trained with the CRB-based loss with a network trained with the commonly used means squared error loss and demonstrate the advantages of the former in numerical, phantom, and in vivo experiments.Comment: Xiaoxia Zhang, Quentin Duchemin, and Kangning Liu contributed equally to this wor