Channel Exchanging Networks for Multimodal and Multitask Dense Image Prediction

Multimodal fusion and multitask learning are two vital topics in machine learning. Despite the fruitful progress, existing methods for both problems are still brittle to the same challenge -- it remains dilemmatic to integrate the common information across modalities (resp. tasks) meanwhile preserving the specific patterns of each modality (resp. task). Besides, while they are actually closely related to each other, multimodal fusion and multitask learning are rarely explored within the same methodological framework before. In this paper, we propose Channel-Exchanging-Network (CEN) which is self-adaptive, parameter-free, and more importantly, applicable for both multimodal fusion and multitask learning. At its core, CEN dynamically exchanges channels between subnetworks of different modalities. Specifically, the channel exchanging process is self-guided by individual channel importance that is measured by the magnitude of Batch-Normalization (BN) scaling factor during training. For the application of dense image prediction, the validity of CEN is tested by four different scenarios: multimodal fusion, cycle multimodal fusion, multitask learning, and multimodal multitask learning. Extensive experiments on semantic segmentation via RGB-D data and image translation through multi-domain input verify the effectiveness of our CEN compared to current state-of-the-art methods. Detailed ablation studies have also been carried out, which provably affirm the advantage of each component we propose.Comment: 18 pages. arXiv admin note: substantial text overlap with arXiv:2011.0500

Supervised learning in time-dependent environments with performance guarantees

151 p.En esta tesis, establecemos metodologías para el aprendizaje supervisado a partir de una secuencia de tareas dependientes del tiempo que explotan eficazmente la información de todas las tareas, proporcionan una adaptación multidimensional a los cambios de tareas y ofrecen garantías de rendimiento ajustadas y computables. Desarrollamos métodos para entornos de aprendizaje supervisado en los que las tareas llegan a lo largo del tiempo, incluidas técnicas de clasificación supervisada bajo concept drift y técnicas de continual learning. Además, presentamos técnicas de previsión de la demanda de energía que pueden adaptarse a los cambios temporales en los patrones de consumo y evaluar las incertidumbres intrínsecas de la demanda de carga. Los resultados numéricos muestran que las metodologías propuestas pueden mejorar significativamente el rendimiento de los métodos existentes utilizando múltiples conjuntos de datos de referencia. Esta tesis hace contribuciones teóricas que conducen a algoritmos eficientes para múltiples escenarios de aprendizaje automático que proporcionan garantías de rendimiento computables y un rendimiento superior al de las técnicas más avanzadas

Supervised Learning in Time-dependent Environments with Performance Guarantees

In practical scenarios, it is common to learn from a sequence of related problems (tasks). Such tasks are usually time-dependent in the sense that consecutive tasks are often significantly more similar. Time-dependency is common in multiple applications such as load forecasting, spam main filtering, and face emotion recognition. For instance, in the problem of load forecasting, the consumption patterns in consecutive time periods are significantly more similar since human habits and weather factors change gradually over time. Learning from a sequence tasks holds promise to enable accurate performance even with few samples per task by leveraging information from different tasks. However, harnessing the benefits of learning from a sequence of tasks is challenging since tasks are characterized by different underlying distributions. Most existing techniques are designed for situations where the tasks’ similarities do not depend on their order in the sequence. Existing techniques designed for timedependent tasks adapt to changes between consecutive tasks accounting for a scalar rate of change by using a carefully chosen parameter such as a learning rate or a weight factor. However, the tasks’ changes are commonly multidimensional, i.e., the timedependency often varies across different statistical characteristics describing the tasks. For instance, in the problem of load forecasting, the statistical characteristics related to weather factors often change differently from those related to generation. In this dissertation, we establish methodologies for supervised learning from a sequence of time-dependent tasks that effectively exploit information from all tasks, provide multidimensional adaptation to tasks’ changes, and provide computable tight performance guarantees. We develop methods for supervised learning settings where tasks arrive over time including techniques for supervised classification under concept drift (SCD) and techniques for continual learning (CL). In addition, we present techniques for load forecasting that can adapt to time changes in consumption patterns and assess intrinsic uncertainties in load demand. The numerical results show that the proposed methodologies can significantly improve the performance of existing methods using multiple benchmark datasets. This dissertation makes theoretical contributions leading to efficient algorithms for multiple machine learning scenarios that provide computable performance guarantees and superior performance than state-of-the-art techniques