Should You Give The Farm or Ranch Away?

4 p

In Estate Planning--A Look At Some Important Items.

2 p

Plan Your Estate Now.

4 p

In Estate Planning--A Look At Some Important Items.

2 p

ISM In-Space Manufacturing

Develop and enable the technologies, materials, and processes required to provide affordable, sustainable on-demand manufacturing, recycling, and repair during Exploration Missions

Robust Multimodal Learning with Missing Modalities via Parameter-Efficient Adaptation

Multimodal learning seeks to utilize data from multiple sources to improve the overall performance of downstream tasks. It is desirable for redundancies in the data to make multimodal systems robust to missing or corrupted observations in some correlated modalities. However, we observe that the performance of several existing multimodal networks significantly deteriorates if one or multiple modalities are absent at test time. To enable robustness to missing modalities, we propose simple and parameter-efficient adaptation procedures for pretrained multimodal networks. In particular, we exploit low-rank adaptation and modulation of intermediate features to compensate for the missing modalities. We demonstrate that such adaptation can partially bridge performance drop due to missing modalities and outperform independent, dedicated networks trained for the available modality combinations in some cases. The proposed adaptation requires extremely small number of parameters (e.g., fewer than 0.7% of the total parameters in most experiments). We conduct a series of experiments to highlight the robustness of our proposed method using diverse datasets for RGB-thermal and RGB-Depth semantic segmentation, multimodal material segmentation, and multimodal sentiment analysis tasks. Our proposed method demonstrates versatility across various tasks and datasets, and outperforms existing methods for robust multimodal learning with missing modalities.Comment: 18 pages, 3 figures, 11 table

Brief Note: Fishes of the Upper Portage River, Ohio, 1973-1975

Author Institution: Aqua Tech Environmental Consultants, Inc. ; Bowling Green State University, Department of Biological Sciences ; Aqua Tech Environmental Consultants, Inc