27 research outputs found
RL-LIM: Reinforcement Learning-based Locally Interpretable Modeling
Understanding black-box machine learning models is important towards their
widespread adoption. However, developing globally interpretable models that
explain the behavior of the entire model is challenging. An alternative
approach is to explain black-box models through explaining individual
prediction using a locally interpretable model. In this paper, we propose a
novel method for locally interpretable modeling - Reinforcement Learning-based
Locally Interpretable Modeling (RL-LIM). RL-LIM employs reinforcement learning
to select a small number of samples and distill the black-box model prediction
into a low-capacity locally interpretable model. Training is guided with a
reward that is obtained directly by measuring agreement of the predictions from
the locally interpretable model with the black-box model. RL-LIM near-matches
the overall prediction performance of black-box models while yielding
human-like interpretability, and significantly outperforms state of the art
locally interpretable models in terms of overall prediction performance and
fidelity.Comment: 18 pages, 7 figures, 7 table
LANISTR: Multimodal Learning from Structured and Unstructured Data
Multimodal large-scale pretraining has shown impressive performance for
unstructured data including language, image, audio, and video. However, a
prevalent real-world scenario involves the combination of structured data types
(tabular, time-series) with unstructured data which has so far been
understudied. To bridge this gap, we propose LANISTR, an attention-based
framework to learn from LANguage, Image, and STRuctured data. The core of
LANISTR's methodology is rooted in \textit{masking-based} training applied
across both unimodal and multimodal levels. In particular, we introduce a new
similarity-based multimodal masking loss that enables it to learn cross-modal
relations from large-scale multimodal data with missing modalities. On two
real-world datastes, MIMIC-IV (healthcare) and Amazon Product Review (retail),
LANISTR demonstrates remarkable absolute improvements of 6.6\% (AUROC) and up
to 14\% (accuracy) when fine-tuned on 0.1\% and 0.01\% of labeled data,
respectively, compared to the state-of-the-art alternatives. Notably, these
improvements are observed even in the presence of considerable missingness
ratios of 35.7\% and 99.8\%, in the respective datasets
Search-Adaptor: Text Embedding Customization for Information Retrieval
Text embeddings extracted by pre-trained Large Language Models (LLMs) have
significant potential to improve information retrieval and search. Beyond the
zero-shot setup in which they are being conventionally used, being able to take
advantage of the information from the relevant query-corpus paired data has the
power to further boost the LLM capabilities. In this paper, we propose a novel
method, Search-Adaptor, for customizing LLMs for information retrieval in an
efficient and robust way. Search-Adaptor modifies the original text embedding
generated by pre-trained LLMs, and can be integrated with any LLM, including
those only available via APIs. On multiple real-world English and multilingual
retrieval datasets, we show consistent and significant performance benefits for
Search-Adaptor -- e.g., more than 5.2% improvements over the Google Embedding
APIs in nDCG@10 averaged over 13 BEIR datasets.Comment: 9 pages, 2 figure