5 research outputs found
Unpacking Large Language Models with Conceptual Consistency
If a Large Language Model (LLM) answers "yes" to the question "Are mountains
tall?" then does it know what a mountain is? Can you rely on it responding
correctly or incorrectly to other questions about mountains? The success of
Large Language Models (LLMs) indicates they are increasingly able to answer
queries like these accurately, but that ability does not necessarily imply a
general understanding of concepts relevant to the anchor query. We propose
conceptual consistency to measure a LLM's understanding of relevant concepts.
This novel metric measures how well a model can be characterized by finding out
how consistent its responses to queries about conceptually relevant background
knowledge are. To compute it we extract background knowledge by traversing
paths between concepts in a knowledge base and then try to predict the model's
response to the anchor query from the background knowledge. We investigate the
performance of current LLMs in a commonsense reasoning setting using the CSQA
dataset and the ConceptNet knowledge base. While conceptual consistency, like
other metrics, does increase with the scale of the LLM used, we find that
popular models do not necessarily have high conceptual consistency. Our
analysis also shows significant variation in conceptual consistency across
different kinds of relations, concepts, and prompts. This serves as a step
toward building models that humans can apply a theory of mind to, and thus
interact with intuitively
Confidence Calibration for Systems with Cascaded Predictive Modules
Existing conformal prediction algorithms estimate prediction intervals at
target confidence levels to characterize the performance of a regression model
on new test samples. However, considering an autonomous system consisting of
multiple modules, prediction intervals constructed for individual modules fall
short of accommodating uncertainty propagation over different modules and thus
cannot provide reliable predictions on system behavior. We address this
limitation and present novel solutions based on conformal prediction to provide
prediction intervals calibrated for a predictive system consisting of cascaded
modules (e.g., an upstream feature extraction module and a downstream
regression module). Our key idea is to leverage module-level validation data to
characterize the system-level error distribution without direct access to
end-to-end validation data. We provide theoretical justification and empirical
experimental results to demonstrate the effectiveness of proposed solutions. In
comparison to prediction intervals calibrated for individual modules, our
solutions generate improved intervals with more accurate performance guarantees
for system predictions, which are demonstrated on both synthetic systems and
real-world systems performing overlap prediction for indoor navigation using
the Matterport3D dataset