Biologic activity in a fragment of recombinant human interferon α

To attempt to locate functionally important regions of the interferon (IFN) molecule, recombinant human IFN-α2 was subjected to proteolytic digestion. The bacterial proteinase thermolysin produced two major complementary fragments, HuIFN-α2-(1-110) and HuIFN-α2-(111-153). After reduction with 2-mercaptoethanol and separation of the two major fragments on NaDodSO4/polyacrylamide gel electrophoresis, antiviral activity persisted in the larger, Mr 12,000, fragment consisting of the amino-terminal 110 amino acids

Characterizing how 'distributional' NLP corpora distance metrics are

A corpus of vector-embedded text documents has some empirical distribution. Given two corpora, we want to calculate a single metric of distance (e.g., Mauve, Frechet Inception) between them. We describe an abstract quality, called `distributionality', of such metrics. A non-distributional metric tends to use very local measurements, or uses global measurements in a way that does not fully reflect the distributions' true distance. For example, if individual pairwise nearest-neighbor distances are low, it may judge the two corpora to have low distance, even if their two distributions are in fact far from each other. A more distributional metric will, in contrast, better capture the distributions' overall distance. We quantify this quality by constructing a Known-Similarity Corpora set from two paraphrase corpora and calculating the distance between paired corpora from it. The distances' trend shape as set element separation increases should quantify the distributionality of the metric. We propose that Average Hausdorff Distance and energy distance between corpora are representative examples of non-distributional and distributional distance metrics, to which other metrics can be compared, to evaluate how distributional they are.Comment: Published in the August 2023 Joint Statistical Meetings proceeding

Reliable and Interpretable Drift Detection in Streams of Short Texts

Data drift is the change in model input data that is one of the key factors leading to machine learning models performance degradation over time. Monitoring drift helps detecting these issues and preventing their harmful consequences. Meaningful drift interpretation is a fundamental step towards effective re-training of the model. In this study we propose an end-to-end framework for reliable model-agnostic change-point detection and interpretation in large task-oriented dialog systems, proven effective in multiple customer deployments. We evaluate our approach and demonstrate its benefits with a novel variant of intent classification training dataset, simulating customer requests to a dialog system. We make the data publicly available.Comment: ACL2023 industry track (9 pages

Detection of data drift and outliers affecting machine learning model performance over time

A trained ML model is deployed on another `test' dataset where target feature values (labels) are unknown. Drift is distribution change between the training and deployment data, which is concerning if model performance changes. For a cat/dog image classifier, for instance, drift during deployment could be rabbit images (new class) or cat/dog images with changed characteristics (change in distribution). We wish to detect these changes but can't measure accuracy without deployment data labels. We instead detect drift indirectly by nonparametrically testing the distribution of model prediction confidence for changes. This generalizes our method and sidesteps domain-specific feature representation. We address important statistical issues, particularly Type-1 error control in sequential testing, using Change Point Models (CPMs; see Adams and Ross 2012). We also use nonparametric outlier methods to show the user suspicious observations for model diagnosis, since the before/after change confidence distributions overlap significantly. In experiments to demonstrate robustness, we train on a subset of MNIST digit classes, then insert drift (e.g., unseen digit class) in deployment data in various settings (gradual/sudden changes in the drift proportion). A novel loss function is introduced to compare the performance (detection delay, Type-1 and 2 errors) of a drift detector under different levels of drift class contamination.Comment: In: JSM Proceedings, Nonparametric Statistics Section, 20202. Philadelphia, PA: American Statistical Association. 144--16

Classifier Data Quality: A Geometric Complexity Based Method for Automated Baseline And Insights Generation

Testing Machine Learning (ML) models and AI-Infused Applications (AIIAs), or systems that contain ML models, is highly challenging. In addition to the challenges of testing classical software, it is acceptable and expected that statistical ML models sometimes output incorrect results. A major challenge is to determine when the level of incorrectness, e.g., model accuracy or F1 score for classifiers, is acceptable and when it is not. In addition to business requirements that should provide a threshold, it is a best practice to require any proposed ML solution to out-perform simple baseline models, such as a decision tree. We have developed complexity measures, which quantify how difficult given observations are to assign to their true class label; these measures can then be used to automatically determine a baseline performance threshold. These measures are superior to the best practice baseline in that, for a linear computation cost, they also quantify each observation' classification complexity in an explainable form, regardless of the classifier model used. Our experiments with both numeric synthetic data and real natural language chatbot data demonstrate that the complexity measures effectively highlight data regions and observations that are likely to be misclassified.Comment: Accepted to EDSMLS workshop at AAAI conferenc

Predicting Question-Answering Performance of Large Language Models through Semantic Consistency

Semantic consistency of a language model is broadly defined as the model's ability to produce semantically-equivalent outputs, given semantically-equivalent inputs. We address the task of assessing question-answering (QA) semantic consistency of contemporary large language models (LLMs) by manually creating a benchmark dataset with high-quality paraphrases for factual questions, and release the dataset to the community. We further combine the semantic consistency metric with additional measurements suggested in prior work as correlating with LLM QA accuracy, for building and evaluating a framework for factual QA reference-less performance prediction -- predicting the likelihood of a language model to accurately answer a question. Evaluating the framework on five contemporary LLMs, we demonstrate encouraging, significantly outperforming baselines, results.Comment: EMNLP2023 GEM workshop, 17 page

Data Drift Monitoring for Log Anomaly Detection Pipelines

Logs enable the monitoring of infrastructure status and the performance of associated applications. Logs are also invaluable for diagnosing the root causes of any problems that may arise. Log Anomaly Detection (LAD) pipelines automate the detection of anomalies in logs, providing assistance to site reliability engineers (SREs) in system diagnosis. Log patterns change over time, necessitating updates to the LAD model defining the `normal' log activity profile. In this paper, we introduce a Bayes Factor-based drift detection method that identifies when intervention, retraining, and updating of the LAD model are required with human involvement. We illustrate our method using sequences of log activity, both from unaltered data, and simulated activity with controlled levels of anomaly contamination, based on real collected log data

Distribution and Emergency

Streaming video requires RealPlayer to view.The University Archives has determined that this item is of continuing value to OSU's history.Humanitarian organizations divide their work into two categories: development aid that improves underlying conditions, and emergency aid, given in response to a natural or manmade disaster. However, Jennifer Rubenstein, a fellow at Princeton University, questioned this distinction. While it might have logistical advantages, she argued, it does not suit the variety of situations and populations requiring aid.Ohio State University. Mershon Center for International Security StudiesEvent webpage, streaming video, photos, power point presentation, and lecture summar

An L Band Spectrum of the Coldest Brown Dwarf

The coldest brown dwarf, WISE 0855, is the closest known planetary-mass, free-floating object and has a temperature nearly as cold as the solar system gas giants. Like Jupiter, it is predicted to have an atmosphere rich in methane, water, and ammonia, with clouds of volatile ices. WISE 0855 is faint at near-infrared wavelengths and emits almost all its energy in the mid-infrared. Skemer et al. 2016 presented a spectrum of WISE 0855 from 4.5-5.1 micron (M band), revealing water vapor features. Here, we present a spectrum of WISE 0855 in L band, from 3.4-4.14 micron. We present a set of atmosphere models that include a range of compositions (metallicities and C/O ratios) and water ice clouds. Methane absorption is clearly present in the spectrum. The mid-infrared color can be better matched with a methane abundance that is depleted relative to solar abundance. We find that there is evidence for water ice clouds in the M band spectrum, and we find a lack of phosphine spectral features in both the L and M band spectra. We suggest that a deep continuum opacity source may be obscuring the near-infrared flux, possibly a deep phosphorous-bearing cloud, ammonium dihyrogen phosphate. Observations of WISE 0855 provide critical constraints for cold planetary atmospheres, bridging the temperature range between the long-studied solar system planets and accessible exoplanets. JWST will soon revolutionize our understanding of cold brown dwarfs with high-precision spectroscopy across the infrared, allowing us to study their compositions and cloud properties, and to infer their atmospheric dynamics and formation processes.Comment: 19 pages, 21 figures. Accepted for publication in Ap