Reliable and Interpretable Drift Detection in Streams of Short Texts

Data drift is the change in model input data that is one of the key factors leading to machine learning models performance degradation over time. Monitoring drift helps detecting these issues and preventing their harmful consequences. Meaningful drift interpretation is a fundamental step towards effective re-training of the model. In this study we propose an end-to-end framework for reliable model-agnostic change-point detection and interpretation in large task-oriented dialog systems, proven effective in multiple customer deployments. We evaluate our approach and demonstrate its benefits with a novel variant of intent classification training dataset, simulating customer requests to a dialog system. We make the data publicly available.Comment: ACL2023 industry track (9 pages

Pengembangan Softskill Dan Hasil Belajar Kimia Dasar Dengan Menggunakan Model Pembelajaran 5-e (Engagement, Exploration, Explanation, Elaboration, Evaluation)

Tujuan dari penelitian ini untuk mengetahui peningkatan hasil belajar kimia dan Soft skill mahasiswa dengan menggunakan strategi pembelajaran 5-E((Engagement, Exploration, Explanation, Elaboration, Evaluation). Penelitian ini merupakan penelitian pengembangan yang terdiri dari tiga tahap. Tiap tahap terdiri dari perencanaan, pelaksanaan, pengamatan, dan refleksi. Subyek penelitian ini adalah mahasiswa Jurusan Pendidikan Kimia UNNES. Fokus yang diteliti dalam penelitian ini adalah hasil belajar dan pengembangan soft skill mahasiswa. Data yang diperoleh kemudian dianalisis dengan menggunakan metode deskriptif untuk mengetahui peningkatan hasil belajar dan pengembangan soft skill mahasiswa. Hasil belajar mahasiswa jurusan Kimia UNNES meningkat. Rerata hasil belajar tahap I adalah 64,62, tahap II adalah 75,85, dan pada tahap III adalah 79,77. Hasil rerata soft skill mahasiswa yang memenuhi kriteria tinggi dan sangat tinggi meningkat dari tahap 1 (77%), tahap 2 (80%) dan tahap 3 (92%)

Predicting Question-Answering Performance of Large Language Models through Semantic Consistency

Semantic consistency of a language model is broadly defined as the model's ability to produce semantically-equivalent outputs, given semantically-equivalent inputs. We address the task of assessing question-answering (QA) semantic consistency of contemporary large language models (LLMs) by manually creating a benchmark dataset with high-quality paraphrases for factual questions, and release the dataset to the community. We further combine the semantic consistency metric with additional measurements suggested in prior work as correlating with LLM QA accuracy, for building and evaluating a framework for factual QA reference-less performance prediction -- predicting the likelihood of a language model to accurately answer a question. Evaluating the framework on five contemporary LLMs, we demonstrate encouraging, significantly outperforming baselines, results.Comment: EMNLP2023 GEM workshop, 17 page

Evaluating the Efficacy of Thiel Cadavers Versus Low Fidelity Simulation Plastic Mannequins for Teaching Intubation Skills in Medical Education

Most medical students across the U.S. currently receive intubation skills training on low fidelity simulation (LFS) plastic mannequins. A limitation of these LFS models is that they do not allow students to attain complete familiarity and proficiency with intubation in a real-world clinical setting. For several years now, ETSU has been utilizing the Thiel embalming technique, which allows for greater joint flexibility and preservation of soft tissues compared to traditional embalming techniques. Our study set out to evaluate the efficacy of Thiel cadavers versus LFS plastic mannequins in assessing whether one model was more suitable in teaching intubation skills. After IRB approval, we exposed thirty-two first-and second-year medical students to both LFS models and Thiel embalmed cadavers in order to assess preparedness and confidence for performing intubations, and overall transference to patient care. Participants were shown an instructional video and given a brief lecture on the steps of a successful intubation by the primary investigators of the project before they were asked to practice intubation on both a Thiel cadaver as well as a LFS plastic mannequin. Participants were randomly assigned to note which teaching method to practice on first (Thiel cadaver or LFS model). Students were given written instructions during the procedure, and they completed a pre and post survey assessing preparedness, confidence, transference to patient care, and overall preference between the Thiel cadaver and LFS model for education. Our study significantly concluded that students felt better prepared and more confident by practicing on the Thiel cadaver and unanimously thought that it had better transference to clinical care. Overall, students also felt that the Thiel cadaver provided more knowledge, and close to two thirds of subjects preferred the Thiel cadaver over the LFS mannequin for learning intubation skills while a third of subjects said that both models should be used in training

Designing self-assembling kinetics with differentiable statistical physics models

The inverse problem of designing component interactions to target emergent structure is fundamental to numerous applications in biotechnology, materials science, and statistical physics. Equally important is the inverse problem of designing emergent kinetics, but this has received considerably less attention. Using recent advances in automatic differentiation, we show how kinetic pathways can be precisely designed by directly differentiating through statistical physics models, namely free energy calculations and molecular dynamics simulations. We consider two systems that are crucial to our understanding of structural self-assembly: bulk crystallization and small nanoclusters. In each case, we are able to assemble precise dynamical features. Using gradient information, we manipulate interactions among constituent particles to tune the rate at which these systems yield specific structures of interest. Moreover, we use this approach to learn nontrivial features about the high-dimensional design space, allowing us to accurately predict when multiple kinetic features can be simultaneously and independently controlled. These results provide a concrete and generalizable foundation for studying nonstructural self-assembly, including kinetic properties as well as other complex emergent properties, in a vast array of systems

Serological evaluation of Newcastle disease protection among broilers at a live bird market in Kano, Northwest Nigeria, and its epidemiological significance

Vaccination is a tool of Newcastle disease (ND) control among broilers. This study aimed at determining the immunity status of sampled broilers against ND at a live bird market in Kano, Northwest Nigeria, and its epidemiological implication. A cross-sectional study of antibodies against Newcastle disease virus (NDV) was carried out among broiler chicken in a live bird market in Kano State, Northwest Nigeria. A total of 471 samples was tested successfully. NDV antibody titer was assayed using hemagglutination-inhibition test (HI) and ND indirect enzyme-linked immunosorbent assay (ELISA). Serological levels of NDV antibodies were 67.9% (ELISA) and 78.1% (HI). Also, 67 (20.9%) samples tested positive for ELISA but negative for HI, whereas 115 (31.3%) samples tested negative for ELISA but positive for HI. There is strong association between the immune status obtained from both tests (P < .05), significant difference exists between the immune titer obtained from both tests (P < .05). Protective antibody titer among the test subjects suggests individual protection against virulent NDV (vNDV) strain; however, protective levels ≥85% that confers herd immunity were not attained. This report emphasizes the need for farmers to be more compliant to ND vaccination schedule and best practices in their poultry farm to enhance ND control in Live Bird Markets (LBMs)

Generalizing the Wythoff Array and other Fibonacci Facts to Tribonacci Numbers

In this paper, we generalize a lot of facts from John Conway and Alex Ryba's paper, \textit{The extra Fibonacci series and the Empire State Building}, where we replace the Fibonacci sequence with the Tribonacci sequence. We study the Tribonacci array, which we also call \textit{the Trithoff array} to emphasize the connection to the Wythoff array. We describe 13 new sequences.Comment: 28 pages, 5 table

Using virtual reality and thermal imagery to improve statistical modelling of vulnerable and protected species.

Biodiversity loss and sparse observational data mean that critical conservation decisions may be based on little to no information. Emerging technologies, such as airborne thermal imaging and virtual reality, may facilitate species monitoring and improve predictions of species distribution. Here we combined these two technologies to predict the distribution of koalas, specialized arboreal foliovores facing population declines in many parts of eastern Australia. For a study area in southeast Australia, we complemented ground-survey records with presence and absence observations from thermal-imagery obtained using Remotely-Piloted Aircraft Systems. These field observations were further complemented with information elicited from koala experts, who were immersed in 360-degree images of the study area. The experts were asked to state the probability of habitat suitability and koala presence at the sites they viewed and to assign each probability a confidence rating. We fit logistic regression models to the ground survey data and the ground plus thermal-imagery survey data and a Beta regression model to the expert elicitation data. We then combined parameter estimates from the expert-elicitation model with those from each of the survey models to predict koala presence and absence in the study area. The model that combined the ground, thermal-imagery and expert-elicitation data substantially reduced the uncertainty around parameter estimates and increased the accuracy of classifications (koala presence vs absence), relative to the model based on ground-survey data alone. Our findings suggest that data elicited from experts using virtual reality technology can be combined with data from other emerging technologies, such as airborne thermal-imagery, using traditional statistical models, to increase the information available for species distribution modelling and the conservation of vulnerable and protected species

Seabirds show foraging site and route fidelity but demonstrate flexibility in response to local information

•Background: Fidelity to a given foraging location or route may be beneficial when environmental conditions are predictable but costly if conditions deteriorate or become unpredictable. Understanding the magnitude of fidelity displayed by different species and the processes that drive or erode it is therefore vital for understanding how fidelity may shape the demographic consequences of anthropogenic change. In particular, understanding the information that individuals may use to adjust their fidelity will facilitate improved predictions of how fidelity may change as environments change and the extent to which it will buffer individuals against such changes. •Methods: We used movement data collected during the breeding season across eight years for common guillemots, Atlantic puffins, razorbills, and black-legged kittiwakes breeding on the Isle of May, Scotland to understand: (1) whether foraging site/route fidelity occurred within and between years, (2) whether the degree of fidelity between trips was predicted by personal foraging effort, and (3) whether different individuals made more similar trips when they overlapped in time at the colony prior to departure and/or when out at sea suggesting the use of the same local environmental cues or information on the decisions made by con- and heterospecifics. •Results: All species exhibited site and route fidelity both within- and between-years, and fidelity between trips in guillemots and razorbills was related to metrics of foraging effort, suggesting they adjust fidelity to their personal foraging experience. We also found evidence that individuals used local environmental cues of prey location or availability and/or information gained by observing conspecifics when choosing foraging routes, particularly in puffins, where trips of individuals that overlapped temporally at the colony or out at sea were more similar. •Conclusions: The fidelity shown by these seabird species has the potential to put them at greater risk in the face of environmental change by driving individuals to continue using areas being degraded by anthropogenic pressures. However, our results suggest that individuals show some flexibility in their fidelity, which may promote resilience under environmental change. The benefits of this flexibility are likely to depend on numerous factors, including the rapidity and spatial scale of environmental change and the reliability of the information individuals use to choose foraging sites or routes, thus highlighting the need to better understand how organisms combine cues, prior experience, and other sources of information to make movement decisions