Bag of Tricks for In-Distribution Calibration of Pretrained Transformers

While pre-trained language models (PLMs) have become a de-facto standard promoting the accuracy of text classification tasks, recent studies find that PLMs often predict over-confidently. Although various calibration methods have been proposed, such as ensemble learning and data augmentation, most of the methods have been verified in computer vision benchmarks rather than in PLM-based text classification tasks. In this paper, we present an empirical study on confidence calibration for PLMs, addressing three categories, including confidence penalty losses, data augmentations, and ensemble methods. We find that the ensemble model overfitted to the training set shows sub-par calibration performance and also observe that PLMs trained with confidence penalty loss have a trade-off between calibration and accuracy. Building on these observations, we propose the Calibrated PLM (CALL), a combination of calibration techniques. The CALL complements the drawbacks that may occur when utilizing a calibration method individually and boosts both classification and calibration accuracy. Design choices in CALL's training procedures are extensively studied, and we provide a detailed analysis of how calibration techniques affect the calibration performance of PLMs

Safety and Security Management with Unmanned Aerial Vehicle (UAV) in Oil and Gas Industry

AbstractWe describe a mathematical model for UAV aided security operations in the oil and gas industry. Operating UAVs can provide seamless awareness on possible emergency situations such as oil spills, shipping incidents, industrial accidents, acts of terrorism, and so on. The primary goal of this model is to generate an optimal UAV operational schedule to meet surveillance needs in the areas of interest in each time period. The performance of these UAVs depends on the risk assessment on spatio-and-temporal characteristics of threats, specifications of available UAVs, and decision makers’ critical information requirements. The models are designed to provide insights into issues associated with designing and operating UAVs for strengthened maritime and port security

Learning to Write with Coherence From Negative Examples

Coherence is one of the critical factors that determine the quality of writing. We propose writing relevance (WR) training method for neural encoder-decoder natural language generation (NLG) models which improves coherence of the continuation by leveraging negative examples. WR loss regresses the vector representation of the context and generated sentence toward positive continuation by contrasting it with the negatives. We compare our approach with Unlikelihood (UL) training in a text continuation task on commonsense natural language inference (NLI) corpora to show which method better models the coherence by avoiding unlikely continuations. The preference of our approach in human evaluation shows the efficacy of our method in improving coherence.Comment: 4+1 pages, 4 figures, 2 tables. ICASSP 2022 rejecte

Liquefied Natural Gas Ship Route Planning Model Considering Market Trend Change

We consider a new biannual liquefi ed natural gas (LNG) ship routing and scheduling problem and a stochastic extension under boil-off gas (BOG) uncertainty while serving geographically dispersed multiple customers using a fl eet of heterogeneous vessels. We are motivated not only by contract trend changes to shorter ones but also by technological advances in LNG vessel design. The mutual coincidence of both transitions enables developing a new LNG shipping strategy to keep up with emerging market trend. We fi rst propose a deterministic LNG scheduling model formulated as a multiple vehicle routing problem (VRP). The model is then extended to consider BOG using a two-stage stochastic modeling approach in which BOG is a random variable. Since the VRP is typically a combinatorial optimization problem, its stochastic extension is much harder to solve. In order to overcome this computational burden, a Monte Carlo sampling optimization is used to reduce the number of scenarios in the stochastic model while ensuring good quality of solutions. The solutions are evaluated using expected value of perfect information (EVPI) and value of stochastic solution (VSS). The result shows that our proposed model yields more stable solutions than the deterministic model. The study was made possible by the NPRP award [NPRP 4-1249-2-492] from the Qatar National Research Fund (a member of the Qatar Foundation)

Quantitative Understanding of Probabilistic Behavior of Living Cells Operated by Vibrant Intracellular Networks

For quantitative understanding of probabilistic behaviors of living cells, it is essential to construct a correct mathematical description of intracellular networks interacting with complex cell environments, which has been a formidable task. Here, we present a novel model and stochastic kinetics for an intracellular network interacting with hidden cell environments, employing a complete description of cell state dynamics and its coupling to the system network. Our analysis reveals that various environmental effects on the product number fluctuation of intracellular reaction networks can be collectively characterized by Laplace transform of the time-correlation function of the product creation rate fluctuation with the Laplace variable being the product decay rate. On the basis of the latter result, we propose an efficient method for quantitative analysis of the chemical fluctuation produced by intracellular networks coupled to hidden cell environments. By applying the present approach to the gene expression network, we obtain simple analytic results for the gene expression variability and the environment-induced correlations between the expression levels of mutually noninteracting genes. The theoretical results compose a unified framework for quantitative understanding of various gene expression statistics observed across a number of different systems with a small number of adjustable parameters with clear physical meanings.National Research Foundation of Korea (Grant 2011-0016412)National Research Foundation of Korea (Priority Research Center Program 2009-0093817

Slow oxidation of magnetite nanoparticles elucidates the limits of the Verwey transition

Magnetite (Fe3O4) is of fundamental importance as the original magnetic material and also for the Verwey transition near T_V = 125 K, below which a complex lattice distortion and electron orders occur. The Verwey transition is suppressed by strain or chemical doping effects giving rise to well-documented first and second-order regimes, but the origin of the order change is unclear. Here, we show that slow oxidation of monodisperse Fe3O4 nanoparticles leads to an intriguing variation of the Verwey transition that elucidates the doping effects. Exposure to various fixed oxygen pressures at ambient temperature leads to an initial drop to TV minima as low as 70 K after 45-75 days, followed by recovery to a constant value of 95 K after 160 days that persists in all experiments for aging times up to 1070 days. A physical model based on both doping and doping-gradient effects accounts quantitatively for this evolution and demonstrates that the persistent 95 K value corresponds to the lower limit for homogenously doped magnetite and hence for the first order regime. In comparison, further suppression down to 70 K results from inhomogeneous strains that characterize the second-order region. This work demonstrates that slow reactions of nanoparticles can give exquisite control and separation of homogenous and inhomogeneous doping or strain effects on an nm scale and offers opportunities for similar insights into complex electronic and magnetic phase transitions in other materials.Comment: 24 pages, 13 figures, 2 tables, the manuscript is accepted for publishing at Nature Communication

Giant thermal hysteresis in Verwey transition of single domain Fe3O4 nanoparticles

Most interesting phenomena of condensed matter physics originate from interactions among different degrees of freedom, making it a very intriguing yet challenging question how certain ground states emerge from only a limited number of atoms in assembly. This is especially the case for strongly correlated electron systems with overwhelming complexity. The Verwey transition of Fe3O4 is a classic example of this category, of which the origin is still elusive 80 years after the first report. Here we report, for the first time, that the Verwey transition of Fe3O4 nanoparticles exhibits size-dependent thermal hysteresis in magnetization, 57Fe NMR, and XRD measurements. The hysteresis width passes a maximum of 11 K when the size is 120 nm while dropping to only 1 K for the bulk sample. This behavior is very similar to that of magnetic coercivity and the critical sizes of the hysteresis and the magnetic single domain are identical. We interpret it as a manifestation of charge ordering and spin ordering correlation in a single domain. This work paves a new way of undertaking researches in the vibrant field of strongly correlated electron physics combined with nanoscience.Comment: 13 pages, 4 figure

Interactive OAISYS: A photorealistic terrain simulation for robotics research

Photorealistic simulation pipelines are crucial for the development of novel robotic methods and modern machine vision approaches. Simulations have been particularly popular for generating labeled synthetic data sets, which otherwise would require vast efforts of manual annotation when using real data. However, these simulators are usually not interactive, and the data generation process cannot be interrupted. Therefore, these simulators are not suitable for evaluating active methods, such as active learning or perception aware path planning, which make decisions based on the observed perception data. In order to address this problem, we propose a modified version of the simulator OAISYS, a photorealistic scene simulator for unstructured outdoor environments. We extended the simulator in order to use it in an interactive way, and implemented a developer-friendly RPC interface so that it is easy for any environment to integrate into the simulator. In this paper, we demonstrate the functionality of the extension on 3D scene reconstruction to show its future research potential and provide an example of the implementation using the middleware ROS. The code is publicly available under https://github.com/DLR-RM/oaisy

Prediction of Obstructive Sleep Apnea Based on Respiratory Sounds Recorded Between Sleep Onset and Sleep Offset

Objectives To develop a simple algorithm for prescreening of obstructive sleep apnea (OSA) on the basis of respiratorysounds recorded during polysomnography during all sleep stages between sleep onset and offset. Methods Patients who underwent attended, in-laboratory, full-night polysomnography were included. For all patients, audiorecordings were performed with an air-conduction microphone during polysomnography. Analyses included allsleep stages (i.e., N1, N2, N3, rapid eye movement, and waking). After noise reduction preprocessing, data were segmentedinto 5-s windows and sound features were extracted. Prediction models were established and validated with10-fold cross-validation by using simple logistic regression. Binary classifications were separately conducted for threedifferent threshold criteria at apnea hypopnea index (AHI) of 5, 15, or 30. Prediction model characteristics, includingaccuracy, sensitivity, specificity, positive predictive value (precision), negative predictive value, and area under thecurve (AUC) of the receiver operating characteristic were computed. Results A total of 116 subjects were included; their mean age, body mass index, and AHI were 50.4 years, 25.5 kg/m2, and23.0/hr, respectively. A total of 508 sound features were extracted from respiratory sounds recorded throughoutsleep. Accuracies of binary classifiers at AHIs of 5, 15, and 30 were 82.7%, 84.4%, and 85.3%, respectively. Predictionperformances for the classifiers at AHIs of 5, 15, and 30 were AUC, 0.83, 0.901, and 0.91; sensitivity, 87.5%,81.6%, and 60%; and specificity, 67.8%, 87.5%, and 94.1%. Respective precision values of the classifiers were89.5%, 87.5%, and 78.2% for AHIs of 5, 15, and 30. Conclusion This study showed that our binary classifier predicted patients with AHI of ≥15 with sensitivity and specificityof >80% by using respiratory sounds during sleep. Since our prediction model included all sleep stage data, algorithmsbased on respiratory sounds may have a high value for prescreening OSA with mobile devices