Reinforced Mnemonic Reader for Machine Reading Comprehension

In this paper, we introduce the Reinforced Mnemonic Reader for machine reading comprehension tasks, which enhances previous attentive readers in two aspects. First, a reattention mechanism is proposed to refine current attentions by directly accessing to past attentions that are temporally memorized in a multi-round alignment architecture, so as to avoid the problems of attention redundancy and attention deficiency. Second, a new optimization approach, called dynamic-critical reinforcement learning, is introduced to extend the standard supervised method. It always encourages to predict a more acceptable answer so as to address the convergence suppression problem occurred in traditional reinforcement learning algorithms. Extensive experiments on the Stanford Question Answering Dataset (SQuAD) show that our model achieves state-of-the-art results. Meanwhile, our model outperforms previous systems by over 6% in terms of both Exact Match and F1 metrics on two adversarial SQuAD datasets.Comment: Published in 27th International Joint Conference on Artificial Intelligence (IJCAI), 201

A Study of Unsupervised Evaluation Metrics for Practical and Automatic Domain Adaptation

Unsupervised domain adaptation (UDA) methods facilitate the transfer of models to target domains without labels. However, these methods necessitate a labeled target validation set for hyper-parameter tuning and model selection. In this paper, we aim to find an evaluation metric capable of assessing the quality of a transferred model without access to target validation labels. We begin with the metric based on mutual information of the model prediction. Through empirical analysis, we identify three prevalent issues with this metric: 1) It does not account for the source structure. 2) It can be easily attacked. 3) It fails to detect negative transfer caused by the over-alignment of source and target features. To address the first two issues, we incorporate source accuracy into the metric and employ a new MLP classifier that is held out during training, significantly improving the result. To tackle the final issue, we integrate this enhanced metric with data augmentation, resulting in a novel unsupervised UDA metric called the Augmentation Consistency Metric (ACM). Additionally, we empirically demonstrate the shortcomings of previous experiment settings and conduct large-scale experiments to validate the effectiveness of our proposed metric. Furthermore, we employ our metric to automatically search for the optimal hyper-parameter set, achieving superior performance compared to manually tuned sets across four common benchmarks. Codes will be available soon

Urban Treetop Detection and Tree-Height Estimation from Unmanned-Aerial-Vehicle Images

Individual tree detection for urban forests in subtropical environments remains a great challenge due to the various types of forest structures, high canopy closures, and the mixture of evergreen and deciduous broadleaved trees. Existing treetop detection methods based on the canopy-height model (CHM) from UAV images cannot resolve commission errors in heterogeneous urban forests with multiple trunks or strong lateral branches. In this study, we improved the traditional local-maximum (LM) algorithm using a dual Gaussian filter, variable window size, and local normalized correlation coefficient (NCC). Specifically, we adapted a crown model of maximum/minimum tree-crown radii and an angle strategy to detect treetops. We then removed and merged the pending tree vertices. Our results showed that our improved LM algorithm had an average user accuracy (UA) of 87.3% (SD± 4.6), an average producer accuracy (PA) of 82.8% (SD± 4.1), and an overall accuracy of 93.3% (SD± 3.9) for sample plots with canopy closures less than 0.5. As for the sample plots with canopy closures from 0.5 to 1, the accuracies were 78.6% (SD± 31.5), 73.8% (SD± 10.3), and 68.1% (SD± 12.7), respectively. The tree-height estimation accuracy reached more than 0.96, with an average RMSE of 0.61 m. Our results show that the UAV-image-derived CHM can be used to accurately detect individual trees in mixed forests in subtropical cities like Shanghai, China, to provide vital tree-structure parameters for precise and sustainable forest management.National Key R&D Program of ChinaNational Natural Science Foundation of ChinaChina Postdoctoral Science FoundationPeer Reviewe

Secrets of RLHF in Large Language Models Part I: PPO

Large language models (LLMs) have formulated a blueprint for the advancement of artificial general intelligence. Its primary objective is to function as a human-centric (helpful, honest, and harmless) assistant. Alignment with humans assumes paramount significance, and reinforcement learning with human feedback (RLHF) emerges as the pivotal technological paradigm underpinning this pursuit. Current technical routes usually include \textbf{reward models} to measure human preferences, \textbf{Proximal Policy Optimization} (PPO) to optimize policy model outputs, and \textbf{process supervision} to improve step-by-step reasoning capabilities. However, due to the challenges of reward design, environment interaction, and agent training, coupled with huge trial and error cost of large language models, there is a significant barrier for AI researchers to motivate the development of technical alignment and safe landing of LLMs. The stable training of RLHF has still been a puzzle. In the first report, we dissect the framework of RLHF, re-evaluate the inner workings of PPO, and explore how the parts comprising PPO algorithms impact policy agent training. We identify policy constraints being the key factor for the effective implementation of the PPO algorithm. Therefore, we explore the PPO-max, an advanced version of PPO algorithm, to efficiently improve the training stability of the policy model. Based on our main results, we perform a comprehensive analysis of RLHF abilities compared with SFT models and ChatGPT. The absence of open-source implementations has posed significant challenges to the investigation of LLMs alignment. Therefore, we are eager to release technical reports, reward models and PPO code

Potential of Core-Collapse Supernova Neutrino Detection at JUNO

JUNO is an underground neutrino observatory under construction in Jiangmen, China. It uses 20kton liquid scintillator as target, which enables it to detect supernova burst neutrinos of a large statistics for the next galactic core-collapse supernova (CCSN) and also pre-supernova neutrinos from the nearby CCSN progenitors. All flavors of supernova burst neutrinos can be detected by JUNO via several interaction channels, including inverse beta decay, elastic scattering on electron and proton, interactions on C12 nuclei, etc. This retains the possibility for JUNO to reconstruct the energy spectra of supernova burst neutrinos of all flavors. The real time monitoring systems based on FPGA and DAQ are under development in JUNO, which allow prompt alert and trigger-less data acquisition of CCSN events. The alert performances of both monitoring systems have been thoroughly studied using simulations. Moreover, once a CCSN is tagged, the system can give fast characterizations, such as directionality and light curve

Detection of the Diffuse Supernova Neutrino Background with JUNO

As an underground multi-purpose neutrino detector with 20 kton liquid scintillator, Jiangmen Underground Neutrino Observatory (JUNO) is competitive with and complementary to the water-Cherenkov detectors on the search for the diffuse supernova neutrino background (DSNB). Typical supernova models predict 2-4 events per year within the optimal observation window in the JUNO detector. The dominant background is from the neutral-current (NC) interaction of atmospheric neutrinos with 12C nuclei, which surpasses the DSNB by more than one order of magnitude. We evaluated the systematic uncertainty of NC background from the spread of a variety of data-driven models and further developed a method to determine NC background within 15\% with {\it{in}} {\it{situ}} measurements after ten years of running. Besides, the NC-like backgrounds can be effectively suppressed by the intrinsic pulse-shape discrimination (PSD) capabilities of liquid scintillators. In this talk, I will present in detail the improvements on NC background uncertainty evaluation, PSD discriminator development, and finally, the potential of DSNB sensitivity in JUNO

Real-time Monitoring for the Next Core-Collapse Supernova in JUNO

Core-collapse supernova (CCSN) is one of the most energetic astrophysical events in the Universe. The early and prompt detection of neutrinos before (pre-SN) and during the SN burst is a unique opportunity to realize the multi-messenger observation of the CCSN events. In this work, we describe the monitoring concept and present the sensitivity of the system to the pre-SN and SN neutrinos at the Jiangmen Underground Neutrino Observatory (JUNO), which is a 20 kton liquid scintillator detector under construction in South China. The real-time monitoring system is designed with both the prompt monitors on the electronic board and online monitors at the data acquisition stage, in order to ensure both the alert speed and alert coverage of progenitor stars. By assuming a false alert rate of 1 per year, this monitoring system can be sensitive to the pre-SN neutrinos up to the distance of about 1.6 (0.9) kpc and SN neutrinos up to about 370 (360) kpc for a progenitor mass of 30$M_{\odot}$ for the case of normal (inverted) mass ordering. The pointing ability of the CCSN is evaluated by using the accumulated event anisotropy of the inverse beta decay interactions from pre-SN or SN neutrinos, which, along with the early alert, can play important roles for the followup multi-messenger observations of the next Galactic or nearby extragalactic CCSN.Comment: 24 pages, 9 figure

Impacts of Energy and Environmental Policies on Air Quality: Bridging Observational Data, Statistical, and Atmospheric Models

As more countries have adopted regulations on ambient air pollution and announced commitments moving away from fossil fuel energy, assessing the impacts of adopted energy and environmental policies on air quality is essential to evaluating policy progress and informing future actions. The increasing amount of measurement data on pollutant concentrations and precursor emissions provides an opportunity for tracking the progress of policies in mitigating air pollution, but key challenges remain. Levels of measured air pollutants and its precursor emissions are subject to variability in both the natural environment and human activities. This thesis incorporates four studies that integrate research tools across disciplines - from statistical causal inference to atmospheric chemistry models - to assess the impacts of adopted energy and environmental policy on air quality, in support of decision making in energy, climate, and environmental governance. The first study estimates the impacts of energy policies on air quality in major energy-intensive industrial sectors in China with both prospective and retrospective methods. It finds that the realized effects of policy on energy and pollution outcomes are generally much smaller than the projected benefits. The differences between projected and realized benefits stem from how policy baselines are selected and reflect heterogeneity in firms' policy responses. The second study evaluates the impacts of wind power development on air quality and related environmental justice issues in the US. We find substantial air quality benefits from existing wind power, but benefits would increase four-fold if policies could prioritize displacing the most damaging units. The fraction of air quality benefits accruing to low income and minority populations fall below a new 40% goal for future US policies, suggesting targeted efforts are needed to address air pollution disparities. The third study designs a statistical method to estimate the average emission factors of vehicles (the relevant outcome for decision making) based on snapshot measurements (the quantity being measured in the field). We find that a much lower fraction of the measured fleet in Europe is in compliance with emission standards compared to previous estimates. We further quantify the uncertainty and effectiveness of detecting high-emitting vehicles with snapshot measurements. The fourth study evaluates the ability of statistical methods to attribute observed pollutant trends to emissions changes under meteorological variability. We show that widely-used regression methods do not perform well, and we propose a machine learning model that offers better performance. We further provide a lower bound of the estimation error due to interactions between meteorology and emissions.Ph.D

Numerical simulation of thermochemical non-equilibrium flow-field characteristics around a hypersonic atmospheric reentry vehicle

A multi-physics thermochemical non-equilibrium model is established to study the flow characteristics of the plasma sheath around an atmospheric reentry demonstrator. This model includes the tight coupling of Navier-Stokes equations, 54 chemical reactions of air, and a four-temperature model. The processes of dissociation, ionization, and the internal energy exchanges of air components were successfully simulated during aerodynamic heating of the reentry vehicle. The distributions of plasma sheath temperature, the molar fraction of air species, stagnation pressure, surface pressure, and electron number density around the reentry vehicle were obtained at different flight altitudes. Additionally, to validate the numerical model developed in this study, the flow characteristics of the Radio Attenuation Measurement-C-II (RAM-C-II) vehicle are also simulated and then compared with corresponding experimental data. They show good consistency in general. It is found that when the vehicle is at a high flight altitude, there is a strong thermochemical non-equilibrium phenomenon around the vehicle. However, the plasma sheath tends to be in local thermal equilibrium at a low flight altitude. The distance from the shock layer to the stagnation point decreases with a decrease in reentry altitude from 90 to 65 km but increases with a decrease from 65 to 40 km. The electron number density in the shock layer is maximum. The distribution of the electron number density in the wake region differs significantly at different flight altitudes