Reinforcement Learning with Stepwise Fairness Constraints

AI methods are used in societally important settings, ranging from credit to employment to housing, and it is crucial to provide fairness in regard to algorithmic decision making. Moreover, many settings are dynamic, with populations responding to sequential decision policies. We introduce the study of reinforcement learning (RL) with stepwise fairness constraints, requiring group fairness at each time step. Our focus is on tabular episodic RL, and we provide learning algorithms with strong theoretical guarantees in regard to policy optimality and fairness violation. Our framework provides useful tools to study the impact of fairness constraints in sequential settings and brings up new challenges in RL.Comment: Fairness, Reinforcement Learnin

BeamSearchQA: Large Language Models are Strong Zero-Shot QA Solver

Open-domain question answering is a crucial task that often requires accessing external information. Existing methods typically adopt a single-turn retrieve-then-read approach, where relevant documents are first retrieved, and questions are then answered based on the retrieved information. However, there are cases where answering a question requires implicit knowledge that is not directly retrievable from the question itself. In this work, we propose a novel question-answering pipeline called BeamSearchQA. Our approach leverages large language models to iteratively generate new questions about the original question, enabling an iterative reasoning process. By iteratively refining and expanding the scope of the question, our method aims to capture and utilize hidden knowledge that may not be directly obtainable through retrieval. We evaluate our approach on the widely-used open-domain NQ and WebQ datasets. The experimental results demonstrate that BeamSearchQA significantly outperforms other zero-shot baselines, indicating its effectiveness in tackling the challenges of open-domain question answering.Comment: Work in progres

LEAD: Liberal Feature-based Distillation for Dense Retrieval

Knowledge distillation is often used to transfer knowledge from a strong teacher model to a relatively weak student model. Traditional knowledge distillation methods include response-based methods and feature-based methods. Response-based methods are used the most widely but suffer from lower upper limit of model performance, while feature-based methods have constraints on the vocabularies and tokenizers. In this paper, we propose a tokenizer-free method liberal feature-based distillation (LEAD). LEAD aligns the distribution between teacher model and student model, which is effective, extendable, portable and has no requirements on vocabularies, tokenizer, or model architecture. Extensive experiments show the effectiveness of LEAD on several widely-used benchmarks, including MS MARCO Passage, TREC Passage 19, TREC Passage 20, MS MARCO Document, TREC Document 19 and TREC Document 20.Comment: Work in progres

The carbonaceous aerosol levels still remain a challenge in the Beijing-Tianjin-Hebei region of China: Insights from continuous high temporal resolution measurements in multiple cities

Carbonaceous aerosols in high emission areas attract worldwide attention of the scientific community and the public due to their adverse impacts on the environment, human health and climate. However, long-term continuous hourly measurements are scarce on the regional scale. In this study, a one-year hourly measurement (from December 1, 2016 to November 30, 2017) of organic carbon (OC) and elemental carbon (EC) in airborne fine particles was performed using semi-continuous OC/EC analyzers in Beijing, Tianjin, Shijiazhuang and Tangshan in the Beijing-Tianjin-Hebei (BTH) region in China, which is one of high emission areas in China, even in the world. Marked spatiotemporal variations were observed. The highest concentrations of OC (22.8 ± 30.6 μg/m 3 ) and EC (5.4 ± 6.5 μg/m 3 ) occurred in Shijiangzhuang while the lowest concentrations of OC (11.0 ± 10.7 μg/m 3 ) and EC (3.1 ± 3.6 μg/m 3 ) were obtained in Beijing and Tianjin, respectively. Pronounced monthly, seasonal and diurnal variations of OC and EC were recorded. Compared to published data from the past two decades for the BTH region, our OC and EC levels were lower, implying some effect of recent measures for improving the air quality. Significant correlations of OC versus EC (p < 0.001) were found throughout the study period with high slopes and correlation coefficients in winter, but low slopes and correlation coefficients in summer. The estimated secondary OC (SOC), based on the minimum R squared (MRS) method, represented 29%, 47%, 38% and 48% of the OC for Beijing, Tianjin, Shijiazhuang and Tangshan, respectively. These percentages are larger than previous ones obtained for the BTH region in the past decade. There were obvious differences in the potential source regions of OC and EC among the four cities. Obvious prominent potential source areas of OC and EC were observed for Beijing, which were mainly located in the central and western areas of Inner Mongolia and even extended to the Mongolian regions, which is different from the findings in previous studies. For all sites, adjacent areas of the main provinces in northern China were found to be important potential source areas. © 2019 The Author

Investigation on Condensation Characteristics and Removal Performance of SO₃ in Low-Low-Temperature Electrostatic Precipitator

The low-low-temperature electrostatic precipitator (LLT-ESP) is considered one of the mainstream technological approaches for achieving ultra-low ash emissions and has already been applied in many coal-fired power plants. Particulate matter and SO3 can both be removed by LLT-ESP. However, the removal performance of SO3 is relatively lower than that of particulate matter, which is caused by the condensation characteristics of SO3. In this paper, the condensation characteristics of SO3 were investigated on a simulated experimental system, and several measurement and characteristic methods were used to investigate mechanisms. After reducing the flue gas temperature with a heat exchanger, the size distribution of particulate matter, the mass concentration of SO3 on different sizes of particulate matter, as well as the microscopic morphology and elemental composition of particulate matter, were all experimentally studied. The results indicate that gaseous SO3 transformed into a liquid phase by heterogeneous or homogeneous condensation and then adhered to the surface of particulate matter through nucleation–condensation, collision–coalescence, and adsorption reactions. Furthermore, the removal efficiency of SO3 in LLT-ESP was also investigated under various conditions, such as ash concentration and flue gas temperature drop, suggesting that a higher ash concentration and a more significant temperature drop were beneficial for improving SO3 removal efficiency. Nevertheless, it is worth noting that the impact was limited by a further increase in ash concentration and a drop in flue gas temperature

DSCC2015-9993 HIERARCHICAL DESIGN FOR CONNECTED CRUISE CONTROL

ABSTRACT In this paper, we propose a hierarchical framework to reduce the design complexity of connected cruise control (CCC), which is used to regulate the longitudinal motion of a vehicle by utilizing wireless vehicle-to-vehicle (V2V) communication. A high-level controller is designed to generate desired motion of the CCC vehicle based on the motion of multiple vehicles ahead. A low-level controller is used to regulate the engine torque and select the appropriate gear to enable the vehicle to track the desired motion. To cope with external disturbances and uncertain physical parameters, we use an adaptive control strategy for the low-level controller. In a case study, we design a specific CCC algorithm by using the presented hierarchical framework. Numerical simulations are used to validate the analytical results and test the system performance

An efficient sparse pruning method for human pose estimation

Human pose estimation (HPE) is crucial for computer vision (CV). Moreover, it’s a vital step for computers to understand human actions and behaviours. However, the huge number of parameters and calculations in the HPE model have brought big challenges to deploy to resource-constrained mobile devices. Aiming to overcome the challenge, we propose a sparse pruning method (SPM) for the HPE model. First, L1 regularisation is added in the training phase of the original model, and network parameters of the convolution layers (CLs) and batch normalisation layers (BNLs) are sparsely trained to obtain a network structure with sparse weights. We then combine the sparse weights of filters with the scaling parameters of the BNLs to determine their importance. Finally, the structured pruning method is used to prune the sparse filters and corresponding channels. SPM can reduce the number of model parameters and calculations without affecting precision. Promising results indicate that SPM outperforms other advanced pruning methods