STEEL: Singularity-aware Reinforcement Learning

Batch reinforcement learning (RL) aims at leveraging pre-collected data to find an optimal policy that maximizes the expected total rewards in a dynamic environment. Nearly all existing algorithms rely on the absolutely continuous assumption on the distribution induced by target policies with respect to the data distribution, so that the batch data can be used to calibrate target policies via the change of measure. However, the absolute continuity assumption could be violated in practice (e.g., no-overlap support), especially when the state-action space is large or continuous. In this paper, we propose a new batch RL algorithm without requiring absolute continuity in the setting of an infinite-horizon Markov decision process with continuous states and actions. We call our algorithm STEEL: SingulariTy-awarE rEinforcement Learning. Our algorithm is motivated by a new error analysis on off-policy evaluation, where we use maximum mean discrepancy, together with distributionally robust optimization, to characterize the error of off-policy evaluation caused by the possible singularity and to enable model extrapolation. By leveraging the idea of pessimism and under some mild conditions, we derive a finite-sample regret guarantee for our proposed algorithm without imposing absolute continuity. Compared with existing algorithms, by requiring only minimal data-coverage assumption, STEEL significantly improves the applicability and robustness of batch RL. Extensive simulation studies and one real experiment on personalized pricing demonstrate the superior performance of our method in dealing with possible singularity in batch RL

Design concept evaluation based on rough number and information entropy theory

Concept evaluation at the early phase of product development plays a crucial role in new product development. It determines the direction of the subsequent design activities. However, the evaluation information at this stage mainly comes from experts' judgments, which is subjective and imprecise. How to manage the subjectivity to reduce the evaluation bias is a big challenge in design concept evaluation. This paper proposes a comprehensive evaluation method which combines information entropy theory and rough number. Rough number is first presented to aggregate individual judgments and priorities and to manipulate the vagueness under a group decision-making environment. A rough number based information entropy method is proposed to determine the relative weights of evaluation criteria. The composite performance values based on rough number are then calculated to rank the candidate design concepts. The results from a practical case study on the concept evaluation of an industrial robot design show that the integrated evaluation model can effectively strengthen the objectivity across the decision-making processes

Peculiarities of Electron-Beam Formation of Hydrophobic and Superhydrophobic Coatings Based on Hydrocarbons of Various Molecular Weights and PTFE

The paper studies the possibility of superhydrophobic coatings formations at exposure of powder mixture of polytetrafluorethylene and hydrocarbons having various molecular weights to low-energy electron beam in vacuum. It is shown that paraffin and PTFE based thin composite coatings may be characterized by superhydrophobic properties. The superhydrophobic properties are attained due to low surface energy of the fluorine-containing component and structured surface due to peculiarities of composite layer formation. The chemical processes observed in electron beam exposed area determine the molecular structure, morphology and the contact angle of thin organic coatings deposited. It is shown that high-molecular-weight hydrocarbon compounds should not be recommended for vacuum electron-beam deposition of superhydrophobic thin coatings because of deep changes in the molecular structure exposed to electron beam. These processes are responsible for high degree of unsaturation of the thin layer formed and for occurrence of oxygen-containing polar groups. The influence of substrate temperature on molecular structure, morphology and hydrophobic properties of thin coatings deposited is investigated. Potentially such coatings may be applied for deposition on the surface of metal capillaries used in biotechnological analyzers

Association Between Education and Health Outcomes Among Adults With Disabilities: Evidence From Shanghai, China

Background Adults with disabilities often have worse health outcomes than do their peers without disabilities. While education is a key determinant of health, there is little research available on the health disparities across education levels among adults with disabilities in developing countries. We therefore examined the association between health outcomes and education among adults with disabilities in Shanghai, China. Methods We used the health examination records of 42,715 adults with disabilities in Shanghai in 2014. Five health outcomes, including two diseases (fatty liver and hemorrhoids) and three risk factors (overweight [body mass index ≥ 24]), high blood glucose, and high blood lipid), were evaluated. Descriptive statistics and Pearson\u27s chi-square test were used to assess differences in participants\u27 demographic and disability characteristics. Pearson\u27s chi-square test and Fisher\u27s exact test were conducted to compare the prevalence of each health outcome among the different education levels. Finally, logistic regression analyses were conducted to explore the association between education and health outcomes after adjusting for sociodemographic characteristics. Results People with an elementary school or lower degree had the highest prevalence of overweight (52.1%) and high blood glucose (20.8%), but the lowest prevalence of hemorrhoids (18.6%) and fatty liver (38.9%). We observed significant differences in the association between education and health outcomes across disability types. For example, in physically disabled adults, higher education was related to higher odds of hemorrhoids (p \u3c 0.001); however, there were no significant disparities in hemorrhoids across the education levels among adults with intellectual disabilities. Discussion: Compared with people without disabilities, adults with disabilities in Shanghai have relatively poor health. The association between education and health outcomes differed according to the health condition and disability type. To reduce the prevalence rate of overweight and high blood glucose among people with disabilities, tailored health promotion initiatives must be developed for people with lower education levels. In contrast, specific attention should be paid to the prevention of hemorrhoids and fatty liver among more-educated people with disabilities. Our study provides important evidence for targeting educational groups with specific disability types for health promotion and intervention