SDRL: Interpretable and Data-efficient Deep Reinforcement Learning Leveraging Symbolic Planning

Deep reinforcement learning (DRL) has gained great success by learning directly from high-dimensional sensory inputs, yet is notorious for the lack of interpretability. Interpretability of the subtasks is critical in hierarchical decision-making as it increases the transparency of black-box-style DRL approach and helps the RL practitioners to understand the high-level behavior of the system better. In this paper, we introduce symbolic planning into DRL and propose a framework of Symbolic Deep Reinforcement Learning (SDRL) that can handle both high-dimensional sensory inputs and symbolic planning. The task-level interpretability is enabled by relating symbolic actions to options.This framework features a planner -- controller -- meta-controller architecture, which takes charge of subtask scheduling, data-driven subtask learning, and subtask evaluation, respectively. The three components cross-fertilize each other and eventually converge to an optimal symbolic plan along with the learned subtasks, bringing together the advantages of long-term planning capability with symbolic knowledge and end-to-end reinforcement learning directly from a high-dimensional sensory input. Experimental results validate the interpretability of subtasks, along with improved data efficiency compared with state-of-the-art approaches

Germination, respiration and photosynthesis in seeds of dwarf mistletoe (Arceuthobium)

Germination, respiration, and photosynthesis in seeds of dwarf mistletoe (Arcenthobium) were studied. The effects of 1 hour soakings of seeds in aqueous solutions of 1, 2, or 3% H2O2 or 1, 2, 3, 4, or 5% Chlorox on germination of seeds were tested. Germination rates for seeds from three consecutive years (1975-1977) were obtained. Germination rates varied widely. This variance transcended subtle changes in treatment. Pretreatment of seeds with Chlorox resulted in significantly lower germination rates than those of the control. Chlorophyll concentrations were determined for seeds and aerial shoot tissue of four species. Seeds of A. douglasii had the highest chlorophyll concentration (0.39 mg/g fresh weight) while seeds of A. tsugense had the lowest concentration (0.25 mg/g fresh weight). Net O2 uptake by seeds of A. campylopodum in dark and in light was measured by manometric and polarographic methods. The mean values were 353 μl O2 g-1 h-1 in the dark and 201 O2 g-1 h-1 in the light. The difference between these rates is apparently due to O2 evolution during photosynthesis. In light the seeds can fix 43% of the CO2 produced by respiration. Experiments in which seeds were exposed to 14CO2 in light confirmed that the seeds are able to fix CO2. Extraction of seeds with ethanol showed that 97-99% of the incorporated 14C was ethanol soluble. Ten to sixteen percent of the ethanol fraction was chloroform soluble while the rest was H2o soluble. Ion exchange separation of the H2o phase showed that 11-25% of 14C activity was cationic, 15-29% anionic, and 53-67% neutral

A Practical Incremental Learning Framework For Sparse Entity Extraction

This work addresses challenges arising from extracting entities from textual data, including the high cost of data annotation, model accuracy, selecting appropriate evaluation criteria, and the overall quality of annotation. We present a framework that integrates Entity Set Expansion (ESE) and Active Learning (AL) to reduce the annotation cost of sparse data and provide an online evaluation method as feedback. This incremental and interactive learning framework allows for rapid annotation and subsequent extraction of sparse data while maintaining high accuracy. We evaluate our framework on three publicly available datasets and show that it drastically reduces the cost of sparse entity annotation by an average of 85% and 45% to reach 0.9 and 1.0 F-Scores respectively. Moreover, the method exhibited robust performance across all datasets.Comment: https://www.aclweb.org/anthology/C18-1059