279 research outputs found
Learning Heuristic Selection with Dynamic Algorithm Configuration
A key challenge in satisficing planning is to use multiple heuristics within
one heuristic search. An aggregation of multiple heuristic estimates, for
example by taking the maximum, has the disadvantage that bad estimates of a
single heuristic can negatively affect the whole search. Since the performance
of a heuristic varies from instance to instance, approaches such as algorithm
selection can be successfully applied. In addition, alternating between
multiple heuristics during the search makes it possible to use all heuristics
equally and improve performance. However, all these approaches ignore the
internal search dynamics of a planning system, which can help to select the
most useful heuristics for the current expansion step. We show that dynamic
algorithm configuration can be used for dynamic heuristic selection which takes
into account the internal search dynamics of a planning system. Furthermore, we
prove that this approach generalizes over existing approaches and that it can
exponentially improve the performance of the heuristic search. To learn dynamic
heuristic selection, we propose an approach based on reinforcement learning and
show empirically that domain-wise learned policies, which take the internal
search dynamics of a planning system into account, can exceed existing
approaches.Comment: Long version of the paper at the International Conference on
Automated Planning and Scheduling (ICAPS) 202
Contextualize Me -- The Case for Context in Reinforcement Learning
While Reinforcement Learning ( RL) has made great strides towards solving
increasingly complicated problems, many algorithms are still brittle to even
slight environmental changes. Contextual Reinforcement Learning (cRL) provides
a framework to model such changes in a principled manner, thereby enabling
flexible, precise and interpretable task specification and generation. Our goal
is to show how the framework of cRL contributes to improving zero-shot
generalization in RL through meaningful benchmarks and structured reasoning
about generalization tasks. We confirm the insight that optimal behavior in cRL
requires context information, as in other related areas of partial
observability. To empirically validate this in the cRL framework, we provide
various context-extended versions of common RL environments. They are part of
the first benchmark library, CARL, designed for generalization based on cRL
extensions of popular benchmarks, which we propose as a testbed to further
study general agents. We show that in the contextual setting, even simple RL
environments become challenging - and that naive solutions are not enough to
generalize across complex context spaces.Comment: arXiv admin note: substantial text overlap with arXiv:2110.0210
Expression of p89c-Mybex9b, an alternatively spliced form of c-Myb, is required for proliferation and survival of p210BCR/ABL-expressing cells
The c-Myb gene encodes the p75c-Myb isoform and less-abundant proteins generated by alternatively spliced transcripts. Among these, the best known is pc-Mybex9b, which contains 121 additional amino acids between exon 9 and 10, in a domain involved in protein–protein interactions and negative regulation. In hematopoietic cells, expression of pc-Mybex9b accounts for 10–15% of total c-Myb; these levels may be biologically relevant because modest changes in c-Myb expression affects proliferation and survival of leukemic cells and lineage choice and frequency of normal hematopoietic progenitors. In this study, we assessed biochemical activities of pc-Mybex9b and the consequences of perturbing its expression in K562 and primary chronic myeloid leukemia (CML) progenitor cells. Compared with p75c-Myb, pc-Mybex9b is more stable and more effective in transactivating Myb-regulated promoters. Ectopic expression of pc-Mybex9b enhanced proliferation and colony formation and reduced imatinib (IM) sensitivity of K562 cells; conversely, specific downregulation of pc-Mybex9b reduced proliferation and colony formation, enhanced IM sensitivity of K562 cells and markedly suppressed colony formation of CML CD34+ cells, without affecting the levels of p75c-Myb. Together, these studies indicate that expression of the low-abundance pc-Mybex9b isoform has an important role for the overall biological effects of c-Myb in BCR/ABL-transformed cells
Dramatic Repositioning of c-Myb to Different Promoters during the Cell Cycle Observed by Combining Cell Sorting with Chromatin Immunoprecipitation
The c-Myb transcription factor is a critical regulator of proliferation and stem cell differentiation, and mutated alleles of c-Myb are oncogenic, but little is known about changes in c-Myb activity during the cell cycle. To map the association of c-Myb with specific target genes during the cell cycle, we developed a novel Fix-Sort-ChIP approach, in which asynchronously growing cells were fixed with formaldehyde, stained with Hoechst 33342 and separated into different cell cycle fractions by flow sorting, then processed for chromatin immunoprecipitation (ChIP) assays. We found that c-Myb actively repositions, binding to some genes only in specific cell cycle phases. In addition, the specificity of c-Myb is dramatically different in small subpopulations of cells, for example cells in the G2/M phase of the cell cycle, than in the bulk population. The repositioning of c-Myb during the cell cycle is not due to changes in its expression and also occurs with ectopically expressed, epitope-tagged versions of c-Myb. The repositioning occurs in established cell lines, in primary human CD34+ hematopoietic progenitors and in primary human acute myeloid leukemia cells. The combination of fixation, sorting and ChIP analysis sheds new light on the dynamic nature of gene regulation during the cell cycle and provides a new type of tool for the analysis of gene regulation in small subsets of cells, such as cells in a specific phase of the cell cycle
GSK-3β Is Required for Memory Reconsolidation in Adult Brain
Activation of GSK-3β is presumed to be involved in various neurodegenerative diseases, including Alzheimer's disease (AD), which is characterized by memory disturbances during early stages of the disease. The normal function of GSK-3β in adult brain is not well understood. Here, we analyzed the ability of heterozygote GSK-3β knockout (GSK+/−) mice to form memories. In the Morris water maze (MWM), learning and memory performance of GSK+/− mice was no different from that of wild-type (WT) mice for the first 3 days of training. With continued learning on subsequent days, however, retrograde amnesia was induced in GSK+/− mice, suggesting that GSK+/− mice might be impaired in their ability to form long-term memories. In contextual fear conditioning (CFC), context memory was normally consolidated in GSK+/− mice, but once the original memory was reactivated, they showed reduced freezing, suggesting that GSK+/− mice had impaired memory reconsolidation. Biochemical analysis showed that GSK-3β was activated after memory reactivation in WT mice. Intraperitoneal injection of a GSK-3 inhibitor before memory reactivation impaired memory reconsolidation in WT mice. These results suggest that memory reconsolidation requires activation of GSK-3β in the adult brain
- …