Hieros: Hierarchical Imagination on Structured State Space Sequence World Models

One of the biggest challenges to modern deep reinforcement learning (DRL) algorithms is sample efficiency. Many approaches learn a world model in order to train an agent entirely in imagination, eliminating the need for direct environment interaction during training. However, these methods often suffer from either a lack of imagination accuracy, exploration capabilities, or runtime efficiency. We propose Hieros, a hierarchical policy that learns time abstracted world representations and imagines trajectories at multiple time scales in latent space. Hieros uses an S5 layer-based world model, which predicts next world states in parallel during training and iteratively during environment interaction. Due to the special properties of S5 layers, our method can train in parallel and predict next world states iteratively during imagination. This allows for more efficient training than RNN-based world models and more efficient imagination than Transformer-based world models. We show that our approach outperforms the state of the art in terms of mean and median normalized human score on the Atari 100k benchmark, and that our proposed world model is able to predict complex dynamics very accurately. We also show that Hieros displays superior exploration capabilities compared to existing approaches.Comment: Submitted to ICLR 2024, 23 pages, 11 figures, code available at: https://github.com/Snagnar/Hiero

Generalized Activity Assessment computed fully distributed within a Wireless Body Area Network

Currently available wearables are usually based on a single sensor node with integrated capabilities for classifying different activities. The next generation of cooperative wearables could be able to identify not only activities, but also to evaluate them qualitatively using the data of several sensor nodes attached to the body, to provide detailed feedback for the improvement of the execution. Especially within the application domains of sports and health-care, such immediate feedback to the execution of body movements is crucial for (re-)learning and improving motor skills. To enable such systems for a broad range of activities, generalized approaches for human motion assessment within sensor networks are required. In this paper, we present a generalized trainable activity assessment chain (AAC) for the online assessment of periodic human activity within a wireless body area network. AAC evaluates the execution of separate movements of a prior trained activity on a fine-grained quality scale. We connect qualitative assessment with human knowledge by projecting the AAC on the hierarchical decomposition of motion performed by the human body as well as establishing the assessment on a kinematic evaluation of biomechanically distinct motion fragments. We evaluate AAC in a real-world setting and show that AAC successfully delimits the movements of correctly performed activity from faulty executions and provides detailed reasons for the activity assessment

Generalized Activity Assessment computed fully distributed within a Wireless Body Area Network

Currently available wearables are usually based on a single sensor node with integrated capabilities for classifying different activities. The next generation of cooperative wearables could be able to identify not only activities, but also to evaluate them qualitatively using the data of several sensor nodes attached to the body, to provide detailed feedback for the improvement of the execution. Especially within the application domains of sports and health-care, such immediate feedback to the execution of body movements is crucial for (re-)learning and improving motor skills. To enable such systems for a broad range of activities, generalized approaches for human motion assessment within sensor networks are required. In this paper, we present a generalized trainable activity assessment chain (AAC) for the online assessment of periodic human activity within a wireless body area network. AAC evaluates the execution of separate movements of a prior trained activity on a fine-grained quality scale. We connect qualitative assessment with human knowledge by projecting the AAC on the hierarchical decomposition of motion performed by the human body as well as establishing the assessment on a kinematic evaluation of biomechanically distinct motion fragments. We evaluate AAC in a real-world setting and show that AAC successfully delimits the movements of correctly performed activity from faulty executions and provides detailed reasons for the activity assessment

Different Outcomes of Experimental Hepatitis E Virus Infection in Diverse Mouse Strains, Wistar Rats, and Rabbits

Hepatitis E virus (HEV) is the causative agent of acute hepatitis E in humans in developing countries, but autochthonous cases of zoonotic genotype 3 (HEV-3) infection also occur in industrialized countries. In contrast to swine, rats, and rabbits, natural HEV infections in mice have not yet been demonstrated. The pig represents a well-established large animal model for HEV-3 infection, but a suitable small animal model mimicking natural HEV-3 infection is currently missing. Therefore, we experimentally inoculated C57BL/6 mice (wild-type, IFNAR−/−, CD4−/−, CD8−/−) and BALB/c nude (nu/nu) mice, Wistar rats, and European rabbits with a wild boar-derived HEV-3 strain and monitored virus replication and shedding, as well as humoral immune responses. HEV RNA and anti-HEV antibodies were detected in one and two out of eight of the rats and all rabbits inoculated, respectively, but not in any of the mouse strains tested. Remarkably, immunosuppressive dexamethasone treatment of rats did not enhance their susceptibility to HEV infection. In rabbits, immunization with recombinant HEV-3 and ratHEV capsid proteins induced protection against HEV-3 challenge. In conclusion, the rabbit model for HEV-3 infection may serve as a suitable alternative to the non-human primate and swine models, and as an appropriate basis for vaccine evaluation studies

How to Survive Dynamic Pricing Competition in E-commerce

ABSTRACT Pricing on e-commerce platforms is highly challenging. Sellers typically i) rival against dozens of competitors, ii) decide on prices for thousands of products, and iii) face steadily changing market situations. With respect to pricing, the challenge is to circumvent the curse of dimensionality to dynamically price products for a given market situation in a timely manner. In this project, we create a stochastic pricing model by analyzing recorded market data. This pricing model can be applied ad-hoc in less than a millisecond per item, allowing us to react immediately to new market situations. Our pricing approach is currently being applied in practice by a large German book seller on Amazon and outperforms the previous rule-based strategy by over 20% with respect to cash-in per book. CCS Concepts •Applied computing → Online shopping; E-commerce infrastructure; Decision analysis; Keywords Dynamic Pricing; Oligopoly Competition; Online Markets; Demand Estimation CHALLENGE Modern market platforms such as Amazon Marketplace or eBay are highly dynamic as sellers can observe the current market situation at any time and adjust their prices instantly. For sellers that handle large inventories, this dynamic is hard to manage as an optimal pricing decision requires handling a multitude of dimensions for each competitor (e.g., price, quality, shipping time, shipping costs, rating). Moreover, financial aspects such as discounting as well as inventory holding costs have to be taken into account. In this project, we partner with adanbo GmbH. adanbo is among the top 10 sellers for used books on Amazon in Germany with an inventory of over 80,000 distinct books (ISBN), each with multiple items (1-20). Our seller can decide -to some extent -on the replenishment of used books (by choosing purchase prices). However, supply is limited and it is not possible to directly reorder specific books. Hence, the challenge is to extract as much profit as pos- Copyright is held by the author(s). sible from a given number of books (inventory level) in a reasonable amount of time. The pricing strategy of our project partner is characterized by a rule-based system that has been developed over the past years by carefully adjusting rules to lessons learned from selling books on Amazon. As our project partner has more than 10 years of experience in the market, we consider his strategy to be effective and accurate. However, market dynamics are increasingly sophisticated making rule-based strategies increasingly hard to handle and maintain. Our goal is to develop a pricing strategy that maximizes expected discounted long-term profits while taking into account the constraints mentioned above. We seek to compute data-driven pricing strategies that are applicable even for large inventories. DATA-DRIVEN PRICING MODEL The project is devoted to revenue managemen

Netboost: boosting-supported network analysis improves high-dimensional omics prediction in acute myeloid leukemia and Huntington’s disease

State-of-the art selection methods fail to identify weak but cumulative effects of features found in many high-dimensional omics datasets. Nevertheless, these features play an important role in certain diseases. We present Netboost, a three-step dimension reduction technique. First, a boosting-based filter is combined with the topological overlap measure to identify the essential edges of the network. Second, sparse hierarchical clustering is applied on the selected edges to identify modules and finally module information is aggregated by the first principal components. We demonstrate the application of the newly developed Netboost in combination with CoxBoost for survival prediction of DNA methylation and gene expression data from 180 acute myeloid leukemia (AML) patients and show, based on cross-validated prediction error curve estimates, its prediction superiority over variable selection on the full dataset as well as over an alternative clustering approach. The identified signature related to chromatin modifying enzymes was replicated in an independent dataset, the phase II AMLSG 12-09 study. In a second application we combine Netboost with Random Forest classification and improve the disease classification error in RNA-sequencing data of Huntington's disease mice. Netboost is a freely available Bioconductor R package for dimension reduction and hypothesis generation in high-dimensional omics applications

Gas-discharge plasma-assisted functionalization of titanium implant surfaces

A crucial factor for in-growth of metallic implants in the bone stock is the rapid cellular acceptance whilst prevention of bacterial adhesion on the surface. Such contradictorily adhesion events could be triggered by surface properties. There already exists fundamental knowledge about the influence of physicochemical surface properties like roughness, titanium dioxide modifications, cleanness, and (mainly ceramic) coatings on cell and microbial behavior in vitro and in vivo. The titanium surface can be equipped with antimicrobial properties by plasma-based copper implantation, which allows the release and generation of small concentrations of copper ions during contact with water-based biological liquids. Additionally, the titanium surface was equipped with amino groups by the deposition of an ultrathin plasma polymer. This coating on the one hand does not significantly reduce the generation of copper ions, and on the other hand improves the adhesion and spreading of osteoblast cells. The process development was accompanied by physicochemical surface analyses like XPS, FTIR, contact angle, SEM, and AFM. Very thin modified layers were created, which are resistant to hydrolysis and delamination. These titanium surface functionalizations were found to have either an antimicrobial activity or cell-adhesive properties. Intramuscular implantation of titanium samples coated with the cell-adhesive plasma polymer in rats revealed a reduced inflammation reaction compared to uncoated titanium. © (2010) Trans Tech Publications