112 research outputs found

    An Artificial Intelligence Solution for Electricity Procurement in Forward Markets

    Full text link
    Retailers and major consumers of electricity generally purchase an important percentage of their estimated electricity needs years ahead in the forward market. This long-term electricity procurement task consists of determining when to buy electricity so that the resulting energy cost is minimised, and the forecast consumption is covered. In this scientific article, the focus is set on a yearly base load product from the Belgian forward market, named calendar (CAL), which is tradable up to three years ahead of the delivery period. This research paper introduces a novel algorithm providing recommendations to either buy electricity now or wait for a future opportunity based on the history of CAL prices. This algorithm relies on deep learning forecasting techniques and on an indicator quantifying the deviation from a perfectly uniform reference procurement policy. On average, the proposed approach surpasses the benchmark procurement policies considered and achieves a reduction in costs of 1.65% with respect to the perfectly uniform reference procurement policy achieving the mean electricity price. Moreover, in addition to automating the complex electricity procurement task, this algorithm demonstrates more consistent results throughout the years. Eventually, the generality of the solution presented makes it well suited for solving other commodity procurement problems.Comment: Scientific article accepted for publication in the Energies journal edited by MDP

    Risk-Sensitive Policy with Distributional Reinforcement Learning

    Full text link
    peer reviewedClassical reinforcement learning (RL) techniques are generally concerned with the design of decision-making policies driven by the maximisation of the expected outcome. Nevertheless, this approach does not take into consideration the potential risk associated with the actions taken, which may be critical in certain applications. To address that issue, the present research work introduces a novel methodology based on distributional RL to derive sequential decision-making policies that are sensitive to the risk, the latter being modelled by the tail of the return probability distribution. The core idea is to replace the Q function generally standing at the core of learning schemes in RL by another function, taking into account both the expected return and the risk. Named the risk-based utility function U, it can be extracted from the random return distribution Z naturally learnt by any distributional RL algorithm. This enables the spanning of the complete potential trade-off between risk minimisation and expected return maximisation, in contrast to fully risk-averse methodologies. Fundamentally, this research yields a truly practical and accessible solution for learning risk-sensitive policies with minimal modification to the distributional RL algorithm, with an emphasis on the interpretability of the resulting decision-making process

    Dexterous Manipulation During Rhythmic Arm Movements in Mars, Moon, and Micro-Gravity

    Get PDF
    Predicting the consequences of one’s own movements can be challenging when confronted with completely novel environmental dynamics, such as microgravity in space. The absence of gravitational force disrupts internal models of the central nervous system (CNS) that have been tuned to the dynamics of a constant 1-g environment since birth. In the context of object manipulation, inadequate internal models produce prediction uncertainty evidenced by increases in the grip force (GF) safety margin that ensures a stable grip during unpredicted load perturbations. This margin decreases with practice in a novel environment. However, it is not clear how the CNS might react to a reduced, but non-zero, gravitational field, and if adaptation to reduced gravity might be beneficial for subsequent microgravity exposure. That is, we wondered if a transfer of learning can occur across various reduced-gravity environments. In this study, we investigated the kinematics and dynamics of vertical arm oscillations during parabolic flight maneuvers that simulate Mars gravity, Moon gravity, and microgravity, in that order. While the ratio of and the correlation between GF and load force (LF) evolved progressively with practice in Mars gravity, these parameters stabilized much quicker to subsequently presented Moon and microgravity conditions. These data suggest that prior short-term adaptation to one reduced-gravity field facilitates the CNS’s ability to update its internal model during exposure to other reduced gravity fields

    Contrasting frequencies of antitumor and anti-vaccine T cells in metastases of a melanoma patient vaccinated with a MAGE tumor antigen

    Get PDF
    Melanoma patients have high frequencies of T cells directed against antigens of their tumor. The frequency of these antitumor T cells in the blood is usually well above that of the anti-vaccine T cells observed after vaccination with tumor antigens. In a patient vaccinated with a MAGE-3 antigen presented by HLA-A1, we measured the frequencies of anti-vaccine and antitumor T cells in several metastases to evaluate their respective potential contribution to tumor rejection. The frequency of anti–MAGE-3.A1 T cells was 1.5 × 10−5 of CD8 T cells in an invaded lymph node, sixfold higher than in the blood. An antitumor cytotoxic T lymphocyte (CTL) recognizing a MAGE-C2 antigen showed a much higher enrichment with a frequency of ∼10%, 1,000 times higher than its blood frequency. Several other antitumor T clonotypes had frequencies >1%. Similar findings were made on a regressing cutaneous metastasis. Thus, antitumor T cells were ∼10,000 times more frequent than anti-vaccine T cells inside metastases, representing the majority of T cells present there. This suggests that the anti-vaccine CTLs are not the effectors that kill the bulk of the tumor cells, but that their interaction with the tumor generates conditions enabling the stimulation of large numbers of antitumor CTLs that proceed to destroy the tumor cells. Naive T cells appear to be stimulated in the course of this process as new antitumor clonotypes arise after vaccination

    Distributional Reinforcement Learning with Unconstrained Monotonic Neural Networks

    Full text link
    The distributional reinforcement learning (RL) approach advocates for representing the complete probability distribution of the random return instead of only modelling its expectation. A distributional RL algorithm may be characterised by two main components, namely the representation and parameterisation of the distribution and the probability metric defining the loss. This research considers the unconstrained monotonic neural network (UMNN) architecture, a universal approximator of continuous monotonic functions which is particularly well suited for modelling different representations of a distribution (PDF, CDF, quantile function). This property enables the decoupling of the effect of the function approximator class from that of the probability metric. The paper firstly introduces a methodology for learning different representations of the random return distribution. Secondly, a novel distributional RL algorithm named unconstrained monotonic deep Q-network (UMDQN) is presented. Lastly, in light of this new algorithm, an empirical comparison is performed between three probability quasimetrics, namely the Kullback-Leibler divergence, Cramer distance and Wasserstein distance. The results call for a reconsideration of all probability metrics in distributional RL, which contrasts with the dominance of the Wasserstein distance in recent publications
    • …
    corecore