82,275 research outputs found

    Deep Ordinal Reinforcement Learning

    Full text link
    Reinforcement learning usually makes use of numerical rewards, which have nice properties but also come with drawbacks and difficulties. Using rewards on an ordinal scale (ordinal rewards) is an alternative to numerical rewards that has received more attention in recent years. In this paper, a general approach to adapting reinforcement learning problems to the use of ordinal rewards is presented and motivated. We show how to convert common reinforcement learning algorithms to an ordinal variation by the example of Q-learning and introduce Ordinal Deep Q-Networks, which adapt deep reinforcement learning to ordinal rewards. Additionally, we run evaluations on problems provided by the OpenAI Gym framework, showing that our ordinal variants exhibit a performance that is comparable to the numerical variations for a number of problems. We also give first evidence that our ordinal variant is able to produce better results for problems with less engineered and simpler-to-design reward signals.Comment: replaced figures for better visibility, added github repository, more details about source of experimental results, updated target value calculation for standard and ordinal Deep Q-Networ

    Proposal for a Performance Dashboard for the Monitoringof Water and Sewage Service Companies (WaSCs)

    Get PDF
    The water and sewage industry provides an essential service to the community, but it is characterized by natural monopoly tendencies of service suppliers. In this framework, it is very important to assist regulators with a small set of critical indicators (performance dashboard) for the evaluation and monitoring of the service provided by Water and Sewage Companies (WaSCs). The paper originates from the analysis of situation of Piemonte (Italy), where each regional and local body adopts a proprietary Performance Measurement System (PMS). In order to improve the coordination of information flow and to support the definition of common service standards, a methodology to merge existing PMSs and define a unique shared reference system is proposed. The Kaplan and Norton's Balanced Scorecard (BSC) is adopted as the reference model of this approach. BSC is widely recognized to be an exhaustive and balanced framework in describing the performances of an organization and ensures that all the operational aspects of WaSCs are adequately monitored. The output of the proposed procedure is a general performance dashboard for the monitoring of WaSCs. The dashboard is shown and some remarks about indicators properties are developed. In particular, this analysis highlights some common pitfalls originated by a ‘rushed' aggregation of several performance indicators. Description is supported by several example

    Structural determination of archaeal UDP-N-acetylglucosamine 4-epimerase from Methanobrevibacter ruminantium M1 in complex with the bacterial cell wall intermediate UDP-N-acetylmuramic acid

    Get PDF
    The crystal structure of UDP-N-acetylglucosamine 4-epimerase (UDP-GlcNAc 4-epimerase; WbpP; EC 5.1.3.7), from the archaeal methanogen Methanobrevibacter ruminantium strain M1, was determined to a resolution of 1.65 Å. The structure, with a single monomer in the crystallographic asymmetric unit, contained a conserved N-terminal Rossmann fold for nucleotide binding and an active site positioned in the C-terminus. UDP-GlcNAc 4-epimerase is a member of the short-chain dehydrogenase/reductase superfamily, sharing sequence motifs and structural elements characteristic of this family of oxidoreductases and bacterial 4-epimerases. The protein was co-crystallized with coenzyme NADH and UDP-N-acetylmuramic acid, the latter an unintended inclusion and well known product of the bacterial enzyme MurB and a critical intermediate for bacterial cell wall synthesis. This is a non-native UDP sugar amongst archaea and was most likely incorporated from the Eschericha coli expression host during purification of the recombinant enzyme

    Measurements of neutral vector resonance in Higgsless models at the LHC

    Full text link
    In Higgsless models, new vector resonances appear to restore the unitarity of the W_L W_L scattering amplitude without the Higgs boson. In the ideal delocalized three site Higgsless model, one of large prodcution cross section of the neutral vector resonance (Z') at the Large Hadron Collider is the W-associated production, pp \to Z'W \to WWW. Although the dileptonic decay channnel, l\nu l'\nu 'jj, is experimentally clean to search for the Z' signals, it is difficult to reconstruct the Z' invariant mass due to the two neutrinos in the final state. We study collider signatures of Z' using the M_{T2}-Assisted On-Shell (MAOS) reconstruction of the missing neutrino momenta. We show the prospect of the Z' mass determination in the channel, l\nu l'\nu 'jj, at the Large Hadron Collider.Comment: 16 pages, 6 figures, 5 tables; v2: references added, minor corrections, version published in JHE

    ContextVP: Fully Context-Aware Video Prediction

    Full text link
    Video prediction models based on convolutional networks, recurrent networks, and their combinations often result in blurry predictions. We identify an important contributing factor for imprecise predictions that has not been studied adequately in the literature: blind spots, i.e., lack of access to all relevant past information for accurately predicting the future. To address this issue, we introduce a fully context-aware architecture that captures the entire available past context for each pixel using Parallel Multi-Dimensional LSTM units and aggregates it using blending units. Our model outperforms a strong baseline network of 20 recurrent convolutional layers and yields state-of-the-art performance for next step prediction on three challenging real-world video datasets: Human 3.6M, Caltech Pedestrian, and UCF-101. Moreover, it does so with fewer parameters than several recently proposed models, and does not rely on deep convolutional networks, multi-scale architectures, separation of background and foreground modeling, motion flow learning, or adversarial training. These results highlight that full awareness of past context is of crucial importance for video prediction.Comment: 19 pages. ECCV 2018 oral presentation. Project webpage is at https://wonmin-byeon.github.io/publication/2018-ecc

    FROM HIGH-FIDELITY NUMERICAL SIMULATIONS OF A LIQUID-FILM ATOMIZATION TO A REGIME CLASSIFICATION

    Get PDF
    High-fidelity numerical simulations of spray formation were conducted with the aim of improving fundamental understanding of airblast liquid-film atomization. The gas/liquid interaction in the near-nozzle region is investigated for a multitude of operating conditions in order to extrapolate phenomenological and breakup predictions. To reach this goal, the robust conservative level-set (RCLS) method was used. For a fixed prefilmer geometry, we performed a parametric study on the impact of various liquid and gas velocities on the topological evolution of the liquid interface. The behavior and development of the liquid film is found to be influenced mainly by the relative inertia of the gas and the liquid, the liquid surface tension, and interfacial shear stresses. Preliminary regime maps predicting the prefilming liquid-sheet atomization behavior are constructed based on our numerical results. Three distinct types of “regime” are reported: accumulation, ligament-merging, and three-dimensional wave mode. In addition, these results also show the influence of vortex action and rim-driven dynamics on the breakup mechanism at the atomizer edge. An increase in liquid injection speed leads to the generation of smaller droplets; whereas, an increase in air velocity does not point to one simple conclusion

    An additive subfamily of enlargements of a maximally monotone operator

    Get PDF
    We introduce a subfamily of additive enlargements of a maximally monotone operator. Our definition is inspired by the early work of Simon Fitzpatrick. These enlargements constitute a subfamily of the family of enlargements introduced by Svaiter. When the operator under consideration is the subdifferential of a convex lower semicontinuous proper function, we prove that some members of the subfamily are smaller than the classical Ï”\epsilon-subdifferential enlargement widely used in convex analysis. We also recover the epsilon-subdifferential within the subfamily. Since they are all additive, the enlargements in our subfamily can be seen as structurally closer to the Ï”\epsilon-subdifferential enlargement

    Non-minimality of corners in subriemannian geometry

    Get PDF
    We give a short solution to one of the main open problems in subriemannian geometry. Namely, we prove that length minimizers do not have corner-type singularities. With this result we solve Problem II of Agrachev's list, and provide the first general result toward the 30-year-old open problem of regularity of subriemannian geodesics.Comment: 11 pages, final versio
    • 

    corecore