Avoiding Wireheading with Value Reinforcement Learning

How can we design good goals for arbitrarily intelligent agents? Reinforcement learning (RL) is a natural approach. Unfortunately, RL does not work well for generally intelligent agents, as RL agents are incentivised to shortcut the reward sensor for maximum reward -- the so-called wireheading problem. In this paper we suggest an alternative to RL called value reinforcement learning (VRL). In VRL, agents use the reward signal to learn a utility function. The VRL setup allows us to remove the incentive to wirehead by placing a constraint on the agent's actions. The constraint is defined in terms of the agent's belief distributions, and does not require an explicit specification of which actions constitute wireheading.Comment: Artificial General Intelligence (AGI) 201

Analyses of the vrl gene cluster in Desulfococcus multivorans: Homologous to the virulence-associated locus of the ovine footrot pathogen Dichelobacter nodosus strain A198

Major parts of the virulence-associated vrl locus known from the gammaproteobacterium Dichelobacter nodosus, the causative agent of ovine footrot, were analyzed in the genome of the sulfate-reducing deltaproteobacterium Desulfococcus multivorans. In the genome of D. multivorans 13 of the 19 vrl genes described for D. nodosus are present and highly conserved with respect to gene sequence and order. The vrl locus and its flanking regions suggest a bacteriophage-mediated transfer into the genome of D. multivorans. Comparative analysis of the deduced Vrl proteins reveals a wide distribution of parts of the virulence-associated vrl locus in distantly related bacteria. Horizontal transfer is suggested as driving mechanism for the circulation of the vrl genes in bacteria. Except for the vrlBMN genes D. multivorans and Desulfovibrio desulfuricans G20 together contain all vrl genes displaying a high degree of similarity. For D. multivorans it could be shown that guanine plus cytosine (GC) content, GC skew, di-, tri- or tetranucleotide distribution did not differ between the vrl locus and its flanking sequences. This could be a hint that the vrl locus originated from a related organism or at least a genome with similar characteristics. The conspicuous high degree of conservation of the analyzed vrl genes may result from a recent transfer event or reflect a function of the vrl genes, which is still unknown and not necessarily disease associated. The latter is supported by the evidence for expression of the vrl genes in D. multivorans, which has not been described as pathogen or to be associated to any disease pattern before

The molecular hallmarks of primary and secondary vitreoretinal lymphoma

Vitreoretinal lymphoma (VRL) is a rare subtype of diffuse large B-cell lymphoma (DLBCL) considered a variant of primary central nervous system lymphoma (PCNSL). Diagnosis of VRL requires examination of vitreous fluid, but cytologic differentiation from uveitis remains difficult. Due to its rarity and difficulty in obtaining diagnostic material, little is known about the genetic profile of VRL. The aim of our study was to investigate the mutational profile of a large series of primary and secondary VRL. Targeted next generation sequencing using a custom panel containing the most frequent mutations in PCNSL was performed on 34 vitrectomy samples of 31 patients with VRL and negative controls with uveitis. In a subset of cases, genome-wide copy number alterations (CNA) were assessed using the Oncoscan platform. Mutations in MYD88 (74%), PIM1 (71%), CD79B (55%), IGLL5 (52%), TBL1XR1 (48%), ETV6 (45%) and 9p21/CDKN2A deletions (85%) were the most common alterations, with similar frequencies in primary (15), synchronous (3) or secondary (13) VRL. This mutational spectrum is similar to MYD88mut/CD79Bmut (MCD or cluster 5) DLBCL with activation of Toll-like and B-cell receptor pathways and CDKN2A loss, confirming their close relationship. Oncoscan analysis demonstrated a high number of CNAs (mean 18.6/case). Negative controls lacked mutations or CNAs. Using cell free DNA of vitreous fluid supernatant, mutations present in cellular DNA were reliably detected in all examined cases. Mutational analysis is a highly sensitive and specific tool for the diagnosis of VRL and can also be applied successfully to cell free DNA derived from the vitreous.Copyright © 2021 American Society of Hematology

Should undergraduate lectures be compulsory? The views of dental and medical students from a UK university

Formal lectures have been a traditional part of medical and dental education, but there is debate as to their compulsory status. This study was designed to explore dental and medical students’ views on compulsory lectures and the use of Video-Recorded Lectures (VRL). A cross-sectional study of University of Bristol students in Years 2 to 4 was conducted using an online questionnaire. The majority of both dental (76%) and medical (66%) students felt lectures should be non-compulsory. The most common learning resources used by both dental and medical students were live lectures, lecture handouts and VRL. The majority of both dental (84%) and medical (88%) students used VRL. Most students attended lectures all of the time both before and after the introduction of VRL, even though most dental and medical students believe lectures should be non-compulsory. VRL is a popular learning resource. These findings tie-in with General Dental Council and General Medical Council recommendations that encourage self-directed learning. Dental and Medical schools should offer a range of learning resources and make use of current technology, including the use of VRL

Efficient Video Representation Learning via Masked Video Modeling with Motion-centric Token Selection

Self-supervised Video Representation Learning (VRL) aims to learn transferrable representations from uncurated, unlabeled video streams that could be utilized for diverse downstream tasks. With recent advances in Masked Image Modeling (MIM), in which the model learns to predict randomly masked regions in the images given only the visible patches, MIM-based VRL methods have emerged and demonstrated their potential by significantly outperforming previous VRL methods. However, they require an excessive amount of computations due to the added temporal dimension. This is because existing MIM-based VRL methods overlook spatial and temporal inequality of information density among the patches in arriving videos by resorting to random masking strategies, thereby wasting computations on predicting uninformative tokens/frames. To tackle these limitations of Masked Video Modeling, we propose a new token selection method that masks our more important tokens according to the object's motions in an online manner, which we refer to as Motion-centric Token Selection. Further, we present a dynamic frame selection strategy that allows the model to focus on informative and causal frames with minimal redundancy. We validate our method over multiple benchmark and Ego4D datasets, showing that the pre-trained model using our proposed method significantly outperforms state-of-the-art VRL methods on downstream tasks, such as action recognition and object state change classification while largely reducing memory requirements during pre-training and fine-tuning.Comment: 15 page

BIOMECHANICAL TRAITS ANALYSIS WHEN PERFORMING OF JUDO UCHIMATA BY POSTURE AND VOLUNTARY RESISTANCE LEVELS OF UKE

The purpose of this study was to analyze the biomechanical traits variables when performing uchimata (inner thigh reaping throw) by voluntary resistance levels (VRL) and two postures of uke (defender, receiver) in Judo. The postures of uke were shizenhontai (straight natural posture:NP) and jigohontai (straight defensive posture:DP), VRL of uke were 0% and 100%, respectively. The biomechanical variables were temporal (total time-required: TR), postures and COG during performing uchimata. It's important for jUdoists to prepare for individual analysis. prescription and countermeasures because they have experienced several variables when performing techniques according to opponent's postures and VRL in biomechanical aspects

Reusable Software Components for Robots Using Fuzzy Abstractions

Mobile robots today, while varying greatly in design, often have a large number of similarities in terms of their tasks and goals. Navigation, obstacle avoidance, and vision are all examples. In turn, robots of similar design, but with varying configurations, should be able to share the bulk of their controlling software. Any changes required should be minimal and ideally only to specify new hardware configurations. However, it is difficult to achieve such flexibility, mainly due to the enormous variety of robot hardware available and the huge number of possible configurations. Monolithic controllers that can handle such variety are impossible to build. This paper will investigate these portability problems, as well as techniques to manage common abstractions for user-designed components. The challenge is in creating new methods for robot software to support a diverse variety of robots, while also being easily upgraded and extended. These methods can then provide new ways to support the operational and functional reuse of the same high-level components across a variety of robots

Influence of microwave fields on the electron transport through a quantum dot in the presence of a direct tunneling between leads

We consider the time-dependent electron transport through a quantum dot coupled to two leads in the presence of the additional over-dot (bridge) tunneling channel. By using the evolution operator method together with the wide-band limit approximation we derived the analytical formulaes for the quantum dot charge and current flowing in the system. The influence of the external microwave field on the time-average quantum dot charge, the current and the derivatives of the average current with respect to the gate and source-drain voltages has been investigated for a wide range of parameters.Comment: 28 Pages, 11 Postscript figure

Optimizing interferometer experiments for CMB B mode measurement

The sensitivity of interferometers with linear polarizers to the CMB E and B mode are variant under the rotation of the polarizer frame, while interferometer with circular polarizers are equally sensitive to E and B mode. We present analytically and numerically that the diagonal elements of window functions for CMB E/B power spectra are maximized in interferometric measurement of linear polarization, when the polarizer frame is in certain rotation from the associated baseline. We also present the simulated observation to show that the 1-$\sigma$ errors on E/B mode power spectrum estimation are variant under the polarizer frame rotation in the case of linear polarizers, while they are invariant in the case of circular polarizers. Simulation of the configuration similar to the DASI shows that minimum 1-$\sigma$ error on B mode in interferometer measurement with linear polarizers is 26% of that in interferometric measurement with circular polarizers. The simulation also shows that the E/B mixing in interferometer measurement with linear polarizers can be as low as 23% of that in interferometric measurement with circular polarizers. It is not always possible to physically align the polarizer frame with all the associated baselines in the case of an interferometer array (N$>$2). There exist certain linear combinations of visibilities, which are equivalent to visibilities of the optimal polarizer frame rotation. We present the linear combinations, which enables B mode optimization for an interferometer array (N$>$2).Comment: v2: file error corrected, v3: a footnote added, v4: a typo corrected v10: some pdf file problem fixe

Screening donors for xenotransplantation: The potential for xenozoonoses

Xenotransplantation is a potential solution to the current donor shortage for solid organ transplantation. The transmission of infectious agents from donor organs or bone marrow to the recipient is a well-recognized phenomenon following allotransplantation. Thus the prospect of xenotransplantation raises the issue of xenozoonoses-i.e., the transmission of animal infections to the human host. Anticipating an increasing number of baboon to human transplants, 31 adult male baboons (Papio cynocephalus) from a single colony in the United States were screened for the presence of antibody to microbial agents (principally viral) that may pose a significant risk of infection. Antibody to simian cytomegalovirus, simian agent 8 and Epstein-Barr virus, was found in 97% of animals tested. Antibody to simian retroviruses and Toxoplasma gondii was found in 30% and 32% respectively. Discordant results were found when paired samples were examined by two primate laboratories. This was particularly noted when methodologies were based on cross-reaction with human viral antigens. These results highlight the need to develop specific antibody tests against the species used for xenotransplantation. © 1994 Williams & Wilkins