Avoiding Wireheading with Value Reinforcement Learning

How can we design good goals for arbitrarily intelligent agents? Reinforcement learning (RL) is a natural approach. Unfortunately, RL does not work well for generally intelligent agents, as RL agents are incentivised to shortcut the reward sensor for maximum reward -- the so-called wireheading problem. In this paper we suggest an alternative to RL called value reinforcement learning (VRL). In VRL, agents use the reward signal to learn a utility function. The VRL setup allows us to remove the incentive to wirehead by placing a constraint on the agent's actions. The constraint is defined in terms of the agent's belief distributions, and does not require an explicit specification of which actions constitute wireheading.Comment: Artificial General Intelligence (AGI) 201

Self-reflection and academic performance: is there a relationship?

The purposes of the present study were two-fold: first, to evaluate whether reflection journal writing was effective in promoting self-reflection and learning, and whether students become better at self-reflection if they engage continuously in reflection journal writing. To that end, the reflection journals of 690 first-year applied science students at a local polytechnic were studied by means of an automated coding procedures using software. Data was collected twice, once at the beginning and again towards the end of an academic year. Outcomes of the textual content analyses revealed that students reflected on both the process and contents of their learning: critical review of past learning experiences, learning strategies and summaries of what was learned. Correlational analyses showed weak to moderate inter-relationship

The impact of elbow and knee joint lesions on abnormal gait and posture of sows

<p>Abstract</p> <p>Background</p> <p>Joint lesions occur widespread in the Danish sow population and they are the most frequent cause for euthanasia. Clinically, it is generally impossible to differentiate between various types of non-inflammatory joint lesions. Consequently, it is often necessary to perform a post mortem examination in order to diagnose these lesions. A study was performed in order to examine the relation of abnormal gait and posture in sows with specific joint lesions, and thereby obtaining a clinical diagnostic tool, to be used by farmers and veterinarians for the evaluation of sows with joint problems.</p> <p>Methods</p> <p>The gait, posture and lesions in elbow- and knee joints of 60 randomly selected sows from one herd were scored clinically and pathologically. Associations between the scorings were estimated.</p> <p>Results</p> <p>The variables 'fore- and hind legs turned out' and 'stiff in front and rear' were associated with lesions in the elbow joint, and the variables 'hind legs turned out' and 'stiff in rear' were associated with lesions in the knee joint.</p> <p>Conclusion</p> <p>It was shown that specified gait and posture variables reflected certain joint lesions. However, further studies are needed to strengthen and optimize the diagnostic tool.</p

The influence of farmers' mental models on an agroforestry extension program in the Philippines

The influence of farmers' mental models on the success of an agroforestry extension program on Leyte Island in the Philippines was investigated. Knowledge of farmers' mental models and hence the likely acceptance of technology was used to inform the design of a hypothetically expanded program. To gain an insight into the reasons behind differing acceptance of extension assistance, data were collected and analysed from formal interviews, translated conversations and visual observations. The data provided a chain of evidence and triangulation between farmers' stated intentions and their actions. Farmers had little prior knowledge of nursery technology and were highly receptive to extension assistance which enabled them to develop high self-efficacy in seedling production. However, farmers' rejection of silvicultural advice to thin and prune existing plantations was predicated by existing attitudes to forest resource management. Farmers also expressed a strong preference for a low-cost and low-input approach to establishing timber trees. Visual observations of farmers' tree establishment practices indicated the existence of gaps in their knowledge of tree growth processes. This investigation illustrates the need to elicit farmers' mental models as a parallel enquiry to extension activities. If agroforestry extension is to be constructivist and participatory, accommodation of farmers' mental models and modification of program goals may be necessary. Relatively little is known about the reasons for farmers' acceptance or rejection of silviculture in Leyte and these results indicate that further research into the way that farmers' mental models filter and guide acceptance of advice may be worthwhile

Effectiveness of physiotherapy exercise following hip arthroplasty for osteoarthritis: a systematic review of clinical trials

Background: Physiotherapy has long been a routine component of patient rehabilitation following hip joint replacement. The purpose of this systematic review was to evaluate the effectiveness of physiotherapy exercise after discharge from hospital on function, walking, range of motion, quality of life and muscle strength, for osteoarthritic patients following elective primary total hip arthroplasty. Methods: Design: Systematic review, using the Cochrane Collaboration Handbook for Systematic Reviews of Interventions and the Quorom Statement. Database searches: AMED, CINAHL, EMBASE, KingsFund, MEDLINE, Cochrane library (Cochrane reviews, Cochrane Central Register of Controlled Trials, DARE), PEDro, The Department of Health National Research Register. Handsearches: Physiotherapy, Physical Therapy, Journal of Bone and Joint Surgery (Britain) Conference Proceedings. No language restrictions were applied. Selection: Trials comparing physiotherapy exercise versus usual/standard care, or comparing two types of relevant exercise physiotherapy, following discharge from hospital after elective primary total hip replacement for osteoarthritis were reviewed. Outcomes: Functional activities of daily living, walking, quality of life, muscle strength and range of hip joint motion. Trial quality was extensively evaluated. Narrative synthesis plus meta-analytic summaries were performed to summarise the data. Results: 8 trials were identified. Trial quality was mixed. Generally poor trial quality, quantity and diversity prevented explanatory meta-analyses. The results were synthesised and meta-analytic summaries were used where possible to provide a formal summary of results. Results indicate that physiotherapy exercise after discharge following total hip replacement has the potential to benefit patients. Conclusion: Insufficient evidence exists to establish the effectiveness of physiotherapy exercise following primary hip replacement for osteoarthritis. Further well designed trials are required to determine the value of post discharge exercise following this increasingly common surgical procedure

Determinants of rapid weight gain during infancy: baseline results from the NOURISH randomised controlled trial

Background: Rapid weight gain in infancy is an important predictor of obesity in later childhood. Our aim was to determine which modifiable variables are associated with rapid weight gain in early life.Methods: Subjects were healthy infants enrolled in NOURISH, a randomised, controlled trial evaluating an intervention to promote positive early feeding practices. This analysis used the birth and baseline data for NOURISH. Birthweight was collected from hospital records and infants were also weighed at baseline assessment when they were aged 4-7 months and before randomisation. Infant feeding practices and demographic variables were collected from the mother using a self administered questionnaire. Rapid weight gain was defined as an increase in weight-for-age Z-score (using WHO standards) above 0.67 SD from birth to baseline assessment, which is interpreted clinically as crossing centile lines on a growth chart. Variables associated with rapid weight gain were evaluated using a multivariable logistic regression model.Results: Complete data were available for 612 infants (88% of the total sample recruited) with a mean (SD) age of 4.3 (1.0) months at baseline assessment. After adjusting for mother's age, smoking in pregnancy, BMI, and education and infant birthweight, age, gender and introduction of solid foods, the only two modifiable factors associated with rapid weight gain to attain statistical significance were formula feeding [OR = 1.72 (95%CI 1.01-2.94), P = 0.047] and feeding on schedule [OR = 2.29 (95%CI 1.14-4.61), P = 0.020]. Male gender and lower birthweight were non-modifiable factors associated with rapid weight gain.Conclusions: This analysis supports the contention that there is an association between formula feeding, feeding to schedule and weight gain in the first months of life. Mechanisms may include the actual content of formula milk (e.g. higher protein intake) or differences in feeding styles, such as feeding to schedule, which increase the risk of overfeeding.Trial Registration: Australian Clinical Trials Registry ACTRN12608000056392

International Veterinary Epilepsy Task Force Consensus Proposal: Outcome of therapeutic interventions in canine and feline epilepsy

Common criteria for the diagnosis of drug resistance and the assessment of outcome are needed urgently as a prerequisite for standardized evaluation and reporting of individual therapeutic responses in canine epilepsy. Thus, we provide a proposal for the definition of drug resistance and partial therapeutic success in canine patients with epilepsy. This consensus statement also suggests a list of factors and aspects of outcome, which should be considered in addition to the impact on seizures. Moreover, these expert recommendations discuss criteria which determine the validity and informative value of a therapeutic trial in an individual patient and also suggest the application of individual outcome criteria. Agreement on common guidelines does not only render a basis for future optimization of individual patient management, but is also a presupposition for the design and implementation of clinical studies with highly standardized inclusion and exclusion criteria. Respective standardization will improve the comparability of findings from different studies and renders an improved basis for multicenter studies. Therefore, this proposal provides an in-depth discussion of the implications of outcome criteria for clinical studies. In particular ethical aspects and the different options for study design and application of individual patient-centered outcome criteria are considered

From affect programs to dynamical discrete emotions

According to Discrete Emotion Theory, a number of emotions are distinguishable on the basis of neural, physiological, behavioral and expressive features. Critics of this view emphasize the variability and context-sensitivity of emotions. This paper discusses some of these criticisms, and argues that they do not undermine the claim that emotions are discrete. This paper also presents some works in dynamical affective science, and argues that to conceive of discrete emotions as self-organizing and softly assembled patterns of various processes accounts more naturally than traditional Discrete Emotion Theory for the variability and context-sensitivity of emotions

Random Amino Acid Mutations and Protein Misfolding Lead to Shannon Limit in Sequence-Structure Communication

The transmission of genomic information from coding sequence to protein structure during protein synthesis is subject to stochastic errors. To analyze transmission limits in the presence of spurious errors, Shannon's noisy channel theorem is applied to a communication channel between amino acid sequences and their structures established from a large-scale statistical analysis of protein atomic coordinates. While Shannon's theorem confirms that in close to native conformations information is transmitted with limited error probability, additional random errors in sequence (amino acid substitutions) and in structure (structural defects) trigger a decrease in communication capacity toward a Shannon limit at 0.010 bits per amino acid symbol at which communication breaks down. In several controls, simulated error rates above a critical threshold and models of unfolded structures always produce capacities below this limiting value. Thus an essential biological system can be realistically modeled as a digital communication channel that is (a) sensitive to random errors and (b) restricted by a Shannon error limit. This forms a novel basis for predictions consistent with observed rates of defective ribosomal products during protein synthesis, and with the estimated excess of mutual information in protein contact potentials