Continual Learning in the Teacher-Student Setup: Impact of Task Similarity

Continual learning—the ability to learn many tasks in sequence—is critical for artificial learning systems. Yet standard training methods for deep networks often suffer from catastrophic forgetting, where learning new tasks erases knowledge of the earlier tasks. While catastrophic forgetting labels the problem, the theoretical reasons for interference between tasks remain unclear. Here, we attempt to narrow this gap between theory and practice by studying continual learning in the teacher-student setup. We extend previous analytical work on two-layer networks in the teacher-student setup to multiple teachers. Using each teacher to represent a different task, we investigate how the relationship between teachers affects the amount of forgetting and transfer exhibited by the student when the task switches. In line with recent work, we find that when tasks depend on similar features, intermediate task similarity leads to greatest forgetting. However, feature similarity is only one way in which tasks may be related. The teacher-student approach allows us to disentangle task similarity at the level of \textbackslashemphreadouts (hidden-to-output weights) as well as \textbackslashemphfeatures (input-to-hidden weights). We find a complex interplay between both types of similarity, initial transfer/forgetting rates, maximum transfer/forgetting, and the long-time (post-switch) amount of transfer/forgetting. Together, these results help illuminate the diverse factors contributing to catastrophic forgetting

Controlling the accuracy of the density matrix renormalization group method: The Dynamical Block State Selection approach

We have applied the momentum space version of the Density Matrix Renormalization Group method ($k$-DMRG) in quantum chemistry in order to study the accuracy of the algorithm in the new context. We have shown numerically that it is possible to determine the desired accuracy of the method in advance of the calculations by dynamically controlling the truncation error and the number of block states using a novel protocol which we dubbed Dynamical Block State Selection (DBSS). The relationship between the real error and truncation error has been studied as a function of the number of orbitals and the fraction of filled orbitals. We have calculated the ground state of the molecules CH$_2$, H$_2$O, and F$_2$ as well as the first excited state of CH$_2$. Our largest calculations were carried out with 57 orbitals, the largest number of block states was 1500--2000, and the largest dimensions of the Hilbert space of the superblock configuration was 800.000--1.200.000.Comment: 12 page

Loved Ones Near and Far: Feinberg's Personal Significance Theory

This paper examines Todd Feinberg's theory of the misidentification syndromes

Diet and risk for breast cancer recurrence and survival

Dietary factors may influence the risk for breast cancer and also the prognosis following diagnosis and treatment. The aim of this study was to assess whether self-reported prediagnosis diet or other patient factors associated with breast cancer incidence were predictive of recurrence and survival. Patients (n=149) diagnosed with primary breast cancer between 1989 and 1991 were followed for five or more years. Total energy (hazard ratio (HR)=1.58, 95%, confidence interval (CI)= 1.05, 2.38) as well as total (HR = 1.46, 95% CI = 1.05, 2.01), saturated (HR = 1.79,95% CI = 1.05, 3.04), and monounsaturated (HR = 1.65, 95% CI = 1.09,2.49) fat intakes were associated with increased risk, and energy-adjusted bread and cereal consumption (HR = 0.55, 95% CI = 0.33, 0.93) with decreased risk of recurrence. Both total energy (HR = 1.58, 95% CI = 1.03, 2.43) and polyunsaturated fat (HR = 1.84, 95% CI = 1.09, 3.13) intakes were associated with an increased risk of death. All associations between dietary fat and recurrence and survival attenuated following energy adjustment. Oral contraceptive use (HR = 1.28, 95% CI = 1.03, 1.60), lymph node positive status (HR = 2.36, 95% CI = 1.01, 5.49), and tumor stage (HR = 2.22, 95% CI = 1.02, 4.81) were associated with increased risk of recurrence. Tumor stage (HR = 4.96, 95% CI = 1.86, 13.23), lymph node positive status (HR = 3.31, 95% CI = 1.38, 7.95, and estrogen receptor negative status (HR = 2.46, 95% CI = 1.02, 5.94) were associated with increased risk, and arm muscle circumference (HR = 0.27, 95% CI = 0.09, 0.86) and mammographic utilization (HR = 0.77, 95% CI = 0.61, 0.98) with decreased risk of death. Higher levels of energy, fat intakes, and selected patient characteristics (particularly disease stage and anthropometric indicators of adiposity) appear to increase risk of recurrence and/or shortened survival following the diagnosis of breast cancer.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/44203/1/10549_2004_Article_194284.pd

Mind the Gap: Investigating Toddlers’ Sensitivity to Contact Relations in Predictive Events

Toddlers readily learn predictive relations between events (e.g., that event A predicts event B). However, they intervene on A to try to cause B only in a few contexts: When a dispositional agent initiates the event or when the event is described with causal language. The current studies look at whether toddlers’ failures are due merely to the difficulty of initiating interventions or to more general constraints on the kinds of events they represent as causal. Toddlers saw a block slide towards a base, but an occluder prevented them from seeing whether the block contacted the base; after the block disappeared behind the occluder, a toy connected to the base did or did not activate. We hypothesized that if toddlers construed the events as causal, they would be sensitive to the contact relations between the participants in the predictive event. In Experiment 1, the block either moved spontaneously (no dispositional agent) or emerged already in motion (a dispositional agent was potentially present). Toddlers were sensitive to the contact relations only when a dispositional agent was potentially present. Experiment 2 confirmed that toddlers inferred a hidden agent was present when the block emerged in motion. In Experiment 3, the block moved spontaneously, but the events were described either with non-causal (“here’s my block”) or causal (“the block can make it go”) language. Toddlers were sensitive to the contact relations only when given causal language. These findings suggest that dispositional agency and causal language facilitate toddlers’ ability to represent causal relationships.John Templeton Foundation (#12667)James S. McDonnell Foundation (Causal Learning Collaborative Initiative)National Science Foundation (U.S.) (Career Award (# 0744213

Parent and child agreement for acute stress disorder, post-traumatic stress disorder and other psychopathology in a prospective study of children and adolescents exposed to single-event trauma

Examining parent-child agreement for Acute Stress Disorder (ASD) and Post-Traumatic Stress Disorder (PTSD) in children and adolescents is essential for informing the assessment of trauma-exposed children, yet no studies have examined this relationship using appropriate statistical techniques. Parent-child agreement for these disorders was examined by structured interview in a prospective study of assault and motor vehicle accident (MVA) child survivors, assessed at 2-4 weeks and 6 months post-trauma. Children were significantly more likely to meet criteria for ASD, as well as other ASD and PTSD symptom clusters, based on their own report than on their parent's report. Parent-child agreement for ASD was poor (Cohen's κ = -.04), but fair for PTSD (Cohen's κ = .21). Agreement ranged widely for other emotional disorders (Cohen's κ = -.07-.64), with generalised anxiety disorder found to have superior parent-child agreement (when assessed by phi coefficients) relative to ASD and PTSD. The findings support the need to directly interview children and adolescents, particularly for the early screening of posttraumatic stress, and suggest that other anxiety disorders may have a clearer presentation post-trauma

Safeguarding children in dentistry: 1. Child protection training, experience and practice of dental professionals with an interest in paediatric dentistry

* Few dental professionals with child protection training have experience of making referrals. * There is a wide gap in practice between recognising signs of child abuse and neglect and responding effectively. * This may indicate missed opportunities to save children from continuing abuse. * There is a need for improved child protection information, support and training for dental professionals. Abstract Following several highly publicised inquiries into the deaths of children from abuse and neglect, there has been much recent interest in the role and responsibility of all health professionals to protect children at risk of maltreatment. The findings of a postal questionnaire, sent in March 2005 to 789 dentists and dental care professionals with an interest in paediatric dentistry working in varied settings in the UK, are presented in a two-part report and discussed in the context of current multi-agency good practice in safeguarding and promoting the welfare of children. This first part explores reported child protection training, experience and practice. There was a significant gap between recognising signs of abuse and responding effectively: 67% of respondents had suspected abuse or neglect of a child patient at some time in their career but only 29% had ever made a child protection referral. The dental profession is alerted to the need to ensure necessary appropriate action to safeguard children is always taken when child abuse or neglect are suspected

Follow-up after paediatric intensive care treatment: parental postraumatic stress

Aim: To study the prevalence of posttraumatic stress in parents after an acute admission to a paediatric intensive care unit (PICU) and to determine risk factors for the development of posttraumatic stress. Methods: Parents completed posttraumatic stress questionnaires three months after their child's discharge. This questionnaire measures both symptoms of posttraumatic stress disorder (PTSD) and enables determination of the full psychiatric diagnosis of PTSD. Medical and demographic data concerning their child were gathered from physical evaluations three months after discharge. Of 250 eligible families, 144 (57.6%) participated in this study. The questionnaires were completed by 140 mothers and 107 fathers. Results: More than three-quarters of the parents experienced persistent symptoms of PTSD. In 21 mothers (15.0%) and 10 fathers (9.3%), the full psychiatric diagnosis of PTSD was determined. In six families, both parents had PTSD. Furthermore, a significant positive correlation was found between symptoms of PTSD of the mothers and the fathers. No obvious medical risk factors could be distinguished. Conclusion: The unexpected admission of a child to a PICU is a stressful event associated with parental posttraumatic stress. Treatment should not end after discharge. Follow-up care is warranted and research should be focused on prevention of these symptoms. © 2007 The Author(s)

Arbitrary Choice of Basic Variables in Density Functional Theory. II. Illustrative Applications

Our recent theory (Ref. 1) enables us to choose arbitrary quantities as the basic variables of the density functional theory. In this paper we apply it to several cases. In the case where the occupation matrix of localized orbitals is chosen as a basic variable, we can obtain the single-particle equation which is equivalent to that of the LDA+U method. The theory also leads to the Hartree-Fock-Kohn-Sham equation by letting the exchange energy be a basic variable. Furthermore, if the quantity associated with the density of states near the Fermi level is chosen as a basic variable, the resulting single-particle equation includes the additional potential which could mainly modify the energy-band structures near the Fermi level.Comment: 27 page