SuperNeurons: Dynamic GPU Memory Management for Training Deep Neural Networks

Going deeper and wider in neural architectures improves the accuracy, while the limited GPU DRAM places an undesired restriction on the network design domain. Deep Learning (DL) practitioners either need change to less desired network architectures, or nontrivially dissect a network across multiGPUs. These distract DL practitioners from concentrating on their original machine learning tasks. We present SuperNeurons: a dynamic GPU memory scheduling runtime to enable the network training far beyond the GPU DRAM capacity. SuperNeurons features 3 memory optimizations, \textit{Liveness Analysis}, \textit{Unified Tensor Pool}, and \textit{Cost-Aware Recomputation}, all together they effectively reduce the network-wide peak memory usage down to the maximal memory usage among layers. We also address the performance issues in those memory saving techniques. Given the limited GPU DRAM, SuperNeurons not only provisions the necessary memory for the training, but also dynamically allocates the memory for convolution workspaces to achieve the high performance. Evaluations against Caffe, Torch, MXNet and TensorFlow have demonstrated that SuperNeurons trains at least 3.2432 deeper network than current ones with the leading performance. Particularly, SuperNeurons can train ResNet2500 that has $10^4$ basic network layers on a 12GB K40c.Comment: PPoPP '2018: 23nd ACM SIGPLAN Symposium on Principles and Practice of Parallel Programmin

Learning Opinion Dynamics From Social Traces

Opinion dynamics - the research field dealing with how people's opinions form and evolve in a social context - traditionally uses agent-based models to validate the implications of sociological theories. These models encode the causal mechanism that drives the opinion formation process, and have the advantage of being easy to interpret. However, as they do not exploit the availability of data, their predictive power is limited. Moreover, parameter calibration and model selection are manual and difficult tasks. In this work we propose an inference mechanism for fitting a generative, agent-like model of opinion dynamics to real-world social traces. Given a set of observables (e.g., actions and interactions between agents), our model can recover the most-likely latent opinion trajectories that are compatible with the assumptions about the process dynamics. This type of model retains the benefits of agent-based ones (i.e., causal interpretation), while adding the ability to perform model selection and hypothesis testing on real data. We showcase our proposal by translating a classical agent-based model of opinion dynamics into its generative counterpart. We then design an inference algorithm based on online expectation maximization to learn the latent parameters of the model. Such algorithm can recover the latent opinion trajectories from traces generated by the classical agent-based model. In addition, it can identify the most likely set of macro parameters used to generate a data trace, thus allowing testing of sociological hypotheses. Finally, we apply our model to real-world data from Reddit to explore the long-standing question about the impact of backfire effect. Our results suggest a low prominence of the effect in Reddit's political conversation.Comment: Published at KDD202

Antimicrobial resistance characteristics and fitness of Gram-negative fecal bacteria from volunteers treated with minocycline or amoxicillin.

A yearlong study was performed to examine the effect of antibiotic administration on the bacterial gut flora. Gram-negative facultative anaerobic bacteria were recovered from the feces of healthy adult volunteers administered amoxicillin, minocycline or placebo, and changes determined in antimicrobial resistance (AMR) gene carriage. Seventy percent of the 1039 facultative anaerobic isolates recovered were identified by MALDI-TOF as Escherichia coli. A microarray used to determine virulence and resistance gene carriage demonstrated that AMR genes were widespread in all administration groups, with the most common resistance genes being bla TEM, dfr, strB, tet(A), and tet(B). Following amoxicillin administration, an increase in the proportion of amoxicillin resistant E. coli and a three-fold increase in the levels of bla TEM gene carriage was observed, an effect not observed in the other two treatment groups. Detection of virulence genes, including stx1A, indicated not all E. coli were innocuous commensals. Approximately 150 E. coli collected from 6 participants were selected for pulse field gel electrophoresis (PFGE), and a subset used for characterisation of plasmids and Phenotypic Microarrays (PM). PFGE indicated some E. coli clones had persisted in volunteers for up to 1 year, while others were transient. Although there were no unique characteristics associated with plasmids from persistent or transient isolates, PM assays showed transient isolates had greater adaptability to a range of antiseptic biocides and tetracycline; characteristics which were lost in some, but not all persistent isolates. This study indicates healthy individuals carry bacteria harboring resistance to a variety of antibiotics and biocides in their intestinal tract. Antibiotic administration can have a temporary effect of selecting bacteria, showing co-resistance to multiple antibiotics, some of which can persist within the gut for up to 1 year

New treatments for breast cancer: Breakthroughs for patient care or just steps in the right direction?

Three areas of clinical research in breast cancer treatment led to news breaking presentations at the American Society of Clinical Oncology (ASCO) meeting, 1998, in Los Angeles. All three subjects represent important advances in cancer medicine. Prevention: Two related drugs, tamoxifen and raloxifene, were found in placebo controlled trials to significantly reduce the incidence of breast cancer for women at increased risk of developing the disease. Patterns of relapse showed that the reduced rate of breast cancer was exclusively observed for tumors expressing estrogen receptors, while the rate of tumors classified as estrogen-receptor negative was similar for the treatment and the control groups. This may indicate that the observed reduction in breast cancer incidence is due to a treatment effect on occult disease rather than its prevention. We certainly have no adequate information on mortality prevention. Adjuvant therapies: Taxol given every three weeks for four courses following an adjuvant treatment with four courses of doxorubicin and cyclophosphamide (AC) combination was found to be superior to not adding treatment after the four courses of AC in a trial involving 3170 patients. At 22 months of median follow-up, the quoted P-values were P = 0.0077 for disease-free survival and P = 0.039 for overall survival, but these did not cross the prospectively defined interim analysis boundaries for statistical significance at the 0.05 level. The difference was observed early during follow-up, and was exclusively seen in the 40% of patients who had ER-negative primaries and, therefore, did not receive tamoxifen following chemotherapy. One may thus argue that the early difference observed was primarily due to differences in the duration of the treatment regimens in the two groups and the early entry into the trial of patients with particularly aggressive neoplasia (e.g., ER-negative primaries) who would have benefited from a longer duration treatment. Treatment of advanced disease: The use of monoclonal antibodies to c-erb-B2 was found to induce responses in metastatic breast cancer. Patients with tumors expressing c-erb-B2 responded to weekly infusions of this biological agent. It was particularly impressive that the response rate for patients receiving infusion of the monoclonal antibodies together with the cytotoxics was superior to that with chemotherapy alone in a randomized trial. It is important to note that only patients with tumors overexpressing c-erbB-2 (the overall incidence is about 20%) were tested. It must still be demonstrated that the effect of these monoclonal antibodies is indeed confined to cells overexpressing c-erbB-2. Treatment related cardiac tox-icity remains a problem, and the effects of treatment in various subsets of patients need to be defined before starting investigations in the adjuvant setting, which is a clear further objective of this specific research. The significant findings from clinical research opened several new questions, which must be answered before allowing them to be employed in routine patient car

Tubal ligation and risk of breast cancer

Although it has been demonstrated in previous studies that tubal ligation can have widespread effects on ovarian function, including a decrease in the risk of subsequent ovarian cancer, few studies have evaluated effects on breast cancer risk. In a population-based case–control study of breast cancer among women 20–54 years of age conducted in three geographic areas, previous tubal ligations were reported by 25.3% of the 2173 cases and 25.8% of the 1990 controls. Initially it appeared that tubal ligations might impart a slight reduction in risk, particularly among women undergoing the procedure at young ages (< 25 years). However, women were more likely to have had the procedure if they were black, less educated, young when they bore their first child, or multiparous. After accounting for these factors, tubal ligations were unrelated to breast cancer risk (relative risk (RR) = 1.09, 95% confidence interval (CI) 0.9–1.3), with no variation in risk by age at, interval since, or calendar year of the procedure. The relationship of tubal ligations to risk did not vary according to the presence of a number of other risk factors, including menopausal status or screening history. Furthermore, effects of tubal ligation were similar for all stages at breast cancer diagnosis. Further studies would be worthwhile given the biologic plausibility of an association. However, future investigations should include information on type of procedure performed (since this may relate to biologic effects) as well as other breast cancer risk factors. © 2000 Cancer Research Campaig

Rational Moral Ignorance

Philosophy and Phenomenological Research, EarlyView

Martian sub-surface ionising radiation: biosignatures and geology

The surface of Mars, unshielded by thick atmosphere or global magnetic field, is exposed to high levels of cosmic radiation. This ionising radiation field is deleterious to the survival of dormant cells or spores and the persistence of molecular biomarkers in the subsurface, and so its characterisation is of prime astrobiological interest. Here, we present modelling results of the absorbed radiation dose as a function of depth through the Martian subsurface, suitable for calculation of biomarker persistence. A second major implementation of this dose accumulation rate data is in application of the optically stimulated luminescence technique for dating Martian sediments. &lt;br&gt;&lt;br&gt; We present calculations of the dose-depth profile in the Martian subsurface for various scenarios: variations of surface composition (dry regolith, ice, layered permafrost), solar minimum and maximum conditions, locations of different elevation (Olympus Mons, Hellas basin, datum altitude), and increasing atmospheric thickness over geological history. We also model the changing composition of the subsurface radiation field with depth compared between Martian locations with different shielding material, determine the relative dose contributions from primaries of different energies, and discuss particle deflection by the crustal magnetic fields

Multilingual gendered identities: female undergraduate students in London talk about heritage languages

In this paper I explore how a group of female university students, mostly British Asian and in their late teens and early twenties, perform femininities in talk about heritage languages. I argue that analysis of this talk reveals ways in which the participants enact ‘culturally intelligible’ gendered subject positions. This frequently involves negotiating the norms of ‘heteronormativity’, constituting femininity in terms of marriage, motherhood and maintenance of heritage culture and language, and ‘girl power’, constituting femininity in terms of youth, sassiness, glamour and individualism. For these young women, I ask whether higher education can become a site in which they have the opportunities to explore these identifications and examine other ways of imagining the self and what their stories suggest about ‘doing being’ a young British Asian woman in London

On the shopfloor: exploring the impact of teacher trade unions on school-based industrial relations

Teachers are highly unionised workers and their trade unions exert an important influence on the shaping and implementation of educational policy. Despite this importance there is relatively little analysis of the impact of teacher trade unions in educational management literature. Very little empirical research has sought to establish the impact of teacher unions at school level. In an era of devolved management and quasi-markets this omission is significant. New personnel issues continue to emerge at school level and this may well generate increased trade union activity at the workplace. This article explores the extent to which devolved management is drawing school-based union representation into a more prominent role. It argues that whilst there can be significant differences between individual schools, increased school autonomy is raising the profile of trade union activity in the workplace, and this needs to be better reflected in educational management research