Scalable and Sustainable Deep Learning via Randomized Hashing

Current deep learning architectures are growing larger in order to learn from complex datasets. These architectures require giant matrix multiplication operations to train millions of parameters. Conversely, there is another growing trend to bring deep learning to low-power, embedded devices. The matrix operations, associated with both training and testing of deep networks, are very expensive from a computational and energy standpoint. We present a novel hashing based technique to drastically reduce the amount of computation needed to train and test deep networks. Our approach combines recent ideas from adaptive dropouts and randomized hashing for maximum inner product search to select the nodes with the highest activation efficiently. Our new algorithm for deep learning reduces the overall computational cost of forward and back-propagation by operating on significantly fewer (sparse) nodes. As a consequence, our algorithm uses only 5% of the total multiplications, while keeping on average within 1% of the accuracy of the original model. A unique property of the proposed hashing based back-propagation is that the updates are always sparse. Due to the sparse gradient updates, our algorithm is ideally suited for asynchronous and parallel training leading to near linear speedup with increasing number of cores. We demonstrate the scalability and sustainability (energy efficiency) of our proposed algorithm via rigorous experimental evaluations on several real datasets

A New Method of Measuring 81Kr and 85Kr Abundances in Environmental Samples

We demonstrate a new method for determining the 81Kr/Kr ratio in environmental samples based upon two measurements: the 85Kr/81Kr ratio measured by Atom Trap Trace Analysis (ATTA) and the 85Kr/Kr ratio measured by Low-Level Counting (LLC). This method can be used to determine the mean residence time of groundwater in the range of 10^5 - 10^6 a. It requires a sample of 100 micro-l STP of Kr extracted from approximately two tons of water. With modern atmospheric Kr samples, we demonstrate that the ratios measured by ATTA and LLC are directly proportional to each other within the measurement error of +/- 10%; we calibrate the 81Kr/Kr ratio of modern air measured using this method; and we show that the 81Kr/Kr ratios of samples extracted from air before and after the development of the nuclear industry are identical within the measurement error

Discovery of underground argon with low level of radioactive 39Ar and possible applications to WIMP dark matter detectors

We report on the first measurement of 39Ar in argon from underground natural gas reservoirs. The gas stored in the US National Helium Reserve was found to contain a low level of 39Ar. The ratio of 39Ar to stable argon was found to be <=4x10-17 (84% C.L.), less than 5% the value in atmospheric argon (39Ar/Ar=8x10-16). The total quantity of argon currently stored in the National Helium Reserve is estimated at 1000 tons. 39Ar represents one of the most important backgrounds in argon detectors for WIMP dark matter searches. The findings reported demonstrate the possibility of constructing large multi-ton argon detectors with low radioactivity suitable for WIMP dark matter searches.Comment: 6 pages, 2 figures, 2 table

Cell-Autonomous Requirement for Rx Function in the Mammalian Retina and Posterior Pituitary

Rx is a paired-like homeobox gene that is required for vertebrate eye formation. Mice lacking Rx function do not develop eyes or the posterior pituitary. To determine whether Rx is required cell autonomously in these tissues, we generated embryonic chimeras consisting of wild type and Rx−/− cells. We found that in the eye, Rx-deficient cells cannot participate in the formation of the neuroretina, retina pigment epithelium and the distal part of the optic stalk. In addition, in the ventral forebrain, Rx function is required cell autonomously for the formation of the posterior pituitary. Interestingly, Rx−/− and wild type cells segregate before the morphogenesis of these two tissues begins. Our observations suggest that Rx function is not only required for the morphogenesis of the retina and posterior pituitary, but also prior to morphogenesis, for the sorting out of cells to form distinct fields of retinal/pituitary cells

The Teleost Retina as a Model for Developmental and Regeneration Biology

Retinal development in teleosts can broadly be divided into three epochs. The first is the specification of cellular domains in the larval forebrain that give rise to the retinal primordia and undergo early morphogenetic movements. The second is the neurogenic events within the retina proper—proliferation, cell fate determination, and pattern formation—that establish neuronal identities and form retinal laminae and cellular mosaics. The third, which is unique to teleosts and occurs in the functioning eye, is stretching of the retina and persistent neurogenesis that allows the growth of the retina to keep pace with the growth of the eye and other tissues. The first two events are rapid, complete by about 3 days postfertilization in the zebrafish embryo. The third is life-long and accounts for the bulk of retinal growth and the vast majority of adult retinal neurons. In addition, but clearly related to the retina's developmental history, lesions that kill retinal neurons elicit robust neuronal regeneration that originates from cells intrinsic to the retina. This paper reviews recent studies of retinal development in teleosts, focusing on those that shed light on the genetic and molecular regulation of retinal specification and morphogenesis in the embryo, retinal neurogenesis in larvae and adults, and injury-induced neuronal regeneration.Peer Reviewedhttp://deepblue.lib.umich.edu/bitstream/2027.42/63365/1/zeb.2004.1.257.pd

Influenza Virus in Human Exhaled Breath: An Observational Study

Background: Recent studies suggest that humans exhale fine particles during tidal breathing but little is known of their composition, particularly during infection. Methodology/Principal Findings: We conducted a study of influenza infected patients to characterize influenza virus and particle concentrations in their exhaled breath. Patients presenting with influenza-like-illness, confirmed influenza A or B virus by rapid test, and onset within 3 days were recruited at three clinics in Hong Kong, China. We collected exhaled breath from each subject onto Teflon filters and measured exhaled particle concentrations using an optical particle counter. Filters were analyzed for influenza A and B viruses by quantitative polymerase chain reaction (qPCR). Twelve out of thirteen rapid test positive patients provided exhaled breath filter samples (7 subjects infected with influenza B virus and 5 subjects infected with influenza A virus). We detected influenza virus RNA in the exhaled breath of 4 (33%) subjects-three (60%) of the five patients infected with influenza A virus and one (14%) of the seven infected with influenza B virus. Exhaled influenza virus RNA generation rates ranged from <3.2 to 20 influenza virus RNA particles per minute. Over 87% of particles exhaled were under 1 μm in diameter. Conclusions: These findings regarding influenza virus RNA suggest that influenza virus may be contained in fine particles generated during tidal breathing, and add to the body of literature suggesting that fine particle aerosols may play a role in influenza transmission. © 2008 Fabian et al.published_or_final_versio

Failure of Working Memory Training to Enhance Cognition or Intelligence

Fluid intelligence is important for successful functioning in the modern world, but much evidence suggests that fluid intelligence is largely immutable after childhood. Recently, however, researchers have reported gains in fluid intelligence after multiple sessions of adaptive working memory training in adults. The current study attempted to replicate and expand those results by administering a broad assessment of cognitive abilities and personality traits to young adults who underwent 20 sessions of an adaptive dual n-back working memory training program and comparing their post-training performance on those tests to a matched set of young adults who underwent 20 sessions of an adaptive attentional tracking program. Pre- and post-training measurements of fluid intelligence, standardized intelligence tests, speed of processing, reading skills, and other tests of working memory were assessed. Both training groups exhibited substantial and specific improvements on the trained tasks that persisted for at least 6 months post-training, but no transfer of improvement was observed to any of the non-trained measurements when compared to a third untrained group serving as a passive control. These findings fail to support the idea that adaptive working memory training in healthy young adults enhances working memory capacity in non-trained tasks, fluid intelligence, or other measures of cognitive abilities.National Institutes of Health (U.S.) (Blueprint for Neuroscience Research (T90DA022759/R90DA023427)United States. Defense Advanced Research Projects Agency (government contract no. NBCHC070105)United States. Dept. of Defense (National Defense Science and Engineering Fellowship)Massachusetts Institute of Technology (Sheldon Razin (1959) Fellowship

B1 SOX Coordinate Cell Specification with Patterning and Morphogenesis in the Early Zebrafish Embryo

The B1 SOX transcription factors SOX1/2/3/19 have been implicated in various processes of early embryogenesis. However, their regulatory functions in stages from the blastula to early neurula remain largely unknown, primarily because loss-of-function studies have not been informative to date. In our present study, we systematically knocked down the B1 sox genes in zebrafish. Only the quadruple knockdown of the four B1 sox genes sox2/3/19a/19b resulted in very severe developmental abnormalities, confirming that the B1 sox genes are functionally redundant. We characterized the sox2/3/19a/19b quadruple knockdown embryos in detail by examining the changes in gene expression through in situ hybridization, RT–PCR, and microarray analyses. Importantly, these phenotypic analyses revealed that the B1 SOX proteins regulate the following distinct processes: (1) early dorsoventral patterning by controlling bmp2b/7; (2) gastrulation movements via the regulation of pcdh18a/18b and wnt11, a non-canonical Wnt ligand gene; (3) neural differentiation by regulating the Hes-class bHLH gene her3 and the proneural-class bHLH genes neurog1 (positively) and ascl1a (negatively), and regional transcription factor genes, e.g., hesx1, zic1, and rx3; and (4) neural patterning by regulating signaling pathway genes, cyp26a1 in RA signaling, oep in Nodal signaling, shh, and mdkb. Chromatin immunoprecipitation analysis of the her3, hesx1, neurog1, pcdh18a, and cyp26a1 genes further suggests a direct regulation of these genes by B1 SOX. We also found an interesting overlap between the early phenotypes of the B1 sox quadruple knockdown embryos and the maternal-zygotic spg embryos that are devoid of pou5f1 activity. These findings indicate that the B1 SOX proteins control a wide range of developmental regulators in the early embryo through partnering in part with Pou5f1 and possibly with other factors, and suggest that the B1 sox functions are central to coordinating cell fate specification with patterning and morphogenetic processes occurring in the early embryo