Second Order Derivatives for Network Pruning: Optimal Brain Surgeon

We investigate the use of information from all second order derivatives of the error function to perform network pruning (i.e., removing unimportant weights from a trained network) in order to improve generalization, simplify networks, reduce hardware or storage requirements, increase the speed of further training, and in some cases enable rule extraction. Our method, Optimal Brain Surgeon (OBS), is Significantly better than magnitude-based methods and Optimal Brain Damage [Le Cun, Denker and Sol1a, 1990], which often remove the wrong weights. OBS permits the pruning of more weights than other methods (for the same error on the training set), and thus yields better generalization on test data. Crucial to OBS is a recursion relation for calculating the inverse Hessian matrix H^(-1) from training data and structural information of the net. OBS permits a 90%, a 76%, and a 62% reduction in weights over backpropagation with weigh decay on three benchmark MONK's problems [Thrun et aI., 1991]. Of OBS, Optimal Brain Damage, and magnitude-based methods, only OBS deletes the correct weights from a trained XOR network in every case. Finally, whereas Sejnowski and Rosenberg [1987J used 18,000 weights in their NETtalk network, we used OBS to prune a network to just 1560 weights, yielding better generalization

A Simulated Microgravity Environment Causes a Sustained Defect in Epithelial Barrier Function.

Intestinal epithelial cell (IEC) junctions constitute a robust barrier to invasion by viruses, bacteria and exposure to ingested agents. Previous studies showed that microgravity compromises the human immune system and increases enteropathogen virulence. However, the effects of microgravity on epithelial barrier function are poorly understood. The aims of this study were to identify if simulated microgravity alters intestinal epithelial barrier function (permeability), and susceptibility to barrier-disrupting agents. IECs (HT-29.cl19a) were cultured on microcarrier beads in simulated microgravity using a rotating wall vessel (RWV) for 18 days prior to seeding on semipermeable supports to measure ion flux (transepithelial electrical resistance (TER)) and FITC-dextran (FD4) permeability over 14 days. RWV cells showed delayed apical junction localization of the tight junction proteins, occludin and ZO-1. The alcohol metabolite, acetaldehyde, significantly decreased TER and reduced junctional ZO-1 localization, while increasing FD4 permeability in RWV cells compared with static, motion and flask control cells. In conclusion, simulated microgravity induced an underlying and sustained susceptibility to epithelial barrier disruption upon removal from the microgravity environment. This has implications for gastrointestinal homeostasis of astronauts in space, as well as their capability to withstand the effects of agents that compromise intestinal epithelial barrier function following return to Earth

Optimal Brain Surgeon and general network pruning

The use of information from all second-order derivatives of the error function to perform network pruning (i.e., removing unimportant weights from a trained network) in order to improve generalization, simplify networks, reduce hardware or storage requirements, increase the speed of further training, and, in some cases, enable rule extraction, is investigated. The method, Optimal Brain Surgeon (OBS), is significantly better than magnitude-based methods and Optimal Brain Damage, which often remove the wrong weights. OBS, permits pruning of more weights than other methods (for the same error on the training set), and thus yields better generalization on test data. Crucial to OBS is a recursion relation for calculating the inverse Hessian matrix H^-1 from training data and structural information of the set. OBS permits a 76%, a 62%, and a 90% reduction in weights over backpropagation with weight decay on three benchmark MONK'S problems. Of OBS, Optimal Brain Damage, and a magnitude-based method, only OBS deletes the correct weights from a trained XOR network in every case. Finally, whereas Sejnowski and Rosenberg used 18,000 weights in their NETtalk network, we used OBS to prune a network to just 1,560 weights, yielding better generalization

The Conditional Heteroscedasticity of the Yen-Dollar Exchange Rates

Journal of Applied Econometrics13149-5

The common feeder cockroach Blaptica dubia shows increased transmission distance based on mode of acquisition of environmental bacteria

Although some researchers claim that cockroaches are masters of disease transmission, these claims have little to no scientific support. Most studies concerning cockroaches as a vector of disease only focus on the bacteria found on the body surface, not on whether cockroaches have actually transferred pathogenic bacteria via surface contact. We set out to determine if cockroaches would act as a mechanical vector for the transfer of the opportunistic pathogen, E. coli. Roaches were contaminated with Green fluorescent protein expressing E. coli (GFP-E.coli) broth by either walking the roach through a broth culture or by complete immersion in the culture.. We then ran the roaches down a sterile agar track and measured the length of the glowing trail. Roaches were able to transmit E.coli, but only for a continuous distance of less than 50 cm, with the occasional sporadic colony growing after that. Roaches that were immersed in bacterial broth tracked the bacterium further than those that only walked through the solution. This suggests that while cockroaches are capable of acting as a mechanical vector, they are not capable of transporting transient flora over long distances. Future studies should explore this mechanism

In search of Leonardo: Computer-based facial image analysis of Renaissance artworks for identifying Leonardo as subject

One of the enduring mysteries in the history of the Renaissance is the adult appearance of the archetypical "Renaissance Man," Leonardo da Vinci. His only acknowledged self-portrait is from an advanced age, and various candidate images of younger men are difficult to assess given the absence of documentary evidence. One clue about Leonardo's appearance comes from the remark of the contemporary historian, Vasari, that the sculpture of David by Leonardo's master, Andrea del Verrocchio, was based on the appearance of Leonardo when he was an apprentice. Taking a cue from this statement, we suggest that the more mature sculpture of St. Thomas, also by Verrocchio, might also have been a portrait of Leonardo. We tested the possibility Leonardo was the subject for Verrocchio's sculpture by a novel computational technique for the comparison of three-dimensional facial configurations. Based on quantitative measures of similarities, we also assess whether another pair of candidate two-dimensional images are plausibly attributable as being portraits of Leonardo as a young adult. Our results are consistent with the claim Leonardo is indeed the subject in these works, but we need comparisons with images in a larger corpora of candidate artworks before our results achieve statistical significance

Conditioning bounds for traveltime tomography in layered media

This paper revisits the problem of recovering a smooth, isotropic, layered wave speed profile from surface traveltime information. While it is classic knowledge that the diving (refracted) rays classically determine the wave speed in a weakly well-posed fashion via the Abel transform, we show in this paper that traveltimes of reflected rays do not contain enough information to recover the medium in a well-posed manner, regardless of the discretization. The counterpart of the Abel transform in the case of reflected rays is a Fredholm kernel of the first kind which is shown to have singular values that decay at least root-exponentially. Kinematically equivalent media are characterized in terms of a sequence of matching moments. This severe conditioning issue comes on top of the well-known rearrangement ambiguity due to low velocity zones. Numerical experiments in an ideal scenario show that a waveform-based model inversion code fits data accurately while converging to the wrong wave speed profile

Extensive collection of femtoliter pad secretion droplets in beetle Leptinotarsa decemlineata allows nanoliter microrheology

Pads of beetles are covered with long, deformable setae, each ending in a micrometric terminal plate coated with secretory fluid. It was recently shown that the layer of the pad secretion covering the terminal plates is responsible for the generation of strong attractive forces. However, less is known about the fluid itself because it is produced in extremely small quantity. We here present a first experimental investigation of the rheological properties of the pad secretion in the Colorado potato beetle {\it Leptinotarsa decemlineata}. Because the secretion is produced in an extremely small amount at the level of the terminal plate, we first develop a procedure based on capillary effects to collect the secretion. We then manage to incorporate micrometric beads, initially in the form of a dry powder, and record their thermal motion to determine the mechanical properties of the surrounding medium. We achieve such a quantitative measurement within the collected volume, much smaller than the $1 {\rm \mu}$l sample volume usually required for this technique. Surprisingly, the beetle secretion was found to behave as a purely viscous liquid, of high viscosity. This suggests that no specific complex fluid behaviour is needed during beetle locomotion. We build a scenario for the contact formation between the spatula at the setal tip and a substrate, during the insect walk. We show that the attachment dynamics of the insect pad computed from the high measured viscosity is in good agreement with observed insect pace. We finally discuss the consequences of the secretion viscosity on the insect adhesion