Deep Bilevel Learning

We present a novel regularization approach to train neural networks that enjoys better generalization and test error than standard stochastic gradient descent. Our approach is based on the principles of cross-validation, where a validation set is used to limit the model overfitting. We formulate such principles as a bilevel optimization problem. This formulation allows us to define the optimization of a cost on the validation set subject to another optimization on the training set. The overfitting is controlled by introducing weights on each mini-batch in the training set and by choosing their values so that they minimize the error on the validation set. In practice, these weights define mini-batch learning rates in a gradient descent update equation that favor gradients with better generalization capabilities. Because of its simplicity, this approach can be integrated with other regularization methods and training schemes. We evaluate extensively our proposed algorithm on several neural network architectures and datasets, and find that it consistently improves the generalization of the model, especially when labels are noisy.Comment: ECCV 201

Revisiting loss-specific training of filter-based MRFs for image restoration

It is now well known that Markov random fields (MRFs) are particularly effective for modeling image priors in low-level vision. Recent years have seen the emergence of two main approaches for learning the parameters in MRFs: (1) probabilistic learning using sampling-based algorithms and (2) loss-specific training based on MAP estimate. After investigating existing training approaches, it turns out that the performance of the loss-specific training has been significantly underestimated in existing work. In this paper, we revisit this approach and use techniques from bi-level optimization to solve it. We show that we can get a substantial gain in the final performance by solving the lower-level problem in the bi-level framework with high accuracy using our newly proposed algorithm. As a result, our trained model is on par with highly specialized image denoising algorithms and clearly outperforms probabilistically trained MRF models. Our findings suggest that for the loss-specific training scheme, solving the lower-level problem with higher accuracy is beneficial. Our trained model comes along with the additional advantage, that inference is extremely efficient. Our GPU-based implementation takes less than 1s to produce state-of-the-art performance.Comment: 10 pages, 2 figures, appear at 35th German Conference, GCPR 2013, Saarbr\"ucken, Germany, September 3-6, 2013. Proceeding

Simulations ofthe 100kW TJNAF FEL using a step-tapered undulator

The Thomas Jefferson National Accelerator Facility (TJNAF) free electron laser (FEL) can be upgraded to operate at 100kW average power in the near future using a configuration that recirculates the electron beam to recover energy. It is important to extract the maximum energy from the electron beam in a pass through the undulator while inducing the minimum amount of exhaust energy spread. A larger energy extraction reduces the requirement for a large recirculating current, while a smaller exhaust energy spread allows the intense electron beam to be recirculated without damaging components. To improve FEL performance, we explore the use of the step-tapered undulator, which alters the resonance condition halfway through the undulator. Short pulses complicate the desired interaction. Comparisons are made to the conventional periodic and linearly-tapered undulators.The authors are grateful for the support of the Office of Naval Research, Thomas Jefferson National Accelerator Facility, and contributions of Dave Douglas of TJNAF

Effective suckling in relation to naked maternal-infant body contact in the first hour of life: an observation study

Background Best practice guidelines to promote breastfeeding suggest that (i) mothers hold their babies in naked body contact immediately after birth, (ii) babies remain undisturbed for at least one hour and (iii) breastfeeding assistance be offered during this period. Few studies have closely observed the implementation of these guidelines in practice. We sought to evaluate these practices on suckling achievement within the first hour after birth. Methods Observations of seventy-eight mother-baby dyads recorded newborn feeding behaviours, the help received by mothers and birthing room practices each minute, for sixty minutes. Results Duration of naked body contact between mothers and their newborn babies varied widely from 1 to 60 minutes, as did commencement of suckling (range = 10 to 60 minutes). Naked maternal-infant body contact immediately after birth, uninterrupted for at least thirty minutes did not predict effective suckling within the first hour of birth. Newborns were four times more likely to sustain deep rhythmical suckling when their chin made contact with their mother’s breast as they approached the nipple (OR 3.8; CI 1.03 - 14) and if their mothers had given birth previously (OR 6.7; CI 1.35 - 33). Infants who had any naso-oropharyngeal suctioning administered at birth were six times less likely to suckle effectively (OR .176; CI .04 - .9). Conclusion Effective suckling within the first hour of life was associated with a collection of practices including infants positioned so their chin can instinctively nudge the underside of their mother’s breast as they approach to grasp the nipple and attach to suckle. The best type of assistance provided in the birthing room that enables newborns to sustain an effective latch was paying attention to newborn feeding behaviours and not administering naso-oropharyngeal suction routinely

A parametric integer programming algorithm for bilevel mixed integer programs

We consider discrete bilevel optimization problems where the follower solves an integer program with a fixed number of variables. Using recent results in parametric integer programming, we present polynomial time algorithms for pure and mixed integer bilevel problems. For the mixed integer case where the leader's variables are continuous, our algorithm also detects whether the infimum cost fails to be attained, a difficulty that has been identified but not directly addressed in the literature. In this case it yields a ``better than fully polynomial time'' approximation scheme with running time polynomial in the logarithm of the relative precision. For the pure integer case where the leader's variables are integer, and hence optimal solutions are guaranteed to exist, we present two algorithms which run in polynomial time when the total number of variables is fixed.Comment: 11 page

Quantum cascade laser-based reflectance spectroscopy: a robust approach for the classification of plastic type

© The Author(s), 2020. This article is distributed under the terms of the Creative Commons Attribution License. The definitive version was published in Michel, A. P. M., Morrison, A. E., Colson, B. C., Pardis, W. A., Moya, X. A., Harb, C. C., & White, H. K. Quantum cascade laser-based reflectance spectroscopy: a robust approach for the classification of plastic type. Optics Express, 28(12), (2020): 17741-17756, doi:10.1364/OE.393231.The identification of plastic type is important for environmental applications ranging from recycling to understanding the fate of plastics in marine, atmospheric, and terrestrial environments. Infrared reflectance spectroscopy is a powerful approach for plastics identification, requiring only optical access to a sample. The use of visible and near-infrared wavelengths for plastics identification are limiting as dark colored plastics absorb at these wavelengths, producing no reflectance spectra. The use of mid-infrared wavelengths instead enables dark plastics to be identified. Here we demonstrate the capability to utilize a pulsed, widely-tunable (5.59 - 7.41 µm) mid-infrared quantum cascade laser, as the source for reflectance spectroscopy, for the rapid and robust identification of plastics. Through the application of linear discriminant analysis to the resulting spectral data set, we demonstrate that we can correctly classify five plastic types: polyethylene terephthalate (PET), high density polyethylene (HDPE), low density polyethylene (LDPE), polypropylene (PP), and polystyrene (PS), with a 97% accuracy rate.Richard Saltonstall Charitable Foundation; National Academies Keck Futures Initiative (NAKFI DBS13)

Absence of large nanoscale electronic inhomogeneities in the Ba(Fe1-xCox)2As2 pnictide

75As NMR and susceptiblity were measured in a Ba(Fe1-xCox)2As2 single crystal for x=6% for various field H values and orientations. The sharpness of the superconducting and magnetic transitions demonstrates a homogeneity of the Co doping x better than +-0.25%. On the nanometer scale, the paramagnetic part of the NMR spectra is found very anisotropic and very narrow for H//ab which allows to rule out the interpretation of Ref.[6] in terms of strong Co induced electronic inhomogeneities. We propose that a distribution of hyperfine couplings and chemical shifts due to the Co effect on its nearest As explains the observed linewidths and relaxations. All these measurements show that Co substitution induces a very homogeneous electronic doping in BaFe2As2, from nano to micrometer lengthscales, on the contrary to the K doping.Comment: 6 pages, 4 figure

Phase separation of Bose-Einstein condensates

The zero-temperature system of two dilute overlapping Bose-Einstein condensates is unstable against long wavelength excitations if the interaction strength between the distinguishable bosons exceeds the geometric mean of the like-boson interaction strengths. If the condensates attract each other, the instability is similar to the instability of the negative scattering length condensates. If the condensates repel, they separate spatially into condensates of equal pressure. We estimate the boundary size, surface tension and energy of the phase separated condensate system and we discuss the implications for double condensates in atomic traps.Comment: 11 pages, 1 figur

Dynamics of two interacting Bose-Einstein condensates

We analize the dynamics of two trapped interacting Bose-Einstein condensates and indentify two regimes for the evolution: the regime of slow periodic oscillations and the regime of strong non-linear mixing leading to the damping of the relative motion of the condensates. We compare our predictions with an experiment recently performed at JILA.Comment: 4 pages RevTeX, 3 eps figure