On the Optimization of Deep Networks: Implicit Acceleration by Overparameterization

Conventional wisdom in deep learning states that increasing depth improves expressiveness but complicates optimization. This paper suggests that, sometimes, increasing depth can speed up optimization. The effect of depth on optimization is decoupled from expressiveness by focusing on settings where additional layers amount to overparameterization - linear neural networks, a well-studied model. Theoretical analysis, as well as experiments, show that here depth acts as a preconditioner which may accelerate convergence. Even on simple convex problems such as linear regression with $\ell_p$ loss, $p>2$, gradient descent can benefit from transitioning to a non-convex overparameterized objective, more than it would from some common acceleration schemes. We also prove that it is mathematically impossible to obtain the acceleration effect of overparametrization via gradients of any regularizer.Comment: Published at the International Conference on Machine Learning (ICML) 201

Organic Photodiodes with an Extended Responsivity using Ultrastrong Light-Matter Coupling

In organic photodiodes (OPDs) light is absorbed by excitons, which dissociate to generate photocurrent. Here, we demonstrate a novel type of OPD in which light is absorbed by polaritons, hybrid light-matter states. We demonstrate polariton OPDs operating in the ultra-strong coupling regime at visible and infrared wavelengths. These devices can be engineered to show narrow responsivity with a very weak angle-dependence. More importantly, they can be tuned to operate in a spectral range outside that of the bare exciton absorption. Remarkably, we show that the responsivity of a polariton OPD can be pushed to near infrared wavelengths, where few organic absorbers are available, with external quantum efficiencies exceeding those of a control OPD

Inverting Singlet and Triplet Excited States using Strong Light-Matter Coupling

In organic microcavities, hybrid light-matter states can form with energies that differ from the bare molecular excitation energies by nearly 1 eV. A timely question, given recent advances in the development of thermally activated delayed fluorescence materials, is whether strong light-matter coupling can be used to invert the ordering of singlet and triplet states and, in addition, enhance reverse intersystem crossing (RISC) rates. Here, we demonstrate a complete inversion of the singlet lower polariton and triplet excited states. We also unambiguously measure the RISC rate in strongly-coupled organic microcavities and find that, regardless of the large energy level shifts, it is unchanged compared to films of the bare molecules. This observation is a consequence of slow RISC to the lower polariton due to the delocalized nature of the state across many molecules and an inability to compete with RISC to the dark exciton reservoir, which occurs at a rate comparable to that in bare molecules

Triplet harvesting in the polaritonic regime: a variational polaron approach

We explore the electroluminescence efficiency for a quantum mechanical model of a large number of molecular emitters embedded in an optical microcavity. We characterize the circumstances under which a microcavity enhances harvesting of triplet excitons via reverse intersystem-crossing (R-ISC) into singlet populations that can emit light. For that end, we develop a time-local master equation in a variationally optimized frame which allows for the exploration of the population dynamics of chemically relevant species in different regimes of emitter coupling to the condensed phase vibrational bath and to the microcavity photonic mode. For a vibrational bath that equilibrates faster than R-ISC (in emitters with weak singlet-triplet mixing), our results reveal that significant improvements in efficiencies with respect to the cavity-free counterpart can be obtained for strong coupling of the singlet exciton to a photonic mode, as long as the singlet to triplet exciton transition is within the inverted Marcus regime; under these circumstances, we show the possibility to overcome the detrimental delocalization of the polariton states across a macroscopic number of molecules. On the other hand, for a vibrational bath that equilibrates slower than R-ISC (i.e., emitters with strong singlet-triplet mixing), we find that while enhancemnents in photoluminiscence can be obtained via vibrational relaxation into polaritons, this only occurs for small number of emitters coupled to the photon mode, with delocalization of the polaritons across many emitters eventually being detrimental to electroluminescence efficiency. These findings provide insight on the tunability of optoelectronic processes in molecular materials due to weak and strong light-matter coupling

Spiking Optical Patterns and Synchronization

We analyze the time resolved spike statistics of a solitary and two mutually interacting chaotic semiconductor lasers whose chaos is characterized by apparently random, short intensity spikes. Repulsion between two successive spikes is observed, resulting in a refractory period which is largest at laser threshold. For time intervals between spikes greater than the refractory period, the distribution of the intervals follows a Poisson distribution. The spiking pattern is highly periodic over time windows corresponding to the optical length of the external cavity, with a slow change of the spiking pattern as time increases. When zero-lag synchronization between the two lasers is established, the statistics of the nearly perfectly matched spikes are not altered. The similarity of these features to those found in complex interacting neural networks, suggests the use of laser systems as simpler physical models for neural networks

What's Behind the Mask: Estimating Uncertainty in Image-to-Image Problems

Estimating uncertainty in image-to-image networks is an important task, particularly as such networks are being increasingly deployed in the biological and medical imaging realms. In this paper, we introduce a new approach to this problem based on masking. Given an existing image-to-image network, our approach computes a mask such that the distance between the masked reconstructed image and the masked true image is guaranteed to be less than a specified threshold, with high probability. The mask thus identifies the more certain regions of the reconstructed image. Our approach is agnostic to the underlying image-to-image network, and only requires triples of the input (degraded), reconstructed and true images for training. Furthermore, our method is agnostic to the distance metric used. As a result, one can use $L_p$-style distances or perceptual distances like LPIPS, which contrasts with interval-based approaches to uncertainty. Our theoretical guarantees derive from a conformal calibration procedure. We evaluate our mask-based approach to uncertainty on image colorization, image completion, and super-resolution tasks, demonstrating high quality performance on each

A Neural Space-Time Representation for Text-to-Image Personalization

A key aspect of text-to-image personalization methods is the manner in which the target concept is represented within the generative process. This choice greatly affects the visual fidelity, downstream editability, and disk space needed to store the learned concept. In this paper, we explore a new text-conditioning space that is dependent on both the denoising process timestep (time) and the denoising U-Net layers (space) and showcase its compelling properties. A single concept in the space-time representation is composed of hundreds of vectors, one for each combination of time and space, making this space challenging to optimize directly. Instead, we propose to implicitly represent a concept in this space by optimizing a small neural mapper that receives the current time and space parameters and outputs the matching token embedding. In doing so, the entire personalized concept is represented by the parameters of the learned mapper, resulting in a compact, yet expressive, representation. Similarly to other personalization methods, the output of our neural mapper resides in the input space of the text encoder. We observe that one can significantly improve the convergence and visual fidelity of the concept by introducing a textual bypass, where our neural mapper additionally outputs a residual that is added to the output of the text encoder. Finally, we show how one can impose an importance-based ordering over our implicit representation, providing users control over the reconstruction and editability of the learned concept using a single trained model. We demonstrate the effectiveness of our approach over a range of concepts and prompts, showing our method's ability to generate high-quality and controllable compositions without fine-tuning any parameters of the generative model itself.Comment: Project page available at https://neuraltextualinversion.github.io/NeTI

Redundant Wavelets on Graphs and High Dimensional Data Clouds

In this paper, we propose a new redundant wavelet transform applicable to scalar functions defined on high dimensional coordinates, weighted graphs and networks. The proposed transform utilizes the distances between the given data points. We modify the filter-bank decomposition scheme of the redundant wavelet transform by adding in each decomposition level linear operators that reorder the approximation coefficients. These reordering operators are derived by organizing the tree-node features so as to shorten the path that passes through these points. We explore the use of the proposed transform to image denoising, and show that it achieves denoising results that are close to those obtained with the BM3D algorithm.Comment: 4 pages, 4 figures, 1 table, submitted to IEEE Signal Processing Letter