Information-based complexity, feedback and dynamics in convex programming

We study the intrinsic limitations of sequential convex optimization through the lens of feedback information theory. In the oracle model of optimization, an algorithm queries an {\em oracle} for noisy information about the unknown objective function, and the goal is to (approximately) minimize every function in a given class using as few queries as possible. We show that, in order for a function to be optimized, the algorithm must be able to accumulate enough information about the objective. This, in turn, puts limits on the speed of optimization under specific assumptions on the oracle and the type of feedback. Our techniques are akin to the ones used in statistical literature to obtain minimax lower bounds on the risks of estimation procedures; the notable difference is that, unlike in the case of i.i.d. data, a sequential optimization algorithm can gather observations in a {\em controlled} manner, so that the amount of information at each step is allowed to change in time. In particular, we show that optimization algorithms often obey the law of diminishing returns: the signal-to-noise ratio drops as the optimization algorithm approaches the optimum. To underscore the generality of the tools, we use our approach to derive fundamental lower bounds for a certain active learning problem. Overall, the present work connects the intuitive notions of information in optimization, experimental design, estimation, and active learning to the quantitative notion of Shannon information.Comment: final version; to appear in IEEE Transactions on Information Theor

Information Complexity of Black-Box Convex Optimization: A New Look via Feedback Information Theory

This paper revisits information complexity of black-box convex optimization, first studied in the seminal work of Nemirovski and Yudin, from the perspective of feedback information theory. These days, large-scale convex programming arises in a variety of applications, and it is important to refine our understanding of its fundamental limitations. The goal of black-box convex optimization is to minimize an unknown convex objective function from a given class over a compact, convex domain using an iterative scheme that generates approximate solutions by querying an oracle for local information about the function being optimized. The information complexity of a given problem class is defined as the smallest number of queries needed to minimize every function in the class to some desired accuracy. We present a simple information-theoretic approach that not only recovers many of the results of Nemirovski and Yudin, but also gives some new bounds pertaining to optimal rates at which iterative convex optimization schemes approach the solution. As a bonus, we give a particularly simple derivation of the minimax lower bound for a certain active learning problem on the unit interval

Information complexity of black-box convex optimization: A new look via feedback information theory

Abstract-This paper revisits information complexity of black-box convex optimization, first studied in the seminal work of Nemirovski and Yudin, from the perspective of feedback information theory. These days, large-scale convex programming arises in a variety of applications, and it is important to refine our understanding of its fundamental limitations. The goal of black-box convex optimization is to minimize an unknown convex objective function from a given class over a compact, convex domain using an iterative scheme that generates approximate solutions by querying an oracle for local information about the function being optimized. The information complexity of a given problem class is defined as the smallest number of queries needed to minimize every function in the class to some desired accuracy. We present a simple information-theoretic approach that not only recovers many of the results of Nemirovski and Yudin, but also gives some new bounds pertaining to optimal rates at which iterative convex optimization schemes approach the solution. As a bonus, we give a particularly simple derivation of the minimax lower bound for a certain active learning problem on the unit interval

Lower Bounds for Passive and Active Learning

We develop unified information-theoretic machinery for deriving lower bounds for passive and active learning schemes. Our bounds involve the so-called Alexander’s capacity function. The supremum of this function has been recently rediscovered by Hanneke in the context of active learning under the name of “disagreement coefficient. ” For passive learning, our lower bounds match the upper bounds of Giné and Koltchinskii up to constants and generalize analogous results of Massart and Nédélec. For active learning, we provide first known lower bounds based on the capacity function rather than the disagreement coefficient.

Information complexity of black-box convex optimization: A new look via feedback information theory

Abstract-This paper revisits information complexity of black-box convex optimization, first studied in the seminal work of Nemirovski and Yudin, from the perspective of feedback information theory. These days, large-scale convex programming arises in a variety of applications, and it is important to refine our understanding of its fundamental limitations. The goal of black-box convex optimization is to minimize an unknown convex objective function from a given class over a compact, convex domain using an iterative scheme that generates approximate solutions by querying an oracle for local information about the function being optimized. The information complexity of a given problem class is defined as the smallest number of queries needed to minimize every function in the class to some desired accuracy. We present a simple information-theoretic approach that not only recovers many of the results of Nemirovski and Yudin, but also gives some new bounds pertaining to optimal rates at which iterative convex optimization schemes approach the solution. As a bonus, we give a particularly simple derivation of the minimax lower bound for a certain active learning problem on the unit interval

Towards Exciton-Polaritons in an Individual MoS2 Nanotube

We measure low-temperature micro-photoluminescence spectra along a MoS 2 nanotube, which exhibit the peaks of the optical whispering gallery modes below the exciton resonance. The energy fluctuation and width of these peaks are determined by the changes of the nanotube wall thickness and propagation of the optical modes along the nanotube axis, respectively. We demonstrate the potential of the high-quality nanotubes for realization of the strong coupling between exciton and optical modes when the Rabi splitting can reach 400 meV. We show how the formation of exciton-polaritons in such structures will be manifested in the micro-photoluminescence spectra and analyze the conditions needed to realize that

Bright Single-Photon Emitters with a CdSe Quantum Dot and Multimode Tapered Nanoantenna for the Visible Spectral Range

We report on single photon emitters for the green-yellow spectral range, which comprise a CdSe/ZnSe quantum dot placed inside a semiconductor tapered nanocolumn acting as a multimode nanoantenna. Despite the presence of many optical modes inside, such a nanoantenna is able to collect the quantum dot radiation and ensure its effective output. We demonstrate periodic arrays of such emitters, which are fabricated by focused ion beam etching from a II-VI/III-V heterostructure grown using molecular beam epitaxy. With non-resonant optical pumping, the average count rate of emitted single photons exceeds 5 MHz with the second-order correlation function g(2)(0) = 0.25 at 220 K. Such single photon emitters are promising for secure free space optical communication lines

Bright Single-Photon Sources for the Telecommunication O-Band Based on an InAs Quantum Dot with (In)GaAs Asymmetric Barriers in a Photonic Nanoantenna

We report on single-photon emitters for the telecommunication O-band (1260–1360 nm), which comprise an InAs/(In)GaAs quantum dot with asymmetric barriers, placed inside a semiconductor tapered nanocolumn acting as a photonic nanoantenna. The implemented design of the barriers provides a shift in the quantum dot radiation wavelength towards the O-band, while the nanoantenna collects the radiation and ensures its effective output. With non-resonant optical pumping, the average count rate of emitted single photons exceeds 10 MHz with the second-order correlation function g(2)(0) = 0.18 at 8 K