Parametric inference in the large data limit using maximally informative models

Motivated by data-rich experiments in transcriptional regulation and sensory neuroscience, we consider the following general problem in statistical inference. When exposed to a high-dimensional signal S, a system of interest computes a representation R of that signal which is then observed through a noisy measurement M. From a large number of signals and measurements, we wish to infer the "filter" that maps S to R. However, the standard method for solving such problems, likelihood-based inference, requires perfect a priori knowledge of the "noise function" mapping R to M. In practice such noise functions are usually known only approximately, if at all, and using an incorrect noise function will typically bias the inferred filter. Here we show that, in the large data limit, this need for a pre-characterized noise function can be circumvented by searching for filters that instead maximize the mutual information I[M;R] between observed measurements and predicted representations. Moreover, if the correct filter lies within the space of filters being explored, maximizing mutual information becomes equivalent to simultaneously maximizing every dependence measure that satisfies the Data Processing Inequality. It is important to note that maximizing mutual information will typically leave a small number of directions in parameter space unconstrained. We term these directions "diffeomorphic modes" and present an equation that allows these modes to be derived systematically. The presence of diffeomorphic modes reflects a fundamental and nontrivial substructure within parameter space, one that is obscured by standard likelihood-based inference.Comment: To appear in Neural Computatio

Equitability, mutual information, and the maximal information coefficient

Reshef et al. recently proposed a new statistical measure, the "maximal information coefficient" (MIC), for quantifying arbitrary dependencies between pairs of stochastic quantities. MIC is based on mutual information, a fundamental quantity in information theory that is widely understood to serve this need. MIC, however, is not an estimate of mutual information. Indeed, it was claimed that MIC possesses a desirable mathematical property called "equitability" that mutual information lacks. This was not proven; instead it was argued solely through the analysis of simulated data. Here we show that this claim, in fact, is incorrect. First we offer mathematical proof that no (non-trivial) dependence measure satisfies the definition of equitability proposed by Reshef et al.. We then propose a self-consistent and more general definition of equitability that follows naturally from the Data Processing Inequality. Mutual information satisfies this new definition of equitability while MIC does not. Finally, we show that the simulation evidence offered by Reshef et al. was artifactual. We conclude that estimating mutual information is not only practical for many real-world applications, but also provides a natural solution to the problem of quantifying associations in large data sets

Numerical study of large-eddy breakup and its effect on the drag characteristics of boundary layers

The break-up of a field of eddies by a flat-plate obstacle embedded in a boundary layer is studied using numerical solutions to the two-dimensional Navier-Stokes equations. The flow is taken to be incompressible and unsteady. The flow field is initiated from rest. A train of eddies of predetermined size and strength are swept into the computational domain upstream of the plate. The undisturbed velocity profile is given by the Blasius solution. The disturbance vorticity generated at the plate and wall, plus that introduced with the eddies, mix with the background vorticity and is transported throughout the entire flow. All quantities are scaled by the plate length, the unidsturbed free-stream velocity, and the fluid kinematic viscosity. The Reynolds number is 1000, the Blasius boundary layer thickness is 2.0, and the plate is positioned a distance of 1.0 above the wall. The computational domain is four units high and sixteen units long

Aircraft Wing for Over-The-Wing Mounting of Engine Nacelle

An aircraft wing has an inboard section and an outboard section. The inboard section is attached (i) on one side thereof to the aircraft's fuselage, and (ii) on an opposing side thereof to an inboard side of a turbofan engine nacelle in an over-the-wing mounting position. The outboard section's leading edge has a sweep of at least 20 degrees. The inboard section's leading edge has a sweep between -15 and +15 degrees, and extends from the fuselage to an attachment position on the nacelle that is forward of an index position defined as an imaginary intersection between the sweep of the outboard section's leading edge and the inboard side of the nacelle. In an alternate embodiment, the turbofan engine nacelle is replaced with an open rotor engine nacelle

Reply to Murrell et al.: Noise matters

The concept of statistical “equitability” plays a central role in the 2011 paper by Reshef et al. (1). Formalizing equitability first requires formalizing the notion of a “noisy functional relationship,” that is, a relationship between two real variables, X and Y, having the form Y=f(X)+η, where f is a function and η is a noise term. Whether a dependence measure satisfies equitability strongly depends on what mathematical properties the noise term η is allowed to have: the narrower one’s definition of noise, the weaker the equitability criterion becomes