Mixed-Integer Convex Nonlinear Optimization with Gradient-Boosted Trees Embedded

Decision trees usefully represent sparse, high dimensional and noisy data. Having learned a function from this data, we may want to thereafter integrate the function into a larger decision-making problem, e.g., for picking the best chemical process catalyst. We study a large-scale, industrially-relevant mixed-integer nonlinear nonconvex optimization problem involving both gradient-boosted trees and penalty functions mitigating risk. This mixed-integer optimization problem with convex penalty terms broadly applies to optimizing pre-trained regression tree models. Decision makers may wish to optimize discrete models to repurpose legacy predictive models, or they may wish to optimize a discrete model that particularly well-represents a data set. We develop several heuristic methods to find feasible solutions, and an exact, branch-and-bound algorithm leveraging structural properties of the gradient-boosted trees and penalty functions. We computationally test our methods on concrete mixture design instance and a chemical catalysis industrial instance

Minimum and maximum entropy distributions for binary systems with known means and pairwise correlations

Maximum entropy models are increasingly being used to describe the collective activity of neural populations with measured mean neural activities and pairwise correlations, but the full space of probability distributions consistent with these constraints has not been explored. We provide upper and lower bounds on the entropy for the {\em minimum} entropy distribution over arbitrarily large collections of binary units with any fixed set of mean values and pairwise correlations. We also construct specific low-entropy distributions for several relevant cases. Surprisingly, the minimum entropy solution has entropy scaling logarithmically with system size for any set of first- and second-order statistics consistent with arbitrarily large systems. We further demonstrate that some sets of these low-order statistics can only be realized by small systems. Our results show how only small amounts of randomness are needed to mimic low-order statistical properties of highly entropic distributions, and we discuss some applications for engineered and biological information transmission systems.Comment: 34 pages, 7 figure

Linear Programming Decoding of Spatially Coupled Codes

For a given family of spatially coupled codes, we prove that the LP threshold on the BSC of the graph cover ensemble is the same as the LP threshold on the BSC of the derived spatially coupled ensemble. This result is in contrast with the fact that the BP threshold of the derived spatially coupled ensemble is believed to be larger than the BP threshold of the graph cover ensemble as noted by the work of Kudekar et al. (2011, 2012). To prove this, we establish some properties related to the dual witness for LP decoding which was introduced by Feldman et al. (2007) and simplified by Daskalakis et al. (2008). More precisely, we prove that the existence of a dual witness which was previously known to be sufficient for LP decoding success is also necessary and is equivalent to the existence of certain acyclic hyperflows. We also derive a sublinear (in the block length) upper bound on the weight of any edge in such hyperflows, both for regular LPDC codes and for spatially coupled codes and we prove that the bound is asymptotically tight for regular LDPC codes. Moreover, we show how to trade crossover probability for "LP excess" on all the variable nodes, for any binary linear code.Comment: 37 pages; Added tightness construction, expanded abstrac

Critical slowing down and hyperuniformity on approach to jamming

Hyperuniformity characterizes a state of matter that is poised at a critical point at which density or volume-fraction fluctuations are anomalously suppressed at infinite wavelengths. Recently, much attention has been given to the link between strict jamming and hyperuniformity in frictionless hard-particle packings. Doing so requires one to study very large packings, which can be difficult to jam properly. We modify the rigorous linear programming method of Donev et al. [J. Comp. Phys. 197, 139 (2004)] in order to test for jamming in putatively jammed packings of hard-disks in two dimensions. We find that various standard packing protocols struggle to reliably create packings that are jammed for even modest system sizes; importantly, these packings appear to be jammed by conventional tests. We present evidence that suggests that deviations from hyperuniformity in putative maximally random jammed (MRJ) packings can in part be explained by a shortcoming in generating exactly-jammed configurations due to a type of "critical slowing down" as the necessary rearrangements become difficult to realize by numerical protocols. Additionally, various protocols are able to produce packings exhibiting hyperuniformity to different extents, but this is because certain protocols are better able to approach exactly-jammed configurations. Nonetheless, while one should not generally expect exact hyperuniformity for disordered packings with rattlers, we find that when jamming is ensured, our packings are very nearly hyperuniform, and deviations from hyperuniformity correlate with an inability to ensure jamming, suggesting that strict jamming and hyperuniformity are indeed linked. This raises the possibility that the ideal MRJ packings have no rattlers. Our work provides the impetus for the development of packing algorithms that produce large disordered strictly jammed packings that are rattler-free.Comment: 15 pages, 11 figures. Accepted for publication in Phys. Rev.