Contrastive Learning for Lifted Networks

In this work we address supervised learning of neural networks via lifted network formulations. Lifted networks are interesting because they allow training on massively parallel hardware and assign energy models to discriminatively trained neural networks. We demonstrate that the training methods for lifted networks proposed in the literature have significant limitations and show how to use a contrastive loss to address those limitations. We demonstrate that this contrastive training approximates back-propagation in theory and in practice and that it is superior to the training objective regularly used for lifted networks.Comment: 9 pages, BMVC 201

Parallel computing in network optimization

Caption title.Includes bibliographical references (p. 82-95).Supported by the NSF. CCR-9103804Dimitri Bertsekas ... [et al.]

On the Fine-Grain Decomposition of Multicommodity Transportation Problems

We develop algorithms for nonlinear problems with multicommodity transportation constraints. The algorithms are of the row-action type and, when properly applied,decompose the underlying graph alternatingly by nodes and edges. Hence, a fine-grain decomposition scheme is developed that is suitable for massively parallel computer architectures of the SIMD (i.e., single instruction stream, multiple data stream) class. Implementations on the Connection Machine CM-2 are discussed for both dense and sparse transportation problems. The dense implementation achieves computing rate of 1.6-3 GFLOPS. Several aspects of the algorithm are investigated empirically. Computational results are reported for the solution of quadratic programs with approximately 10 million columns and 100 thousand rows

Further applications of a splitting algorithm to decomposition in variational inequalities and convex programming

Cover title.Includes bibliographical references.Partially supported by the U.S. Army Research Office (Center for Intelligent Control Systems) DAAL03-86-K-0171 Partially supported by the National Science Foundation. NSF-ECS-8519058Paul Tseng