Non-convex Optimization for Machine Learning

A vast majority of machine learning algorithms train their models and perform inference by solving optimization problems. In order to capture the learning and prediction problems accurately, structural constraints such as sparsity or low rank are frequently imposed or else the objective itself is designed to be a non-convex function. This is especially true of algorithms that operate in high-dimensional spaces or that train non-linear models such as tensor models and deep networks. The freedom to express the learning problem as a non-convex optimization problem gives immense modeling power to the algorithm designer, but often such problems are NP-hard to solve. A popular workaround to this has been to relax non-convex problems to convex ones and use traditional methods to solve the (convex) relaxed optimization problems. However this approach may be lossy and nevertheless presents significant challenges for large scale optimization. On the other hand, direct approaches to non-convex optimization have met with resounding success in several domains and remain the methods of choice for the practitioner, as they frequently outperform relaxation-based techniques - popular heuristics include projected gradient descent and alternating minimization. However, these are often poorly understood in terms of their convergence and other properties. This monograph presents a selection of recent advances that bridge a long-standing gap in our understanding of these heuristics. The monograph will lead the reader through several widely used non-convex optimization techniques, as well as applications thereof. The goal of this monograph is to both, introduce the rich literature in this area, as well as equip the reader with the tools and techniques needed to analyze these simple procedures for non-convex problems.Comment: The official publication is available from now publishers via http://dx.doi.org/10.1561/220000005

Human factors aspects of control room design: Guidelines and annotated bibliography

A human factors analysis of the workstation design for the Earth Radiation Budget Satellite mission operation room is discussed. The relevance of anthropometry, design rules, environmental design goals, and the social-psychological environment are discussed

Chromatic roots are dense in the whole complex plane

I show that the zeros of the chromatic polynomials P-G(q) for the generalized theta graphs Theta((s.p)) are taken together, dense in the whole complex plane with the possible exception of the disc \q - l\ < l. The same holds for their dichromatic polynomials (alias Tutte polynomials, alias Potts-model partition functions) Z(G)(q,upsilon) outside the disc \q + upsilon\ < \upsilon\. An immediate corollary is that the chromatic roots of not-necessarily-planar graphs are dense in the whole complex plane. The main technical tool in the proof of these results is the Beraha-Kahane-Weiss theorem oil the limit sets of zeros for certain sequences of analytic functions, for which I give a new and simpler proof

Information structure and the referential status of linguistic expression : workshop as part of the 23th annual meetings of the Deutsche Gesellschaft für Sprachwissenschaft in Leipzig, Leipzig, February 28 - March 2, 2001

This volume comprises papers that were given at the workshop Information Structure and the Referential Status of Linguistic Expressions, which we organized during the Deutsche Gesellschaft für Sprachwissenschaft (DGfS) Conference in Leipzig in February 2001. At this workshop we discussed the connection between information structure and the referential interpretation of linguistic expressions, a topic mostly neglected in current linguistics research. One common aim of the papers is to find out to what extent the focus-background as well as the topic-comment structuring determine the referential interpretation of simple arguments like definite and indefinite NPs on the one hand and sentences on the other

A Metaheuristic for the Pickup and Delivery Problem with Split-Loads and its Extension

In this dissertation, we study improvements in the Pickup and Delivery Problem that can be achieved by allowing multiple vehicle trips to serve a common load. We explore how costs can be reduced through the elimination of the constraint that a load must be served by only one vehicle trip. Specifically, we investigate the problem of routing vehicles to serve loads that have distinct origins and destinations, with no constraint on the amount of a load that a vehicle may serve at a time. We develop a metaheuristic to solve large scale practical size problems in this form and apply the metaheuristic to randomly generated data sets. The metaheuristic is based on a predetermined fixed number of restarts of annealing-like procedure with tabu-lists to avoid cycling in the search process and the annealing-like procedure is to guide the local search in three neighborhoods defined to solve the problem. We test the algorithm on several sets of problem instances generated with different transportation requests and over different load size ranges. The experimental results on these problem sets have shown that benefits are common if split loads are adopted in designing practical sized transportation network for different load size configurations, and the most benefit is achieved when all the loads are just a little above half of the vehicle capacity and have small variations, and this most benefit is around 33% for all the three 75-, 100-, and 125-request problem sets, which overtakes the one reported in previous literature. In a more general setting when some load sizes are greater than the vehicle capacity and have to be split, there are also certain cost reduction if split loads are applied. We also generate numeral tests on different load size ranges and split the loads that are greater than the vehicle capacity using different ”splitting” strategy, in term of how much amount to split from the original load to form a new load, and find that there seem to be no optimal ”splitting” strategy, which can assure the best quality of solutions using the metaheuristic developed in the dissertation

Lecturer receptivity to a major educational change in the context of planned change at Rajabhats in Thailand

In accordance with the National Education Act of t999, the educational system in Thailand was changed after 1999, the largest educational change in Thailand in 50 years. The achievable aims of the change were divided into eight main aspects covering, primary, secondary and higher education. These were: (1) ensuring access to basic education for all; (2) reform of the curriculum and learning processes; (3) encouraging participation and partnership in education; (4) restructuring of educational administration; (5) enhancing educational standards and quality assurance; (6) reform of teachers; faculty staff, and educational personnel; (7) mobilisation of resources and investment for education; and (8) utilisation of technologies for education. This study focuses on higher education and aims to: (1) investigate lecturer receptivity to the major change, in the context of planned educational change at Rajabhat Universities, (2) investigate the relationships between lecturer receptivity, and nine aspects to the change, and (3) investigate why Thai lecturers at Rajabhats hold the attitudes that they do. Lecturer receptivity was conceptualised as composed of nine aspects jointly influencing receptivity. They were: (I) attitude to the change in comparison with the previous system, (2) prac1icality in the classroom, (3) alleviation of concerns, (4) learning about the change, (5) participation in decision making.(6) personal cost appraisal, (7) collaboration with other lecturers, (8) opportunities for lecturer improvement, and (9) perceived value for students. For each aspect lecturers would have developed expectations that would, in part, influence their behaviours, and their receptivity to the change

Daily Eastern News: February 29, 1996

https://thekeep.eiu.edu/den_1996_feb/1017/thumbnail.jp

Towards a unified analysis of the syntax and semantics of get constructions

PhDThis thesis develops a novel, unified, syntactic and semantic analysis for a range of get constructions including those with adjectival, prepositional and verbal complements. There are two reasons to believe that such an approach is justified. First, the relevant get constructions demonstrate similar semantic characteristics across complement types, e.g. the presence of Cause (in the sense of Pylkkanen 2008), leading to an obligatorily resultative change-of-state interpretation. Second, the range of constructions display syntactic similarities: for each get construction with no external argument there is a corresponding construction with an external argument; and all of the relevant get constructions take a predicative small clause complement. The approach defended here utilises a formal syntactic and semantic framework to propose an analysis in which get is interpreted as a causative functional head which takes a PredP complement whose function is to add a Holder argument to the property expression in its complement (Bowers 1993, Adger and Ramchand 2003). At this point one of two things may occur. Either (i), the Holder argument raises to the sentential subject position, or (ii), it remains in-situ and an argument external to the causative head is introduced, and then raised to subject position. The thesis shows that, contra Pylkk¨anen 2008, and unlike any other English constructions, get constructions may project Cause without necessarily ‘bundling’ it together with Voice in the syntax. The resulting claims impact on topics in theoretical linguistics as varied as predication, causation, reflexivity and binding, property theory and passivisation, and hold consequences for the nature of the syntax semantics interface

Daily Eastern News: February 29, 1996

https://thekeep.eiu.edu/den_1996_feb/1017/thumbnail.jp