Parallelism with limited nondeterminism

Computational complexity theory studies which computational problems can be solved with limited access to resources. The past fifty years have seen a focus on the relationship between intractable problems and efficient algorithms. However, the relationship between inherently sequential problems and highly parallel algorithms has not been as well studied. Are there efficient but inherently sequential problems that admit some relaxed form of highly parallel algorithm? In this dissertation, we develop the theory of structural complexity around this relationship for three common types of computational problems. Specifically, we show tradeoffs between time, nondeterminism, and parallelizability. By clearly defining the notions and complexity classes that capture our intuition for parallelizable and sequential problems, we create a comprehensive framework for rigorously proving parallelizability and non-parallelizability of computational problems. This framework provides the means to prove whether otherwise tractable problems can be effectively parallelized, a need highlighted by the current growth of multiprocessor systems. The views adopted by this dissertation—alternate approaches to solving sequential problems using approximation, limited nondeterminism, and parameterization—can be applied practically throughout computer science

Scalable optimization algorithms for recommender systems

Recommender systems have now gained significant popularity and been widely used in many e-commerce applications. Predicting user preferences is a key step to providing high quality recommendations. In practice, however, suggestions made to users must not only consider user preferences in isolation; a good recommendation engine also needs to account for certain constraints. For instance, an online video rental that suggests multimedia items (e.g., DVDs) to its customers should consider the availability of DVDs in stock to reduce customer waiting times for accepted recommendations. Moreover, every user should receive a small but sufficient number of suggestions that the user is likely to be interested in. This thesis aims to develop and implement scalable optimization algorithms that can be used (but are not restricted) to generate recommendations satisfying certain objectives and constraints like the ones above. State-of-the-art approaches lack efficiency and/or scalability in coping with large real-world instances, which may involve millions of users and items. First, we study large-scale matrix completion in the context of collaborative filtering in recommender systems. For such problems, we propose a set of novel shared-nothing algorithms which are designed to run on a small cluster of commodity nodes and outperform alternative approaches in terms of efficiency, scalability, and memory footprint. Next, we view our recommendation task as a generalized matching problem, and propose the first distributed solution for solving such problems at scale. Our algorithm is designed to run on a small cluster of commodity nodes (or in a MapReduce environment) and has strong approximation guarantees. Our matching algorithm relies on linear programming. To this end, we present an efficient distributed approximation algorithm for mixed packing-covering linear programs, a simple but expressive subclass of linear programs. Our approximation algorithm requires a poly-logarithmic number of passes over the input, is simple, and well-suited for parallel processing on GPUs, in shared-memory architectures, as well as on a small cluster of commodity nodes.Empfehlungssysteme haben eine beachtliche Popularität erreicht und werden in zahlreichen E-Commerce Anwendungen eingesetzt. Entscheidend für die Generierung hochqualitativer Empfehlungen ist die Vorhersage von Nutzerpräferenzen. Jedoch sollten in der Praxis nicht nur Vorschläge auf Basis von Nutzerpräferenzen gegeben werden, sondern gute Empfehlungssysteme müssen auch bestimmte Nebenbedingungen berücksichtigen. Zum Beispiel sollten online Videoverleihfirmen, welche ihren Kunden multimediale Produkte (z.B. DVDs) vorschlagen, die Verfügbarkeit von vorrätigen DVDs beachten, um die Wartezeit der Kunden für angenommene Empfehlungen zu reduzieren. Darüber hinaus sollte jeder Kunde eine kleine, aber ausreichende Anzahl an Vorschlägen erhalten, an denen er interessiert sein könnte. Diese Arbeit strebt an skalierbare Optimierungsalgorithmen zu entwickeln und zu implementieren, die (unter anderem) eingesetzt werden können Empfehlungen zu generieren, welche weitere Zielvorgaben und Restriktionen einhalten. Derzeit existierenden Ansätzen mangelt es an Effizienz und/oder Skalierbarkeit im Umgang mit sehr großen, durchaus realen Datensätzen von, beispielsweise Millionen von Nutzern und Produkten. Zunächst analysieren wir die Vervollständigung großskalierter Matrizen im Kontext von kollaborativen Filtern in Empfehlungssystemen. Für diese Probleme schlagen wir verschiedene neue, verteilte Algorithmen vor, welche konzipiert sind auf einer kleinen Anzahl von gängigen Rechnern zu laufen. Zudem können sie alternative Ansätze hinsichtlich der Effizienz, Skalierbarkeit und benötigten Speicherkapazität überragen. Als Nächstes haben wir die Empfehlungsproblematik als ein generalisiertes Zuordnungsproblem betrachtet und schlagen daher die erste verteilte Lösung für großskalierte Zuordnungsprobleme vor. Unser Algorithmus funktioniert auf einer kleinen Gruppe von gängigen Rechnern (oder in einem MapReduce-Programmierungsmodel) und erzielt gute Approximationsgarantien. Unser Zuordnungsalgorithmus beruht auf linearer Programmierung. Daher präsentieren wir einen effizienten, verteilten Approximationsalgorithmus für vermischte lineare Packungs- und Überdeckungsprobleme, eine einfache aber expressive Unterklasse der linearen Programmierung. Unser Algorithmus benötigt eine polylogarithmische Anzahl an Scans der Eingabedaten. Zudem ist er einfach und sehr gut geeignet für eine parallele Verarbeitung mithilfe von Grafikprozessoren, unter einer gemeinsam genutzten Speicherarchitektur sowie auf einer kleinen Gruppe von gängigen Rechnern

The Parallel complexity of positive linear programming

In this paper we study the parallel complexity of Positive Linear Programming (PLP), i.e. the special case of Linear Programming in packing/covering form where the input constraint matrix and constraint vector consist entirely of positive entries. We show that the problem of exactly solving PLP is P-complete. Luby and Nisan gave an NC approximation algorithm for PLP, and their algorithm can be used to approximate the size of the largest matching in bipartite graphs, or to approximate the size of the set cover to within a factor $(1+epsilon) ln Delta$, where $Delta$ is the maximum degree in the set system. Trevisan used positive linear programming in combination with Luby and Nisan's algorithm to obtain an NC $(3/4-epsilon)$-approximate algorithm for Max SAT. An important implication of our result is that, by using the Linear Programming technique, we cannot exactly compute in NC the cardinality of Maximum Matching in bipartite graphs or finding a $(ln Delta)$-approximation for Minimum Set Cover, or a 3/4-approximation of an instance of Maximum SAT, unless P=NC

The Parallel complexity of positive linear programming

In this paper we study the parallel complexity of Positive Linear Programming (PLP), i.e. the special case of Linear Programming in packing/covering form where the input constraint matrix and constraint vector consist entirely of positive entries. We show that the problem of exactly solving PLP is P-complete. Luby and Nisan gave an NC approximation algorithm for PLP, and their algorithm can be used to approximate the size of the largest matching in bipartite graphs, or to approximate the size of the set cover to within a factor $(1+epsilon) ln Delta$, where $Delta$ is the maximum degree in the set system. Trevisan used positive linear programming in combination with Luby and Nisan's algorithm to obtain an NC $(3/4-epsilon)$-approximate algorithm for Max SAT. An important implication of our result is that, by using the Linear Programming technique, we cannot exactly compute in NC the cardinality of Maximum Matching in bipartite graphs or finding a $(ln Delta)$-approximation for Minimum Set Cover, or a 3/4-approximation of an instance of Maximum SAT, unless P=NC.Postprint (published version

The Parallel Complexity of Positive Linear Programming

In this paper we study the parallel complexity of Positive Linear Programming (PLP), i.e. the special case of Linear Programming in packing/covering form where the input constraint matrix and constraint vector consist entirely of positive entries. We show that the problem of exactly solving PLP is P-complete

The Parallel complexity of positive linear programming

In this paper we study the parallel complexity of Positive Linear Programming (PLP), i.e. the special case of Linear Programming in packing/covering form where the input constraint matrix and constraint vector consist entirely of positive entries. We show that the problem of exactly solving PLP is P-complete. Luby and Nisan gave an NC approximation algorithm for PLP, and their algorithm can be used to approximate the size of the largest matching in bipartite graphs, or to approximate the size of the set cover to within a factor $(1+epsilon) ln Delta$, where $Delta$ is the maximum degree in the set system. Trevisan used positive linear programming in combination with Luby and Nisan's algorithm to obtain an NC $(3/4-epsilon)$-approximate algorithm for Max SAT. An important implication of our result is that, by using the Linear Programming technique, we cannot exactly compute in NC the cardinality of Maximum Matching in bipartite graphs or finding a $(ln Delta)$-approximation for Minimum Set Cover, or a 3/4-approximation of an instance of Maximum SAT, unless P=NC

The parallel complexity of positive linear programming

In this paper we study the parallel complexity of Positive Linear Programming (PLP), i.e. the special case of Linear Programming in packing/covering form where the input constraint matrix and constraint vector consist entirely of positive entries. We show that the problem of exactly solving PLP is P-complete.Peer Reviewe

The Parallel Complexity of Positive Linear Programming

In this paper we study the parallel complexity of Positive Linear Programming (PLP), i.e. the special case of Linear Programming in packing/covering form where the input constraint matrix and constraint vector consist entirely of positive entries. We show that the problem of exactly solving PLP is P-complete. Luby and Nisan gave an NC approximation algorithm for PLP, and their algorithm can be used to approximate the size of the largest matching in bipartite graphs, or to approximate the size of the set cover to within a factor (1+ ffl) ln \Delta, where \Delta is the maximum degree in the set system. Trevisan used positive linear programming in combination with Luby and Nisan's algorithm to obtain an NC (3=4 \Gamma ffl)-approximate algorithm for Max SAT. An important implication of our result is that, by using the Linear Programming technique, we cannot exactly compute in NC the cardinality of Maximum Matching in bipartite graphs or finding a (ln \Delta)-approximation for Minimum Set C..