Linearized Alternating Direction Method with Parallel Splitting and Adaptive Penalty for Separable Convex Programs in Machine Learning

Many problems in machine learning and other fields can be (re)for-mulated as linearly constrained separable convex programs. In most of the cases, there are multiple blocks of variables. However, the traditional alternating direction method (ADM) and its linearized version (LADM, obtained by linearizing the quadratic penalty term) are for the two-block case and cannot be naively generalized to solve the multi-block case. So there is great demand on extending the ADM based methods for the multi-block case. In this paper, we propose LADM with parallel splitting and adaptive penalty (LADMPSAP) to solve multi-block separable convex programs efficiently. When all the component objective functions have bounded subgradients, we obtain convergence results that are stronger than those of ADM and LADM, e.g., allowing the penalty parameter to be unbounded and proving the sufficient and necessary conditions} for global convergence. We further propose a simple optimality measure and reveal the convergence rate of LADMPSAP in an ergodic sense. For programs with extra convex set constraints, with refined parameter estimation we devise a practical version of LADMPSAP for faster convergence. Finally, we generalize LADMPSAP to handle programs with more difficult objective functions by linearizing part of the objective function as well. LADMPSAP is particularly suitable for sparse representation and low-rank recovery problems because its subproblems have closed form solutions and the sparsity and low-rankness of the iterates can be preserved during the iteration. It is also highly parallelizable and hence fits for parallel or distributed computing. Numerical experiments testify to the advantages of LADMPSAP in speed and numerical accuracy.Comment: Preliminary version published on Asian Conference on Machine Learning 201

An optimal variance estimate in stochastic homogenization of discrete elliptic equations

We consider a discrete elliptic equation on the $d$-dimensional lattice $\mathbb{Z}^d$ with random coefficients $A$ of the simplest type: they are identically distributed and independent from edge to edge. On scales large w.r.t. the lattice spacing (i.e., unity), the solution operator is known to behave like the solution operator of a (continuous) elliptic equation with constant deterministic coefficients. This symmetric ``homogenized'' matrix $A_{\mathrm {hom}}=a_{\mathrm {hom}}\operatorname {Id}$ is characterized by $\xi\cdot A_{\mathrm {hom}}\xi=\langle(\xi+\nabla\phi )\cdot A(\xi+\nabla\phi)\rangle$ for any direction $\xi\in\mathbb {R}^d$, where the random field $\phi$ (the ``corrector'') is the unique solution of $-\nabla^*\cdot A(\xi+\nabla\phi)=0$ such that $\phi(0)=0$, $\nabla\phi$ is stationary and $\langle\nabla\phi\rangle=0$, $\langle\cdot\rangle$ denoting the ensemble average (or expectation). It is known (``by ergodicity'') that the above ensemble average of the energy density $\mathcal {E}=(\xi+\nabla\phi)\cdot A(\xi+\nabla\phi)$, which is a stationary random field, can be recovered by a system average. We quantify this by proving that the variance of a spatial average of $\mathcal {E}$ on length scales $L$ satisfies the optimal estimate, that is, $\operatorname {var}[\sum \mathcal {E}\eta_L]\lesssim L^{-d}$, where the averaging function [i.e., $\sum\eta_L=1$, $\operatorname {supp}(\eta_L)\subset\{|x|\le L\}$] has to be smooth in the sense that $|\nabla\eta_L|\lesssim L^{-1-d}$. In two space dimensions (i.e., $d=2$), there is a logarithmic correction. This estimate is optimal since it shows that smooth averages of the energy density $\mathcal {E}$ decay in $L$ as if $\mathcal {E}$ would be independent from edge to edge (which it is not for $d>1$). This result is of practical significance, since it allows to estimate the dominant error when numerically computing $a_{\mathrm {hom}}$.Comment: Published in at http://dx.doi.org/10.1214/10-AOP571 the Annals of Probability (http://www.imstat.org/aop/) by the Institute of Mathematical Statistics (http://www.imstat.org

Deaf, Dumb, and Chatting Robots, Enabling Distributed Computation and Fault-Tolerance Among Stigmergic Robot

We investigate ways for the exchange of information (explicit communication) among deaf and dumb mobile robots scattered in the plane. We introduce the use of movement-signals (analogously to flight signals and bees waggle) as a mean to transfer messages, enabling the use of distributed algorithms among the robots. We propose one-to-one deterministic movement protocols that implement explicit communication. We first present protocols for synchronous robots. We begin with a very simple coding protocol for two robots. Based on on this protocol, we provide one-to-one communication for any system of n \geq 2 robots equipped with observable IDs that agree on a common direction (sense of direction). We then propose two solutions enabling one-to-one communication among anonymous robots. Since the robots are devoid of observable IDs, both protocols build recognition mechanisms using the (weak) capabilities offered to the robots. The first protocol assumes that the robots agree on a common direction and a common handedness (chirality), while the second protocol assumes chirality only. Next, we show how the movements of robots can provide implicit acknowledgments in asynchronous systems. We use this result to design asynchronous one-to-one communication with two robots only. Finally, we combine this solution with the schemes developed in synchronous settings to fit the general case of asynchronous one-to-one communication among any number of robots. Our protocols enable the use of distributing algorithms based on message exchanges among swarms of Stigmergic robots. Furthermore, they provides robots equipped with means of communication to overcome faults of their communication device

Designing the Sakai Open Academic Environment: A distributed cognition account of the design of a large scale software system

Social accounts of technological change make the flexibility and openness of interpretations the starting point of an argument against technological determinism. They suggest that technological change unfolds in the semantic domain, but they focus on the social processes around the interpretations of new technologies, and do not address the conceptual processes of change in interpretations. The dissertation presents an empirically grounded case study of the design process of an open-source online software platform based on the framework of distributed cognition to argue that the cognitive perspective is needed for understanding innovation in software, because it allows us to describe the reflexive and expansive contribution of conceptual processes to new software and the significance of professional epistemic practices in framing the direction of innovation. The framework of distributed cognition brings the social and cognitive perspectives together on account of its understanding of conceptual processes as distributed over time, among people, and between humans and artifacts. The dissertation argues that an evolving open-source software landscape became translated into the open-ended local design space of a new software project in a process of infrastructural implosion, and the design space prompted participants to outline and pursue epistemic strategies of sense-making and learning about the contexts of use. The result was a process of conceptual modeling, which resulted in a conceptually novel user interface. Prototyping professional practices of user-centered design lent directionality to this conceptual process in terms of a focus on individual activities with the user interface. Social approaches to software design under the broad umbrella of human-centered computing have been seeking to inform the design on the basis of empirical contributions about a social context. The analysis has shown that empirical engagement with the contexts of use followed from conceptual modeling, and concern about real world contexts was aligned with the user-centered direction that design was taking. I also point out a social-technical gap in the design process in connection with the repeated performance challenges that the platform was facing, and describe the possibility of a social-technical imagination.Ph.D

Minimal chordal sense of direction and circulant graphs

A sense of direction is an edge labeling on graphs that follows a globally consistent scheme and is known to considerably reduce the complexity of several distributed problems. In this paper, we study a particular instance of sense of direction, called a chordal sense of direction (CSD). In special, we identify the class of k-regular graphs that admit a CSD with exactly k labels (a minimal CSD). We prove that connected graphs in this class are Hamiltonian and that the class is equivalent to that of circulant graphs, presenting an efficient (polynomial-time) way of recognizing it when the graphs' degree k is fixed

A Decentralized Mobile Computing Network for Multi-Robot Systems Operations

Collective animal behaviors are paradigmatic examples of fully decentralized operations involving complex collective computations such as collective turns in flocks of birds or collective harvesting by ants. These systems offer a unique source of inspiration for the development of fault-tolerant and self-healing multi-robot systems capable of operating in dynamic environments. Specifically, swarm robotics emerged and is significantly growing on these premises. However, to date, most swarm robotics systems reported in the literature involve basic computational tasks---averages and other algebraic operations. In this paper, we introduce a novel Collective computing framework based on the swarming paradigm, which exhibits the key innate features of swarms: robustness, scalability and flexibility. Unlike Edge computing, the proposed Collective computing framework is truly decentralized and does not require user intervention or additional servers to sustain its operations. This Collective computing framework is applied to the complex task of collective mapping, in which multiple robots aim at cooperatively map a large area. Our results confirm the effectiveness of the cooperative strategy, its robustness to the loss of multiple units, as well as its scalability. Furthermore, the topology of the interconnecting network is found to greatly influence the performance of the collective action.Comment: Accepted for Publication in Proc. 9th IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conferenc