Learning with Latent Language

The named concepts and compositional operators present in natural language provide a rich source of information about the kinds of abstractions humans use to navigate the world. Can this linguistic background knowledge improve the generality and efficiency of learned classifiers and control policies? This paper aims to show that using the space of natural language strings as a parameter space is an effective way to capture natural task structure. In a pretraining phase, we learn a language interpretation model that transforms inputs (e.g. images) into outputs (e.g. labels) given natural language descriptions. To learn a new concept (e.g. a classifier), we search directly in the space of descriptions to minimize the interpreter's loss on training examples. Crucially, our models do not require language data to learn these concepts: language is used only in pretraining to impose structure on subsequent learning. Results on image classification, text editing, and reinforcement learning show that, in all settings, models with a linguistic parameterization outperform those without

A compositional framework for determining pattern applicability.

The notion of ‘pattern’ originates in the work of Christopher Alexander and, in recent years, patterns have become a popular part of software development. A pattern is defined as a ‘three-part rule’: a relationship between a given context, a recurring system of forces peculiar to that context, and a specific spatial configuration that permits resolution of these forces. In essence, the ‘context’ of a pattern is the whole system under construction and its state in the construction process at the point at which the pattern is being applied. The nature of the context, therefore, changes at every step of the process and this has significant implications for how patterns should be used. Specifically, applying each pattern changes the context by changing the state of the system under construction and creates both a new design problem and a new context for the next pattern to be applied. The next picked pattern must have a certain criteria in order for it to be applied successfully and this is will be determined by the characteristics of the new context just created. The issue of composing pattern sequences is therefore more temporal than it is static and structural (as provided currently via pattern maps). The decision as to which one to use is temporally constrained in the sense that the choice is made only at a particular point in the construction process of some specific system, and may well be determined, or at least further constrained, by the current state of that system. The fundamental research question that is addressed here is: how is this dynamically changing context to be presented to guide pattern applications? In this thesis, a framework is presented to provide a systematic analysis of composition of pattern applications in terms of the properties of their context. Such an approach will reveal the ordering of patterns in space and time dimensions. Examples of composition of pattern applications include: - One pattern contains or generalises another smaller-scale pattern (this will be called in thesis refinement); - Two patterns are complementary, i.e., one pattern needs the other to be applied before (Sequential Order); - Two patterns solve different problems that overlap and coexist on the same level (Parallel Order); - Two patterns solve the same problem in alternative, but equally valid ways (Choice in Order). At the design phase, the framework provides mechanisms for analysing the choice of composition to ensure the correctness of a design or to compare between two different designs or to modify an existing design. This framework describes a pattern's context via a pair of constraints, known as Assumption and Commitment. In general, the Assumption is a constraint placed on the context and the Commitment is what the solution provided by the pattern commits to after the pattern's application. In addition, the thesis provides a set of composition rules that can be applied to aid in the analysis of the application of pattern sequences. The approach is domain independent as it does not depend on the nature of the catalogue from which the patterns originate. The work has been evaluated using various existing patterns from Ian Graham’s web usability (WU) pattern bank and the User Interface (UI) patterns of Welie

Society's Nervous System: Building Effective Government, Energy, and Public Health Systems

Drawing on a unique, multi-year collaboration with the heads of major IT, wireless, hardware, health, and financial firms, as well as the heads of American, EU, and other regulatory organizations, and a variety of NGOs [1,2],I describe the potential for pervasive and mobile sensing and computing over the next decade, and the challenges that will have to be faced in order to realize this potential.United States. Army Research Laboratory (Cooperative Agreement Number W911NF-09-2-0053)United States. Air Force Office of Scientific Research (Award Number FA9550-10-1-0122