Noise-Tolerant Parallel Learning of Geometric Concepts

We present several efficient parallel algorithms for PAC-learning geometric concepts in a constant-dimensional space. The algorithms are robust even against malicious classification noise of any rate less than 1/2. We first give an efficient noise-tolerant parallel algorithm to PAC-learn the class of geometric concepts defined by a polynomial number of (d-1)-dimensional hyperplanes against an arbitrary distribution where each hyperplane has a slope from a set of known slopes. We then describe how boosting techniques can be used so that our algorithms\u27 dependence on {GREEK LETTER} and {DELTA} does not depend on d. Next we give an efficient noise-tolerant parallel algorithm to PAC-learn the class of geometric concepts defined by a polynomial number of (d-1)-dimensional hyperplanes (of unrestricted slopes) against a uniform distribution. We then show how to extend our algorithm to learn this class against any (unknown) product distribution. Next we defined a complexity measure of any set S of (d-1)-dimensional surfaces that we call the variant of S and prove that the class of geometric concepts defined by surfaces of polynomial variant can be efficienty learned in parallel under a product distribution (even under malicious classification noise). Furthermore, we show that the VC-dimension of the class of geometric concepts defined by a set of surfaces S of variant v is at least v. Finally, we give an efficient, parallel, noise-tolerant algorithm to PAC-learn any geometric concept defined by a set S of (d-1)-dimensional surfaces of polynomial area under a uniform distribution

Quantum machine learning: a classical perspective

Recently, increased computational power and data availability, as well as algorithmic advances, have led machine learning techniques to impressive results in regression, classification, data-generation and reinforcement learning tasks. Despite these successes, the proximity to the physical limits of chip fabrication alongside the increasing size of datasets are motivating a growing number of researchers to explore the possibility of harnessing the power of quantum computation to speed-up classical machine learning algorithms. Here we review the literature in quantum machine learning and discuss perspectives for a mixed readership of classical machine learning and quantum computation experts. Particular emphasis will be placed on clarifying the limitations of quantum algorithms, how they compare with their best classical counterparts and why quantum resources are expected to provide advantages for learning problems. Learning in the presence of noise and certain computationally hard problems in machine learning are identified as promising directions for the field. Practical questions, like how to upload classical data into quantum form, will also be addressed.Comment: v3 33 pages; typos corrected and references adde

Empiricism without Magic: Transformational Abstraction in Deep Convolutional Neural Networks

In artificial intelligence, recent research has demonstrated the remarkable potential of Deep Convolutional Neural Networks (DCNNs), which seem to exceed state-of-the-art performance in new domains weekly, especially on the sorts of very difficult perceptual discrimination tasks that skeptics thought would remain beyond the reach of artificial intelligence. However, it has proven difficult to explain why DCNNs perform so well. In philosophy of mind, empiricists have long suggested that complex cognition is based on information derived from sensory experience, often appealing to a faculty of abstraction. Rationalists have frequently complained, however, that empiricists never adequately explained how this faculty of abstraction actually works. In this paper, I tie these two questions together, to the mutual benefit of both disciplines. I argue that the architectural features that distinguish DCNNs from earlier neural networks allow them to implement a form of hierarchical processing that I call “transformational abstraction”. Transformational abstraction iteratively converts sensory-based representations of category exemplars into new formats that are increasingly tolerant to “nuisance variation” in input. Reflecting upon the way that DCNNs leverage a combination of linear and non-linear processing to efficiently accomplish this feat allows us to understand how the brain is capable of bi-directional travel between exemplars and abstractions, addressing longstanding problems in empiricist philosophy of mind. I end by considering the prospects for future research on DCNNs, arguing that rather than simply implementing 80s connectionism with more brute-force computation, transformational abstraction counts as a qualitatively distinct form of processing ripe with philosophical and psychological significance, because it is significantly better suited to depict the generic mechanism responsible for this important kind of psychological processing in the brain

Modeling, Control and Estimation of Reconfigurable Cable Driven Parallel Robots

The motivation for this thesis was to develop a cable-driven parallel robot (CDPR) as part of a two-part robotic device for concrete 3D printing. This research addresses specific research questions in this domain, chiefly, to present advantages offered by the addition of kinematic redundancies to CDPRs. Due to the natural actuation redundancy present in a fully constrained CDPR, the addition of internal mobility offers complex challenges in modeling and control that are not often encountered in literature. This work presents a systematic analysis of modeling such kinematic redundancies through the application of reciprocal screw theory (RST) and Lie algebra while further introducing specific challenges and drawbacks presented by cable driven actuators. It further re-contextualizes well-known performance indices such as manipulability, wrench closure quality, and the available wrench set for application with reconfigurable CDPRs. The existence of both internal redundancy and static redundancy in the joint space offers a large subspace of valid solutions that can be condensed through the selection of appropriate objective priorities, constraints or cost functions. Traditional approaches to such redundancy resolution necessitate computationally expensive numerical optimization. The control of both kinematic and actuation redundancies requires cascaded control frameworks that cannot easily be applied towards real-time control. The selected cost functions for numerical optimization of rCDPRs can be globally (and sometimes locally) non-convex. In this work we present two applied examples of redundancy resolution control that are unique to rCDPRs. In the first example, we maximize the directional wrench ability at the end-effector while minimizing the joint torque requirement by utilizing the fitness of the available wrench set as a constraint over wrench feasibility. The second example focuses on directional stiffness maximization at the end-effector through a variable stiffness module (VSM) that partially decouples the tension and stiffness. The VSM introduces an additional degrees of freedom to the system in order to manipulate both reconfigurability and cable stiffness independently. The controllers in the above examples were designed with kinematic models, but most CDPRs are highly dynamic systems which can require challenging feedback control frameworks. An approach to real-time dynamic control was implemented in this thesis by incorporating a learning-based frameworks through deep reinforcement learning. Three approaches to rCDPR training were attempted utilizing model-free TD3 networks. Robustness and safety are critical features for robot development. One of the main causes of robot failure in CDPRs is due to cable breakage. This not only causes dangerous dynamic oscillations in the workspace, but also leads to total robot failure if the controllability (due to lack of cables) is lost. Fortunately, rCDPRs can be utilized towards failure tolerant control for task recovery. The kinematically redundant joints can be utilized to help recover the lost degrees of freedom due to cable failure. This work applies a Multi-Model Adaptive Estimation (MMAE) framework to enable online and automatic objective reprioritization and actuator retasking. The likelihood of cable failure(s) from the estimator informs the mixing of the control inputs from a bank of feedforward controllers. In traditional rigid body robots, safety procedures generally involve a standard emergency stop procedure such as actuator locking. Due to the flexibility of cable links, the dynamic oscillations of the end-effector due to cable failure must be actively dampened. This work incorporates a Linear Quadratic Regulator (LQR) based feedback stabilizer into the failure tolerant control framework that works to stabilize the non-linear system and dampen out these oscillations. This research contributes to a growing, but hitherto niche body of work in reconfigurable cable driven parallel manipulators. Some outcomes of the multiple engineering design, control and estimation challenges addressed in this research warrant further exploration and study that are beyond the scope of this thesis. This thesis concludes with a thorough discussion of the advantages and limitations of the presented work and avenues for further research that may be of interest to continuing scholars in the community

Deep Neural Networks Rival the Representation of Primate IT Cortex for Core Visual Object Recognition

The primate visual system achieves remarkable visual object recognition performance even in brief presentations and under changes to object exemplar, geometric transformations, and background variation (a.k.a. core visual object recognition). This remarkable performance is mediated by the representation formed in inferior temporal (IT) cortex. In parallel, recent advances in machine learning have led to ever higher performing models of object recognition using artificial deep neural networks (DNNs). It remains unclear, however, whether the representational performance of DNNs rivals that of the brain. To accurately produce such a comparison, a major difficulty has been a unifying metric that accounts for experimental limitations such as the amount of noise, the number of neural recording sites, and the number trials, and computational limitations such as the complexity of the decoding classifier and the number of classifier training examples. In this work we perform a direct comparison that corrects for these experimental limitations and computational considerations. As part of our methodology, we propose an extension of "kernel analysis" that measures the generalization accuracy as a function of representational complexity. Our evaluations show that, unlike previous bio-inspired models, the latest DNNs rival the representational performance of IT cortex on this visual object recognition task. Furthermore, we show that models that perform well on measures of representational performance also perform well on measures of representational similarity to IT and on measures of predicting individual IT multi-unit responses. Whether these DNNs rely on computational mechanisms similar to the primate visual system is yet to be determined, but, unlike all previous bio-inspired models, that possibility cannot be ruled out merely on representational performance grounds.Comment: 35 pages, 12 figures, extends and expands upon arXiv:1301.353

Parallel Architectures for Planetary Exploration Requirements (PAPER)

The Parallel Architectures for Planetary Exploration Requirements (PAPER) project is essentially research oriented towards technology insertion issues for NASA's unmanned planetary probes. It was initiated to complement and augment the long-term efforts for space exploration with particular reference to NASA/LaRC's (NASA Langley Research Center) research needs for planetary exploration missions of the mid and late 1990s. The requirements for space missions as given in the somewhat dated Advanced Information Processing Systems (AIPS) requirements document are contrasted with the new requirements from JPL/Caltech involving sensor data capture and scene analysis. It is shown that more stringent requirements have arisen as a result of technological advancements. Two possible architectures, the AIPS Proof of Concept (POC) configuration and the MAX Fault-tolerant dataflow multiprocessor, were evaluated. The main observation was that the AIPS design is biased towards fault tolerance and may not be an ideal architecture for planetary and deep space probes due to high cost and complexity. The MAX concepts appears to be a promising candidate, except that more detailed information is required. The feasibility for adding neural computation capability to this architecture needs to be studied. Key impact issues for architectural design of computing systems meant for planetary missions were also identified

Real-time support for high performance aircraft operation

The feasibility of real-time processing schemes using artificial neural networks (ANNs) is investigated. A rationale for digital neural nets is presented and a general processor architecture for control applications is illustrated. Research results on ANN structures for real-time applications are given. Research results on ANN algorithms for real-time control are also shown