Gated networks: an inventory

Gated networks are networks that contain gating connections, in which the outputs of at least two neurons are multiplied. Initially, gated networks were used to learn relationships between two input sources, such as pixels from two images. More recently, they have been applied to learning activity recognition or multi-modal representations. The aims of this paper are threefold: 1) to explain the basic computations in gated networks to the non-expert, while adopting a standpoint that insists on their symmetric nature. 2) to serve as a quick reference guide to the recent literature, by providing an inventory of applications of these networks, as well as recent extensions to the basic architecture. 3) to suggest future research directions and applications.Comment: Unpublished manuscript, 17 page

Data-Augmented Structure-Property Mapping for Accelerating Computational Design of Advanced Material Systems

abstract: Advanced material systems refer to materials that are comprised of multiple traditional constituents but complex microstructure morphologies, which lead to their superior properties over conventional materials. This dissertation is motivated by the grand challenge in accelerating the design of advanced material systems through systematic optimization with respect to material microstructures or processing settings. While optimization techniques have mature applications to a large range of engineering systems, their application to material design meets unique challenges due to the high dimensionality of microstructures and the high costs in computing process-structure-property (PSP) mappings. The key to addressing these challenges is the learning of material representations and predictive PSP mappings while managing a small data acquisition budget. This dissertation thus focuses on developing learning mechanisms that leverage context-specific meta-data and physics-based theories. Two research tasks will be conducted: In the first, we develop a statistical generative model that learns to characterize high-dimensional microstructure samples using low-dimensional features. We improve the data efficiency of a variational autoencoder by introducing a morphology loss to the training. We demonstrate that the resultant microstructure generator is morphology-aware when trained on a small set of material samples, and can effectively constrain the microstructure space during material design. In the second task, we investigate an active learning mechanism where new samples are acquired based on their violation to a theory-driven constraint on the physics-based model. We demonstrate using a topology optimization case that while data acquisition through the physics-based model is often expensive (e.g., obtaining microstructures through simulation or optimization processes), the evaluation of the constraint can be far more affordable (e.g., checking whether a solution is optimal or equilibrium). We show that this theory-driven learning algorithm can lead to much improved learning efficiency and generalization performance when such constraints can be derived. The outcomes of this research is a better understanding of how physics knowledge about material systems can be integrated into machine learning frameworks, in order to achieve more cost-effective and reliable learning of material representations and predictive models, which are essential to accelerate computational material design.Dissertation/ThesisDoctoral Dissertation Mechanical Engineering 201

Domain value mutation and other techniques for constraint satisfaction problems

The term Constraint Satisfaction Problem (CSP) refers to a class of NP-complete problems, a collection of difficult problems for which no fast solution is known. The standard definition of a CSP involves variables, values, and constraints: each variable must be assigned a value from a designated group of possible values (also known as the variable’s domain), while a constraint on a set of variables indicates permissible combinations of values for these variables. Given a CSP, an important objective is to query whether it has a solution — an assignment of each variable to a value such that all constraints are satisfied. Solving a CSP usually requires chronological backtracking search that interleaves variable assignments with various kinds of inferences in order to reduce the search space. This dissertation comprises two parts. The first part deals with a modification of the classical CSP model that allows a value to be broken up and multiple values to be combined. The second part deals with generalized arc consistency algorithms. Both parts share a common theme in that extensional constraints --‐ the most basic expression possible for constraints --- play the central role. Despite being an important class, extensional constraints have received much less attention recently as most efforts have been channelled toward identifying new types of specialized constraints and coming up with corresponding algorithms. Regardless, improvements to algorithms for extensional constraints are more fundamental. This dissertation will attempt to improve existing techniques and algorithms for extensional constraints by examining them critically from the bottom up and approaching them from a novel direction

Physics-Constrained Neural Network for the Analysis and Feature-Based Optimization of Woven Composites

Woven composites are produced by interlacing warp and weft fibers in a pattern or weave style. By changing the pattern or material, the mechanical properties of woven composites can be significantly changed; however, the role of woven composite architecture (pattern, material) on the mechanical properties is not well understood. In this paper, we explore the relationship between woven composite architectures (weave pattern, weave material sequence) and the corresponding modulus through our proposed Physics-Constrained Neural Network (PCNN). Furthermore, we apply statistical learning methods to optimize the woven composite architecture to improve mechanical responses. Our results show that PCNN can effectively predict woven architecture for the desired modulus with much higher accuracy than several baseline models. PCNN can be further combined with feature-based optimization to determine the optimal woven composite architecture at the initial design stage. In addition to relating woven composite architecture to its mechanical responses, our research also provides an in-depth understanding of how architectural features govern mechanical responses. We anticipate our proposed frameworks will primarily facilitate the woven composite analysis and optimization process and be a starting point to introduce Physics knowledge-guided Neural Networks into the complex structural analysis

A survey of visual preprocessing and shape representation techniques

Many recent theories and methods proposed for visual preprocessing and shape representation are summarized. The survey brings together research from the fields of biology, psychology, computer science, electrical engineering, and most recently, neural networks. It was motivated by the need to preprocess images for a sparse distributed memory (SDM), but the techniques presented may also prove useful for applying other associative memories to visual pattern recognition. The material of this survey is divided into three sections: an overview of biological visual processing; methods of preprocessing (extracting parts of shape, texture, motion, and depth); and shape representation and recognition (form invariance, primitives and structural descriptions, and theories of attention)