Generative-Discriminative Complementary Learning

Majority of state-of-the-art deep learning methods are discriminative approaches, which model the conditional distribution of labels given inputs features. The success of such approaches heavily depends on high-quality labeled instances, which are not easy to obtain, especially as the number of candidate classes increases. In this paper, we study the complementary learning problem. Unlike ordinary labels, complementary labels are easy to obtain because an annotator only needs to provide a yes/no answer to a randomly chosen candidate class for each instance. We propose a generative-discriminative complementary learning method that estimates the ordinary labels by modeling both the conditional (discriminative) and instance (generative) distributions. Our method, we call Complementary Conditional GAN (CCGAN), improves the accuracy of predicting ordinary labels and can generate high-quality instances in spite of weak supervision. In addition to the extensive empirical studies, we also theoretically show that our model can retrieve the true conditional distribution from the complementarily-labeled data

Causal and causally-inspired learning

University of Technology Sydney. Faculty of Engineering and Information Technology.A main goal of statistics and machine learning is to discover statistical dependencies between random variables, and these dependencies will be used to perform predictions on future observations. However, many scientific investigations involve causal predictions, the aim of which is to infer how the data generating system should behave under changing conditions, for example, changes induced by external interventions. To perform causal predictions, we need both statistical dependencies as well as causal structures to determine the behaviour of the system. The standard way to identify causal structures is to use randomized controlled experiments. However, conducting these experiments is usually expensive or even impossible in many scenarios. As a consequence, inferring cause and effect relationships from purely observational data, known as causal discovery or causal learning, has drawn much attention. Various causal discovery methods have been proposed in the past decades, including constraint-based methods, structural equation models-based methods, and time series-based methods. Among these methods, time series-based methods, e.g., Granger causality, are relatively well-established as the temporal information excludes the case that effects happen before causes. Many of the existing time series-based methods assume that the data are measured at the right frequency; however, in practice the sampling frequency of the data is often lower than the true causal frequency. In this thesis, we consider learning high-resolution causal relationships at the causal frequency from subsampled time series. Existing methods suffer from the identifiability problems: under the Gaussianity assumption of the data, the solutions are generally not unique. We prove that, however, if the noise terms are non-Gaussian, the underlying model is identifiable from subsampled time series under mild conditions. We then propose an Expectation-Maximization approach and a variational inference approach to recover causal relations from subsampled data. More recently, researchers began to touch upon implications of causal models for machine learning tasks such as semi-supervised learning and domain adaptation. In this thesis, we develop causally-inspired learning methods for domain adaptation in both multi-source and single-source settings. In particular, we use causal models to represent the relationship between the features and labels, and consider possible situations where different modules of the causal model change with the domain. In each situation, we investigate what knowledge is appropriate to transfer and find the optimal target-domain hypothesis. Furthermore, we propose methods to correct distribution shift in the general situation where the marginal distribution of features and conditional distribution of labels given features both change, under the assumption that labels are causes for features. We provide theoretical analysis and empirical evaluation on both synthetic and real-world data to show the effectiveness of our methods

Effects of stochastic tow waviness on stiffness and strength of plain-weave ceramic matrix composites

This article presents the development of a finite element model, which considers stochastic tow waviness using a Markov Chain algorithm and non-linear material properties using Binary Model, to predict the stress–strain and fracture behaviour of plain-weave ceramic matrix composites under uniaxial extension. The stochastic waviness is described by fluctuations in the centroid coordinates of tow positioning. The tow deviations are generated by marching sequentially from one grid point to next along a tow path. The deviations depend only on the deviation of the previous point using a probability transition matrix. A non-linear orthotropic constitutive model was implemented in a commercial finite element code Abaqus using a user-defined subroutine. Two 2 × 2 unit cell models of a plain-weave ceramic matrix composite laminate are created using stochastic tow elements generated by the virtual specimen generator, which was developed on the basis of the Markov Chain algorithm. A comparison has been made between the systematic and stochastic models to assess the effects of stochastic tow waviness on the stiffness and strength of the laminate. The numerical results have been validated by the comparison of predictions with the experimental data. The stochastic model which considers random waviness correlates well with the experimental data

Adaptive Edge-to-Edge Interaction Learning for Point Cloud Analysis

Recent years have witnessed the great success of deep learning on various point cloud analysis tasks, e.g., classification and semantic segmentation. Since point cloud data is sparse and irregularly distributed, one key issue for point cloud data processing is extracting useful information from local regions. To achieve this, previous works mainly extract the points' features from local regions by learning the relation between each pair of adjacent points. However, these works ignore the relation between edges in local regions, which encodes the local shape information. Associating the neighbouring edges could potentially make the point-to-point relation more aware of the local structure and more robust. To explore the role of the relation between edges, this paper proposes a novel Adaptive Edge-to-Edge Interaction Learning module, which aims to enhance the point-to-point relation through modelling the edge-to-edge interaction in the local region adaptively. We further extend the module to a symmetric version to capture the local structure more thoroughly. Taking advantage of the proposed modules, we develop two networks for segmentation and shape classification tasks, respectively. Various experiments on several public point cloud datasets demonstrate the effectiveness of our method for point cloud analysis.Comment: Technical Repor