Bayesian Structure Learning for Markov Random Fields with a Spike and Slab Prior

In recent years a number of methods have been developed for automatically learning the (sparse) connectivity structure of Markov Random Fields. These methods are mostly based on L1-regularized optimization which has a number of disadvantages such as the inability to assess model uncertainty and expensive cross-validation to find the optimal regularization parameter. Moreover, the model's predictive performance may degrade dramatically with a suboptimal value of the regularization parameter (which is sometimes desirable to induce sparseness). We propose a fully Bayesian approach based on a "spike and slab" prior (similar to L0 regularization) that does not suffer from these shortcomings. We develop an approximate MCMC method combining Langevin dynamics and reversible jump MCMC to conduct inference in this model. Experiments show that the proposed model learns a good combination of the structure and parameter values without the need for separate hyper-parameter tuning. Moreover, the model's predictive performance is much more robust than L1-based methods with hyper-parameter settings that induce highly sparse model structures.Comment: Accepted in the Conference on Uncertainty in Artificial Intelligence (UAI), 201

Efficient and Robust Neuromorphic Computing Design

In recent years, brain inspired neuromorphic computing system (NCS) has been intensively studied in both circuit level and architecture level. NCS has demonstrated remarkable advantages for its high-energy efficiency, extremely compact space occupation and parallel data processing. However, due to the limited hardware resources, severe IR-Drop and process variation problems for synapse crossbar, and limited synapse device resolution, it’s still a great challenge for hardware NCS design to catch up with the fast development of software deep neural networks (DNNs). This dissertation explores model compression and acceleration methods for deep neural networks to save both memory and computation resources for the hardware implementation of DNNs. Firstly, DNNs’ weights quantization work is presented to use three orthogonal methods to learn synapses with one-level precision, namely, distribution-aware quantization, quantization regularization and bias tuning, to make image classification accuracy comparable to the state-ofthe-art. And then a two-step framework named group scissor, including rank clipping and group connection deletion methods, is presented to address the problems on large synapse crossbar consuming and high routing congestion between crossbars. Results show that after applying weights quantization methods, accuracy drop can be well controlled within negligible level for MNIST and CIFAR-10 dataset, compared to an ideal system without quantization. And for the group scissor framework method, crossbar area and routing area could be reduced to 8% (at most) of original size, indicating that the hardware implementation area has been saved a lot. Furthermore, the system scalability has been improved significantly

Radiation pattern prediction for Metasurfaces: A Neural Network based approach

As the current standardization for the 5G networks nears completion, work towards understanding the potential technologies for the 6G wireless networks is already underway. One of these potential technologies for the 6G networks are Reconfigurable Intelligent Surfaces (RISs). They offer unprecedented degrees of freedom towards engineering the wireless channel, i.e., the ability to modify the characteristics of the channel whenever and however required. Nevertheless, such properties demand that the response of the associated metasurface (MSF) is well understood under all possible operational conditions. While an understanding of the radiation pattern characteristics can be obtained through either analytical models or full wave simulations, they suffer from inaccuracy under certain conditions and extremely high computational complexity, respectively. Hence, in this paper we propose a novel neural networks based approach that enables a fast and accurate characterization of the MSF response. We analyze multiple scenarios and demonstrate the capabilities and utility of the proposed methodology. Concretely, we show that this method is able to learn and predict the parameters governing the reflected wave radiation pattern with an accuracy of a full wave simulation (98.8%-99.8%) and the time and computational complexity of an analytical model. The aforementioned result and methodology will be of specific importance for the design, fault tolerance and maintenance of the thousands of RISs that will be deployed in the 6G network environment.Comment: Submitted to IEEE OJ-COM

On discriminative semi-supervised incremental learning with a multi-view perspective for image concept modeling

This dissertation presents the development of a semi-supervised incremental learning framework with a multi-view perspective for image concept modeling. For reliable image concept characterization, having a large number of labeled images is crucial. However, the size of the training set is often limited due to the cost required for generating concept labels associated with objects in a large quantity of images. To address this issue, in this research, we propose to incrementally incorporate unlabeled samples into a learning process to enhance concept models originally learned with a small number of labeled samples. To tackle the sub-optimality problem of conventional techniques, the proposed incremental learning framework selects unlabeled samples based on an expected error reduction function that measures contributions of the unlabeled samples based on their ability to increase the modeling accuracy. To improve the convergence property of the proposed incremental learning framework, we further propose a multi-view learning approach that makes use of multiple features such as color, texture, etc., of images when including unlabeled samples. For robustness to mismatches between training and testing conditions, a discriminative learning algorithm, namely a kernelized maximal- figure-of-merit (kMFoM) learning approach is also developed. Combining individual techniques, we conduct a set of experiments on various image concept modeling problems, such as handwritten digit recognition, object recognition, and image spam detection to highlight the effectiveness of the proposed framework.PhDCommittee Chair: Lee, Chin-Hui; Committee Member: Clements, Mark; Committee Member: Lee, Hsien-Hsin; Committee Member: McClellan, James; Committee Member: Yuan, Min