Investigating Machine Learning Techniques for Gesture Recognition with Low-Cost Capacitive Sensing Arrays

Machine learning has proven to be an effective tool for forming models to make predictions based on sample data. Supervised learning, a subset of machine learning, can be used to map input data to output labels based on pre-existing paired data. Datasets for machine learning can be created from many different sources and vary in complexity, with popular datasets including the MNIST handwritten dataset and CIFAR10 image dataset. The focus of this thesis is to test and validate multiple machine learning models for accurately classifying gestures performed on a low-cost capacitive sensing array. Multiple neural networks are trained using gesture datasets obtained from the capacitance board. In this paper, I train and compare different machine learning models on recognizing gesture datasets. Learning hyperparameters are also adjusted for results. Two datasets are used for the training: one containing simple gestures and another containing more complicated gestures. Accuracy and loss for the models are calculated and compared to determine which models excel at recognizing performed gestures

Efficient ConvNets for Analog Arrays

Analog arrays are a promising upcoming hardware technology with the potential to drastically speed up deep learning. Their main advantage is that they compute matrix-vector products in constant time, irrespective of the size of the matrix. However, early convolution layers in ConvNets map very unfavorably onto analog arrays, because kernel matrices are typically small and the constant time operation needs to be sequentially iterated a large number of times, reducing the speed up advantage for ConvNets. Here, we propose to replicate the kernel matrix of a convolution layer on distinct analog arrays, and randomly divide parts of the compute among them, so that multiple kernel matrices are trained in parallel. With this modification, analog arrays execute ConvNets with an acceleration factor that is proportional to the number of kernel matrices used per layer (here tested 16-128). Despite having more free parameters, we show analytically and in numerical experiments that this convolution architecture is self-regularizing and implicitly learns similar filters across arrays. We also report superior performance on a number of datasets and increased robustness to adversarial attacks. Our investigation suggests to revise the notion that mixed analog-digital hardware is not suitable for ConvNets

Detail-Preserving Pooling in Deep Networks

Most convolutional neural networks use some method for gradually downscaling the size of the hidden layers. This is commonly referred to as pooling, and is applied to reduce the number of parameters, improve invariance to certain distortions, and increase the receptive field size. Since pooling by nature is a lossy process, it is crucial that each such layer maintains the portion of the activations that is most important for the network's discriminability. Yet, simple maximization or averaging over blocks, max or average pooling, or plain downsampling in the form of strided convolutions are the standard. In this paper, we aim to leverage recent results on image downscaling for the purposes of deep learning. Inspired by the human visual system, which focuses on local spatial changes, we propose detail-preserving pooling (DPP), an adaptive pooling method that magnifies spatial changes and preserves important structural detail. Importantly, its parameters can be learned jointly with the rest of the network. We analyze some of its theoretical properties and show its empirical benefits on several datasets and networks, where DPP consistently outperforms previous pooling approaches.Comment: To appear at CVPR 201

Half Gaussian-based wavelet transform for pooling layer for convolution neural network

Pooling methods are used to select most significant features to be aggregated to small region. In this paper, anew pooling method is proposed based on probability function. Depending on the fact that, most information is concentrated from mean of the signal to its maximum values, upper half of Gaussian function is used to determine weights of the basic signal statistics, which is used to determine the transform of the original signal into more concise formula, which can represent signal features, this method named half gaussian transform (HGT). Based on strategy of transform computation, Three methods are proposed, the first method (HGT1) is used basic statistics after normalized it as weights to be multiplied by original signal, second method (HGT2) is used determined statistics as features of the original signal and multiply it with constant weights based on half Gaussian, while the third method (HGT3) is worked in similar to (HGT1) except, it depend on entire signal. The proposed methods are applied on three databases, which are (MNIST, CIFAR10 and MIT-BIH ECG) database. The experimental results show that, our methods are achieved good improvement, which is outperformed standard pooling methods such as max pooling and average pooling