Geometry and Expressive Power of Conditional Restricted Boltzmann Machines

Conditional restricted Boltzmann machines are undirected stochastic neural networks with a layer of input and output units connected bipartitely to a layer of hidden units. These networks define models of conditional probability distributions on the states of the output units given the states of the input units, parametrized by interaction weights and biases. We address the representational power of these models, proving results their ability to represent conditional Markov random fields and conditional distributions with restricted supports, the minimal size of universal approximators, the maximal model approximation errors, and on the dimension of the set of representable conditional distributions. We contribute new tools for investigating conditional probability models, which allow us to improve the results that can be derived from existing work on restricted Boltzmann machine probability models.Comment: 30 pages, 5 figures, 1 algorith

Universal Approximation of Markov Kernels by Shallow Stochastic Feedforward Networks

We establish upper bounds for the minimal number of hidden units for which a binary stochastic feedforward network with sigmoid activation probabilities and a single hidden layer is a universal approximator of Markov kernels. We show that each possible probabilistic assignment of the states of $n$ output units, given the states of $k\geq1$ input units, can be approximated arbitrarily well by a network with $2^{k-1}(2^{n-1}-1)$ hidden units.Comment: 13 pages, 3 figure

Application of deep learning techniques for biomedical data analysis

Deep learning and machine learning methods have been used for addressing the problems in the biomedical applications, such as diabetic retinopathy assessment and Parkinson's disease diagnosis. The severity of diabetic retinopathy is estimated by the expert's examination of fundus images based on the amount and location of three diabetic retinopathy signs (i.e., exudates, hemorrhages, and microaneurysms). An automatic and accurate system for detection of these signs can significantly help clinicians to make the best possible prognosis can result in reducing the risk of vision loss. For Parkinson's disease diagnosis, analysis of a speech voice is considered as the earliest symptom with the advantage of being non-intrusive and suitable for online applications. While some reported outcomes of the developed techniques have shown the good results and ongoing progress for these two applications, designing new algorithms is a thriving research field to overcome the poor sensitivity and specificity of the outcomes as well as the limitations such as dataset size and heuristic selection of the network parameters. This thesis has comprehensively studied and developed various deep learning frameworks for detection of diabetic retinopathy signs and diagnosis of Parkinson's disease. To improve the performance of the current systems, this work has had an investigation on different techniques: (i) color space investigation, (ii) examination of various deep learning methods, (iii) development of suitable pre/post-processing algorithms and (iv) appropriate selection of deep learning architectures and parameters. For diabetic retinopathy assessment, this thesis has proposed the new color space as the input for the deep learning models that obtained better replicability compared with the conventional color spaces. This has also shown the pre-trained model can extract more relevant features compared to the models which were trained from scratch. This has also presented a deep learning framework combined with the suitable pre and post-processing algorithms that increased the performance of the system. By investigation different architectures and parameters, the suitable deep learning model has been presented to distinguish between Parkinson's disease and healthy speech signal