FPGA implementation of artificial neural networks

As the title suggests our project deals with a hardware implementation of artificial neural networks, specifically a FPGA implementation. During the course of this project we learnt about ANNs and the uses of such soft computing approaches, FPGAs, VHDL and use of various tools like Xilinx ISE Project Navigator and ModelSim. As numerous hardware implementations of ANNs already exist our aim was to come up with an approach that would facilitate topology evolution of the ANN as well

An Adaptive Locally Connected Neuron Model: Focusing Neuron

This paper presents a new artificial neuron model capable of learning its receptive field in the topological domain of inputs. The model provides adaptive and differentiable local connectivity (plasticity) applicable to any domain. It requires no other tool than the backpropagation algorithm to learn its parameters which control the receptive field locations and apertures. This research explores whether this ability makes the neuron focus on informative inputs and yields any advantage over fully connected neurons. The experiments include tests of focusing neuron networks of one or two hidden layers on synthetic and well-known image recognition data sets. The results demonstrated that the focusing neurons can move their receptive fields towards more informative inputs. In the simple two-hidden layer networks, the focusing layers outperformed the dense layers in the classification of the 2D spatial data sets. Moreover, the focusing networks performed better than the dense networks even when 70$\%$ of the weights were pruned. The tests on convolutional networks revealed that using focusing layers instead of dense layers for the classification of convolutional features may work better in some data sets.Comment: 45 pages, a national patent filed, submitted to Turkish Patent Office, No: -2017/17601, Date: 09.11.201

Neural Network Formalization

In order to assist the field of neural networks in its maturing, a formalization and a solid foundation are essential. Additionally, to permit the introduction of formal proofs, it is essential to have an all encompassing formal mathematical definition of a neural network. Most neural networks, even biological ones, exhibit a layered structure. This publication shows that all neural networks can be represented as layered structures. This layeredness is therefore chosen as the basis for a formal neural network framework. This publication offers a neural network formalization consisting of a topological taxonomy, a uniform nomenclature, and an accompanying consistent mnemonic notation. Supported by this formalization, both a flexible hierarchical and a universal mathematical definition are presented

Bayesian autoencoders for data-driven discovery of coordinates, governing equations and fundamental constants

Recent progress in autoencoder-based sparse identification of nonlinear dynamics (SINDy) under $\ell_1$ constraints allows joint discoveries of governing equations and latent coordinate systems from spatio-temporal data, including simulated video frames. However, it is challenging for $\ell_1$-based sparse inference to perform correct identification for real data due to the noisy measurements and often limited sample sizes. To address the data-driven discovery of physics in the low-data and high-noise regimes, we propose Bayesian SINDy autoencoders, which incorporate a hierarchical Bayesian sparsifying prior: Spike-and-slab Gaussian Lasso. Bayesian SINDy autoencoder enables the joint discovery of governing equations and coordinate systems with a theoretically guaranteed uncertainty estimate. To resolve the challenging computational tractability of the Bayesian hierarchical setting, we adapt an adaptive empirical Bayesian method with Stochatic gradient Langevin dynamics (SGLD) which gives a computationally tractable way of Bayesian posterior sampling within our framework. Bayesian SINDy autoencoder achieves better physics discovery with lower data and fewer training epochs, along with valid uncertainty quantification suggested by the experimental studies. The Bayesian SINDy autoencoder can be applied to real video data, with accurate physics discovery which correctly identifies the governing equation and provides a close estimate for standard physics constants like gravity $g$, for example, in videos of a pendulum.Comment: 28 pages, 11 figure

Identification Of Streptococcus Pyogenes Using Raman Spectroscopy

Despite the attention that Raman Spectroscopy has gained recently in the area of pathogen identification, the spectra analyses techniques are not well developed. In most scenarios, they rely on expert intervention to detect and assign the peaks of the spectra to specific molecular vibration. Although some investigators have used machine-learning techniques to classify pathogens, these studies are usually limited to a specific application, and the generalization of these techniques is not clear. Also, a wide range of algorithms have been developed for classification problems, however, there is less insight to applying such methods on Raman spectra. Furthermore, analyzing the Raman spectra requires pre-processing of the raw spectra, in particular, background removing. Various techniques are developed to remove the background of the raw spectra accurately and with or without less expert intervention. Nevertheless, as the background of the spectra varies in the different media, these methods still require expert effort adding complexity and inefficiency to the identification task. This dissertation describes the development of state-of-the-art classification techniques to identify S. pyogenes from other species, including water and other confounding background pathogens. We compared these techniques in terms of their classification accuracy, sensitivity, and specificity in addition to providing a bias-variance insight in selecting the number of principal components in a principal component analysis (PCA). It was observed that Random Forest provided a better result with an accuracy of 94.11%. Next, a novel deep learning technique was developed to remove the background of the Raman spectra and then identify the pathogen. The architecture of the network was discussed and it was found that this method yields an accuracy of 100% in our test samples. This outperforms other traditional machine learning techniques as discussed. In clinical applications of Raman Spectroscopy, the samples have confounding background creates a challenging task for the removal of the spectral background and subsequent identification of the pathogen in real- time. We tested our methodology on datasets composed of confounding background such as throat swabs from patients and discussed the robustness and generalization of the developed method. It was found that the misclassification error of the test dataset was around 3.7%. Also, the realization of the trained model is discussed in detail to provide a better understating and insight into the efficacy of the deep learning architecture. This technique provides a platform for general analysis of other pathogens in confounding environments as well

Resolving Cell Fate: Experimental, computational and mathematical methods in single cell transcriptomic analysis

Deeper understanding of the embryological origins of tissues and organs is likely to provide insights into novel clinically relevant preventative and therapeutic strategies. To do so effectively and at a large scale so as to have clinical significance requires an exhaustive and meticulously accurate knowledge of normal morphological development and the underlying molecular pathways. A variety of lineage tracing and classical molecular biology techniques have led to key insights but emerging technologies now offer the possibility of more fine grained and precise measurements at the level of the single cell rather than ensembles of heterogeneous cells. In this study the rapidly developing technology of single cell RNA sequencing was combined with the development of state of the art computational methods to study murine gastrulation and early organogenesis at a hitherto unprecedented granularity. A comprehensive analysis of FLK1+ cells harvested from gastrulating murine embryos was performed. In-silico reconstruction of pseudo-temporal and pseudo-spacial relations were demonstrated. Using prior knowledge of known driver genes deeper substructure was revealed in clusters that were initially defined by unsupervised algorithms, illustrating that careful implementation of supervised approaches can outperform naïve unsupervised methods. In this way multiple members of the leukotriene branch of the arachidonic acid pathway were found to be enriched within a subset of the endothelial cluster that has a molecular signature consistent with yolk sac derived definitive wave haematopoiesis. This was subsequently validated in an in-vitro embryonic stem cell differentiation colony assay. By developing a novel adaptation to tSNE dimensionality reduction that now allows new data points to be mapped backed to previously calculated embeddings, the FLK1+ data set became a reference to which cells from other experiments were mapped. The Tal1-/- knockout mutant was characterised, reaffirming the known phenotype with complete failure of embryonic haematopoiesis. Analysing the Tal1-/- endothelial cluster shows definitively in the in-vivo organism no activation of an alternative cardiac fate programme as was previously postulated from an in-vitro model system. Additionally tSNE mapped Brachyury cells into the void in the FLK1+ dataset and identified a node like population. Loss of spacial context remains an Achilles’ heel of single cell protocols. Generation of the single cell suspensions leads to disruption of cellular contacts and loss of any spacial information. A novel method is described which uses tSNE of bulk spacial data to pre- condition a tSNE map upon which single cells can then be computationally positioned reconstructing spacial context. To model gene interactions and perturbations from single-cell data, a hybrid feed-forward deep neural network was trained on branching pseudo-temporally arranged single cell qPCR data of in-vivo wild-type murine developmental haematopoiesis. Strikingly despite the model never having ‘seen’ a mutant, in-silico gene perturbations in the deep neural network are able to faithfully reproduce the Tal1-/- phenotype. In summary the use of single cell transcriptomics to probe early murine embryology com- bined with development of new methods has uncovered a novel pathway in embryonic haematopoietic development and allowed in-silico reconstruction of a short period of early embryonic haematopoiesis. Critically these methods have broad application within the fields of developmental and stem cell biology.Wellcome Trust GRANT_NUMBER: 103392/Z/13/A GRANT_NUMBER: 103392/Z/13/

Laws of Conservation as Related to Brain Growth, Aging, and Evolution: Symmetry of the Minicolumn

Development, aging, and evolution offer different time scales regarding possible anatomical transformations of the brain. This article expands on the perspective that the cerebral cortex exhibits a modular architecture with invariant properties in regards to these time scales. These properties arise from morphometric relations of the ontogenetic minicolumn as expressed in Noether’s first theorem, i.e., that for each continuous symmetry there is a conserved quantity. Whenever minicolumnar symmetry is disturbed by either developmental or aging processes the principle of least action limits the scope of morphometric alterations. Alternatively, local and global divergences from these laws apply to acquired processes when the system is no longer isolated from its environment. The underlying precepts to these physical laws can be expressed in terms of mathematical equations that are conservative of quantity. Invariant properties of the brain include the rotational symmetry of minicolumns, a scaling proportion or “even expansion” between pyramidal cells and core minicolumnar size, and the translation of neuronal elements from the main axis of the minicolumn. It is our belief that a significant portion of the architectural complexity of the cerebral cortex, its response to injury, and its evolutionary transformation, can all be captured by a small set of basic physical laws dictated by the symmetry of minicolumns. The putative preservations of parameters related to the symmetry of the minicolumn suggest that the development and final organization of the cortex follows a deterministic process