See the Difference: Direct Pre-Image Reconstruction and Pose Estimation by Differentiating HOG

The Histogram of Oriented Gradient (HOG) descriptor has led to many advances in computer vision over the last decade and is still part of many state of the art approaches. We realize that the associated feature computation is piecewise differentiable and therefore many pipelines which build on HOG can be made differentiable. This lends to advanced introspection as well as opportunities for end-to-end optimization. We present our implementation of $\nabla$HOG based on the auto-differentiation toolbox Chumpy and show applications to pre-image visualization and pose estimation which extends the existing differentiable renderer OpenDR pipeline. Both applications improve on the respective state-of-the-art HOG approaches

Improving the Caenorhabditis elegans Genome Annotation Using Machine Learning

For modern biology, precise genome annotations are of prime importance, as they allow the accurate definition of genic regions. We employ state-of-the-art machine learning methods to assay and improve the accuracy of the genome annotation of the nematode Caenorhabditis elegans. The proposed machine learning system is trained to recognize exons and introns on the unspliced mRNA, utilizing recent advances in support vector machines and label sequence learning. In 87% (coding and untranslated regions) and 95% (coding regions only) of all genes tested in several out-of-sample evaluations, our method correctly identified all exons and introns. Notably, only 37% and 50%, respectively, of the presently unconfirmed genes in the C. elegans genome annotation agree with our predictions, thus we hypothesize that a sizable fraction of those genes are not correctly annotated. A retrospective evaluation of the Wormbase WS120 annotation [1] of C. elegans reveals that splice form predictions on unconfirmed genes in WS120 are inaccurate in about 18% of the considered cases, while our predictions deviate from the truth only in 10%–13%. We experimentally analyzed 20 controversial genes on which our system and the annotation disagree, confirming the superiority of our predictions. While our method correctly predicted 75% of those cases, the standard annotation was never completely correct. The accuracy of our system is further corroborated by a comparison with two other recently proposed systems that can be used for splice form prediction: SNAP and ExonHunter. We conclude that the genome annotation of C. elegans and other organisms can be greatly enhanced using modern machine learning technology

Applied Mathematics and Computational Physics

As faster and more efficient numerical algorithms become available, the understanding of the physics and the mathematical foundation behind these new methods will play an increasingly important role. This Special Issue provides a platform for researchers from both academia and industry to present their novel computational methods that have engineering and physics applications

Stochastic selection of activation layers for convolutional neural networks

In recent years, the field of deep learning has achieved considerable success in pattern recognition, image segmentation, and many other classification fields. There are many studies and practical applications of deep learning on images, video, or text classification. Activation functions play a crucial role in discriminative capabilities of the deep neural networks and the design of new \u201cstatic\u201d or \u201cdynamic\u201d activation functions is an active area of research. The main difference between \u201cstatic\u201d and \u201cdynamic\u201d functions is that the first class of activations considers all the neurons and layers as identical, while the second class learns parameters of the activation function independently for each layer or even each neuron. Although the \u201cdynamic\u201d activation functions perform better in some applications, the increased number of trainable parameters requires more computational time and can lead to overfitting. In this work, we propose a mixture of \u201cstatic\u201d and \u201cdynamic\u201d activation functions, which are stochastically selected at each layer. Our idea for model design is based on a method for changing some layers along the lines of different functional blocks of the best performing CNN models, with the aim of designing new models to be used as stand-alone networks or as a component of an ensemble. We propose to replace each activation layer of a CNN (usually a ReLU layer) by a different activation function stochastically drawn from a set of activation functions: in this way, the resulting CNN has a different set of activation function layers. The code developed for this work will be available at https://github.com/LorisNanni

Channel Coding in Molecular Communication

This dissertation establishes and analyzes a complete molecular transmission system from a communication engineering perspective. Its focus is on diffusion-based molecular communication in an unbounded three-dimensional fluid medium. As a basis for the investigation of transmission algorithms, an equivalent discrete-time channel model (EDTCM) is developed and the characterization of the channel is described by an analytical derivation, a random walk based simulation, a trained artificial neural network (ANN), and a proof of concept testbed setup. The investigated transmission algorithms cover modulation schemes at the transmitter side, as well as channel equalizers and detectors at the receiver side. In addition to the evaluation of state-of-the-art techniques and the introduction of orthogonal frequency-division multiplexing (OFDM), the novel variable concentration shift keying (VCSK) modulation adapted to the diffusion-based transmission channel, the lowcomplex adaptive threshold detector (ATD) working without explicit channel knowledge, the low-complex soft-output piecewise linear detector (PLD), and the optimal a posteriori probability (APP) detector are of particular importance and treated. To improve the error-prone information transmission, block codes, convolutional codes, line codes, spreading codes and spatial codes are investigated. The analysis is carried out under various approaches of normalization and gains or losses compared to the uncoded transmission are highlighted. In addition to state-of-the-art forward error correction (FEC) codes, novel line codes adapted to the error statistics of the diffusion-based channel are proposed. Moreover, the turbo principle is introduced into the field of molecular communication, where extrinsic information is exchanged iteratively between detector and decoder. By means of an extrinsic information transfer (EXIT) chart analysis, the potential of the iterative processing is shown and the communication channel capacity is computed, which represents the theoretical performance limit for the system under investigation. In addition, the construction of an irregular convolutional code (IRCC) using the EXIT chart is presented and its performance capability is demonstrated. For the evaluation of all considered transmission algorithms the bit error rate (BER) performance is chosen. The BER is determined by means of Monte Carlo simulations and for some algorithms by theoretical derivation

Neural network reconstruction of the dense matter equation of state from neutron star observables

The Equation of State (EoS) of strongly interacting cold and hot ultra-dense QCD matter remains a major challenge in the field of nuclear astrophysics. With the advancements in measurements of neutron star masses, radii, and tidal deformabilities, from electromagnetic and gravitational wave observations, neutron stars play an important role in constraining the ultra-dense QCD matter EoS. In this work, we present a novel method that exploits deep learning techniques to reconstruct the neutron star EoS from mass-radius (M-R) observations. We employ neural networks (NNs) to represent the EoS in a model-independent way, within the range of $\sim$1-7 times the nuclear saturation density. The unsupervised Automatic Differentiation (AD) framework is implemented to optimize the EoS, so as to yield through TOV equations, an M-R curve that best fits the observations. We demonstrate that this method works by rebuilding the EoS on mock data, i.e., mass-radius pairs derived from a randomly generated polytropic EoS. The reconstructed EoS fits the mock data with reasonable accuracy, using just 11 mock M-R pairs observations, close to the current number of actual observations.Comment: 24 pages, 14 figures, https://github.com/ss-fias/nn-eo