Deep Logic Networks: Inserting and Extracting Knowledge from Deep Belief Networks

Developments in deep learning have seen the use of layerwise unsupervised learning combined with supervised learning for fine-tuning. With this layerwise approach, a deep network can be seen as a more modular system that lends itself well to learning representations. In this paper, we investigate whether such modularity can be useful to the insertion of background knowledge into deep networks, whether it can improve learning performance when it is available, and to the extraction of knowledge from trained deep networks, and whether it can offer a better understanding of the representations learned by such networks. To this end, we use a simple symbolic language - a set of logical rules that we call confidence rules - and show that it is suitable for the representation of quantitative reasoning in deep networks. We show by knowledge extraction that confidence rules can offer a low-cost representation for layerwise networks (or restricted Boltzmann machines). We also show that layerwise extraction can produce an improvement in the accuracy of deep belief networks. Furthermore, the proposed symbolic characterization of deep networks provides a novel method for the insertion of prior knowledge and training of deep networks. With the use of this method, a deep neural-symbolic system is proposed and evaluated, with the experimental results indicating that modularity through the use of confidence rules and knowledge insertion can be beneficial to network performance

Generalising the Discriminative Restricted Boltzmann Machine

We present a novel theoretical result that generalises the Discriminative Restricted Boltzmann Machine (DRBM). While originally the DRBM was defined assuming the {0, 1}-Bernoulli distribution in each of its hidden units, this result makes it possible to derive cost functions for variants of the DRBM that utilise other distributions, including some that are often encountered in the literature. This is illustrated with the Binomial and {-1, +1}-Bernoulli distributions here. We evaluate these two DRBM variants and compare them with the original one on three benchmark datasets, namely the MNIST and USPS digit classification datasets, and the 20 Newsgroups document classification dataset. Results show that each of the three compared models outperforms the remaining two in one of the three datasets, thus indicating that the proposed theoretical generalisation of the DRBM may be valuable in practice

Homogenization of weakly coupled systems of Hamilton--Jacobi equations with fast switching rates

We consider homogenization for weakly coupled systems of Hamilton--Jacobi equations with fast switching rates. The fast switching rate terms force the solutions converge to the same limit, which is a solution of the effective equation. We discover the appearance of the initial layers, which appear naturally when we consider the systems with different initial data and analyze them rigorously. In particular, we obtain matched asymptotic solutions of the systems and rate of convergence. We also investigate properties of the effective Hamiltonian of weakly coupled systems and show some examples which do not appear in the context of single equations.Comment: final version, to appear in Arch. Ration. Mech. Ana

Search for heavy neutrinos mixing with tau neutrinos

We report on a search for heavy neutrinos (\nus) produced in the decay D_s\to \tau \nus at the SPS proton target followed by the decay \nudecay in the NOMAD detector. Both decays are expected to occur if \nus is a component of $\nu_{\tau}$.\ From the analysis of the data collected during the 1996-1998 runs with $4.1\times10^{19}$ protons on target, a single candidate event consistent with background expectations was found. This allows to derive an upper limit on the mixing strength between the heavy neutrino and the tau neutrino in the \nus mass range from 10 to 190 $\rm MeV$. Windows between the SN1987a and Big Bang Nucleosynthesis lower limits and our result are still open for future experimental searches. The results obtained are used to constrain an interpretation of the time anomaly observed in the KARMEN1 detector.\Comment: 20 pages, 7 figures, a few comments adde

Final NOMAD results on nu_mu->nu_tau and nu_e->nu_tau oscillations including a new search for nu_tau appearance using hadronic tau decays

Results from the nu_tau appearance search in a neutrino beam using the full NOMAD data sample are reported. A new analysis unifies all the hadronic tau decays, significantly improving the overall sensitivity of the experiment to oscillations. The "blind analysis" of all topologies yields no evidence for an oscillation signal. In the two-family oscillation scenario, this sets a 90% C.L. allowed region in the sin^2(2theta)-Delta m^2 plane which includes sin^2(2theta)<3.3 x 10^{-4} at large Delta m^2 and Delta m^2 < 0.7 eV^2/c^4 at sin^2(2theta)=1. The corresponding contour in the nu_e->nu_tau oscillation hypothesis results in sin^2(2theta)<1.5 x 10^{-2} at large Delta m^2 and Delta m^2 < 5.9 eV^2/c^4 at sin^2(2theta)=1. We also derive limits on effective couplings of the tau lepton to nu_mu or nu_e.Comment: 46 pages, 16 figures, Latex, to appear on Nucl. Phys.

Inclusive production of ρ0(770),f0(980)\rho^{0}(770), f_0(980) and f2(1270)f_2(1270) mesons in νμ\nu_{\mu} charged current interactions

The inclusive production of the meson resonances $\rho^{0}(770)$, $f_0(980)$ and $f_2(1270)$ in neutrino-nucleus charged current interactions has been studied with the NOMAD detector exposed to the wide band neutrino beam generated by 450 GeV protons at the CERN SPS. For the first time the $f_{0}(980)$ meson is observed in neutrino interactions. The statistical significance of its observation is 6 standard deviations. The presence of $f_{2}(1270)$ in neutrino interactions is reliably established. The average multiplicity of these three resonances is measured as a function of several kinematic variables. The experimental results are compared to the multiplicities obtained from a simulation based on the Lund model. In addition, the average multiplicity of $\rho^{0}(770)$ in antineutrino - nucleus interactions is measured.Comment: 23 pages, 14 figures, 8 tables. To appear in Nucl. Phys.

Search for the exotic Θ+\Theta^+ resonance in the NOMAD experiment

A search for exotic Theta baryon via Theta -> proton +Ks decay mode in the NOMAD muon neutrino DIS data is reported. The special background generation procedure was developed. The proton identification criteria are tuned to maximize the sensitivity to the Theta signal as a function of xF which allows to study the Theta production mechanism. We do not observe any evidence for the Theta state in the NOMAD data. We provide an upper limit on Theta production rate at 90% CL as 2.13 per 1000 of neutrino interactions.Comment: Accepted to European Physics Journal

Prediction of Neutrino Fluxes in the NOMAD Experiment

The method developed for the calculation of the flux and composition of the West Area Neutrino Beam used by NOMAD in its search for neutrino oscillations is described. The calculation is based on particle production rates computed using a recent version of FLUKA and modified to take into account the cross sections measured by the SPY and NA20 experiments. These particles are propagated through the beam line taking into account the material and magnetic fields they traverse. The neutrinos produced through their decays are tracked to the NOMAD detector. The fluxes of the four neutrino flavours at NOMAD are predicted with an uncertainty of about 8% for nu(mu) and nu(e), 10% for antinu(mu), and 12% for antinu(e). The energy-dependent uncertainty achieved on the R(e, mu) prediction needed for a nu(mu)->nu(e) oscillation search ranges from 4% to 7%, whereas the overall normalization uncertainty on this ratio is 4.2%.Comment: 43 pages, 20 figures. Submitted to Nucl. Phys.

Search for nu(mu)-->nu(e) Oscillations in the NOMAD Experiment

We present the results of a search for nu(mu)-->nu(e) oscillations in the NOMAD experiment at CERN. The experiment looked for the appearance of nu(e) in a predominantly nu(mu) wide-band neutrino beam at the CERN SPS. No evidence for oscillations was found. The 90% confidence limits obtained are delta m^2 < 0.4 eV^2 for maximal mixing and sin^2(2theta) < 1.4x10^{-3} for large delta m^2. This result excludes the LSND allowed region of oscillation parameters with delta m^2 > 10 eV^2.Comment: 19 pages, 8 figures. Submitted to Phys. Lett.