52 research outputs found
Fixed Points in Two--Neuron Discrete Time Recurrent Networks: Stability and Bifurcation Considerations
The position, number and stability types of fixed points of a two--neuron
recurrent network with nonzero weights are investigated. Using simple
geometrical arguments in the space of derivatives of the sigmoid transfer
function with respect to the weighted sum of neuron inputs, we partition
the network state space into several regions corresponding to stability
types of the fixed points. If the neurons have the same mutual
interaction pattern, i.e. they either mutually inhibit or mutually excite
themselves, a lower bound on the rate of convergence of the attractive
fixed points towards the saturation values, as the absolute values of
weights on the self--loops grow, is given. The role of weights in location
of fixed points is explored through an intuitively appealing
characterization of neurons according to their inhibition/excitation
performance in the network. In particular, each neuron can be of one of
the four types: greedy, enthusiastic, altruistic or depressed. Both with
and without the external inhibition/excitation sources, we investigate the
position and number of fixed points according to character of the neurons.
When both neurons self-excite (or self-inhibit) themselves and have the
same mutual interaction pattern, the mechanism of creation of a new
attractive fixed point is shown to be that of saddle node bifurcation.
(Also cross-referenced as UMIACS-TR-95-51
Learning a Class of Large Finite State Machines with a Recurrent Neural Network
One of the issues in any learning model is how it scales with problem
size. Neural networks have not been immune to scaling issues. We show that a
dynamically-driven discrete-time recurrent network (DRNN) can learn rather
large grammatical inference problems when the strings of a finite memory
machine (FMM) are encoded as temporal sequences. FMMs are a subclass of finite
state machines which have a finite memory or a finite order of inputs and
outputs. The DRNN that learns the FMM is a neural network that maps directly
from the sequential machine implementation of the FMM. It has feedback only
from the output and not from any hidden units; an example is the recurrent
network of Narendra and Parthasarathy. (FMMs that have zero order in the
feedback of outputs are called definite memory machines and are analogous to
Time-delay or Finite Impulse Response neural networks.) Due to their
topology these DRNNs are as least as powerful as any sequential machine
implementation of a FMM and should be capable of representing any FMM. We
choose to learn ``particular FMMs.\' Specifically, these FMMs have a large
number of states (simulations are for and state FMMs) but have
minimal order, relatively small depth and little logic when the FMM is
implemented as a sequential machine. Simulations for the number of training
examples versus generalization performance and FMM extraction size show that
the number of training samples necessary for perfect generalization
is less than that necessary to completely characterize the FMM to be learned.
This is in a sense a best case learning problem since any arbitrarily chosen
FMM with a minimal number of states would have much more order and string
depth and most likely require more logic in its sequential machine
implementation.
(Also cross-referenced as UMIACS-TR-94-94
Learning Long-Term Dependencies is Not as Difficult With NARX Recurrent Neural Networks
It has recently been shown that gradient descent learning algorithms for
recurrent neural networks can perform poorly on tasks that involve long-
term dependencies, i.e. those problems for which the desired output
depends on inputs presented at times far in the past.
In this paper we explore the long-term dependencies problem for a class of
architectures called NARX recurrent neural networks, which have power
ful representational capabilities. We have previously reported that gradient
descent learning is more effective in NARX networks than in recurrent
neural network architectures that have ``hidden states'' on problems includ
ing grammatical inference and nonlinear system identification. Typically,
the network converges much faster and generalizes better than other net
works. The results in this paper are an attempt to explain this phenomenon.
We present some experimental results which show that NARX networks
can often retain information for two to three times as long as conventional
recurrent neural networks. We show that although NARX networks do not
circumvent the problem of long-term dependencies, they can greatly
improve performance on long-term dependency problems.
We also describe in detail some of the assumption regarding what it means
to latch information robustly and suggest possible ways to loosen these
assumptions.
(Also cross-referenced as UMIACS-TR-95-78
Product Unit Learning
Product units provide a method of automatically learning the
higher-order input combinations required for the efficient synthesis of
Boolean logic functions by neural networks. Product units also have a
higher information capacity than sigmoidal networks. However, this
activation function has not received much attention in the literature. A
possible reason for this is that one encounters some problems when
using standard backpropagation to train networks containing these
units. This report examines these problems, and evaluates the
performance of three training algorithms on networks of this
type. Empirical results indicate that the error surface of networks
containing product units have more local minima than corresponding
networks with summation units. For this reason, a combination of local
and global training algorithms were found to provide the most reliable
convergence.
We then investigate how `hints' can be added to the training algorithm.
By extracting a common frequency from the input weights,
and training this frequency separately, we show that convergence can
be accelerated.
A constructive algorithm is then introduced which adds product units
to a network as required by the problem. Simulations show that
for the same problems this method creates a network with significantly
less neurons than those constructed by the tiling and upstart algorithms.
In order to compare their performance with other transfer functions,
product units were implemented as candidate units in the Cascade
Correlation (CC) \cite{Fahlman90} system. Using these candidate units
resulted in smaller networks which trained faster than when the any of
the standard (three sigmoidal types and one Gaussian) transfer
functions were used. This superiority was confirmed when a pool of
candidate units of four different nonlinear activation functions were
used, which have to compete for addition to the network. Extensive
simulations showed that for the problem of implementing random Boolean
logic functions, product units are always chosen above any of
the other transfer functions.
(Also cross-referenced as UMIACS-TR-95-80
Performance of On-Line Learning Methods in Predicting Multiprocessor Memory Access Patterns
Shared memory multiprocessors require reconfigurable interconnection
networks (INs) for scalability. These INs are reconfigured by an IN
control unit. However, these INs are often plagued by undesirable
reconfiguration time that is primarily due to control latency, the
amount of time delay that the control unit takes to decide on a
desired new IN configuration. To reduce control latency, a trainable
prediction unit (PU) was devised and added to the IN controller. The
PU's job is to anticipate and reduce control configuration time, the
major component of the control latency. Three different on-line
prediction techniques were tested to learn and predict repetitive
memory access patterns for three typical parallel processing applications,
the 2-D relaxation algorithm, matrix multiply and Fast Fourier Transform.
The predictions were then used by a routing control algorithm to reduce
control latency by configuring the IN to provide needed memory access
paths before they were requested. Three prediction techniques were used
and tested: 1). a Markov predictor, 2). a linear predictor and 3). a
time delay neural network (TDNN) predictor. As expected, different
predictors performed best on different applications, however, the TDNN
produced the best overall results.
(Also cross-referenced as UMIACS-TR-96-59
Recommended from our members
Building WF16: construction of a PPNA pisé structure in Southern Jordan
The Pre-Pottery Neolithic A (PPNA) period in Southwest Asia is essential for our understanding of the transition to sedentary, agricultural communities. Developments in architecture are key to understanding this transition, but many aspects of PPNA architecture remain elusive, such as construction techniques, the selection of building materials, and the functional use of space. The primary aim of the research described within this contribution was to build a PPNA-like structure in order to answer questions about PPNA architecture in general, while specifically addressing issues raised by the excavation of structures at the site of WF16, Southern Jordan. The second aim was to display a ‘PPNA’ building to visitors in Wadi Faynan to enhance their understanding of the period. The experimental construction based on one of the WF16 structures showed that 1) required materials can be acquired locally; 2) a construction technique using mud layers as described in this paper was likely used; 3) flat, or very slightly dome-shaped, roofs are functional and can also be used as a solid working platform; 4) the WF16 small semi-subterranean buildings appear inappropriate for housing a nuclear family unit
TOI-1695 b:A Water World Orbiting an Early-M Dwarf in the Planet Radius Valley
Characterizing the bulk compositions of transiting exoplanets within the M dwarf radius valley offers a unique means to establish whether the radius valley emerges from an atmospheric mass-loss process or is imprinted by planet formation itself. We present the confirmation of such a planet orbiting an early-M dwarf (Tmag = 11.0294 ± 0.0074, Ms = 0.513 ± 0.012 M⊙, Rs = 0.515 ± 0.015 R⊙, and Teff = 3690 ± 50 K): TOI-1695 b (P = 3.13 days and Rp = 1.90−0.14+0.16 R⊕ ). TOI-1695 b’s radius and orbital period situate the planet between model predictions from thermally driven mass loss versus gas depleted formation, offering an important test case for radius valley emergence models around early-M dwarfs. We confirm the planetary nature of TOI-1695 b based on five sectors of TESS data and a suite of follow-up observations including 49 precise radial velocity measurements taken with the HARPS-N spectrograph. We measure a planetary mass of 6.36 ± 1.00 M⊕, which reveals that TOI-1695 b is inconsistent with a purely terrestrial composition of iron and magnesium silicate, and instead is likely a water-rich planet. Our finding that TOI-1695 b is not terrestrial is inconsistent with the planetary system being sculpted by thermally driven mass loss. We present a statistical analysis of seven well-characterized planets within the M dwarf radius valley demonstrating that a thermally driven mass-loss scenario is unlikely to explain this population.</p
TOI-2196 b : Rare planet in the hot Neptune desert transiting a G-type star
Funding: C.M.P., M.F., I.G., and J.K. gratefully acknowledge the support of the Swedish National Space Agency (DNR 65/19, 174/18, 177/19, 2020-00104). L.M.S and D.G. gratefully acknowledge financial support from the CRT foundation under Grant No. 2018.2323 “Gaseous or rocky? Unveiling the nature of small worlds”. P.K. acknowledges support from grant LTT-20015. E.G. acknowledge the support of the Thüringer Ministerium für Wirtschaft, Wissenschaft und Digitale Gesellschaft. J.S.J. gratefully acknowledges support by FONDECYT grant 1201371 and from the ANID BASAL projects ACE210002 and FB210003. H.J.D. acknowledges support from the Spanish Research Agency of the Ministry of Science and Innovation (AEI-MICINN) under grant PID2019-107061GBC66, DOI: 10.13039/501100011033. D.D. acknowledges support from the TESS Guest Investigator Program grants 80NSSC21K0108 and 80NSSC22K0185. M.E. acknowledges the support of the DFG priority program SPP 1992 "Exploring the Diversity of Extrasolar Planets" (HA 3279/12-1). K.W.F.L. was supported by Deutsche Forschungsgemeinschaft grants RA714/14-1 within the DFG Schwerpunkt SPP 1992, Exploring the Diversity of Extrasolar Planets. N.N. acknowledges support from JSPS KAKENHI Grant Number JP18H05439, JST CREST Grant Number JPMJCR1761. M.S.I.P. is funded by NSF.The hot Neptune desert is a region hosting a small number of short-period Neptunes in the radius-instellation diagram. Highly irradiated planets are usually either small (R ≲ 2 R⊕) and rocky or they are gas giants with radii of ≳1 RJ. Here, we report on the intermediate-sized planet TOI-2196 b (TIC 372172128.01) on a 1.2 day orbit around a G-type star (V = 12.0, [Fe/H] = 0.14 dex) discovered by the Transiting Exoplanet Survey Satellite in sector 27. We collected 41 radial velocity measurements with the HARPS spectrograph to confirm the planetary nature of the transit signal and to determine the mass. The radius of TOI-2196 b is 3.51 ± 0.15 R⊕, which, combined with the mass of 26.0 ± 1.3 M⊕, results in a bulk density of 3.31−0.43+0.51 g cm−3. Hence, the radius implies that this planet is a sub-Neptune, although the density is twice than that of Neptune. A significant trend in the HARPS radial velocity measurements points to the presence of a distant companion with a lower limit on the period and mass of 220 days and 0.65 MJ, respectively, assuming zero eccentricity. The short period of planet b implies a high equilibrium temperature of 1860 ± 20 K, for zero albedo and isotropic emission. This places the planet in the hot Neptune desert, joining a group of very few planets in this parameter space discovered in recent years. These planets suggest that the hot Neptune desert may be divided in two parts for planets with equilibrium temperatures of ≳1800 K: a hot sub-Neptune desert devoid of planets with radii of ≈ 1.8−3 R⊕ and a sub-Jovian desert for radii of ≈5−12 R⊕. More planets in this parameter space are needed to further investigate this finding. Planetary interior structure models of TOI-2196 b are consistent with a H/He atmosphere mass fraction between 0.4% and 3%, with a mean value of 0.7% on top of a rocky interior. We estimated the amount of mass this planet might have lost at a young age and we find that while the mass loss could have been significant, the planet had not changed in terms of character: it was born as a small volatile-rich planet and it remains one at present.Publisher PDFPeer reviewe
Finishing the euchromatic sequence of the human genome
The sequence of the human genome encodes the genetic instructions for human physiology, as well as rich information about human evolution. In 2001, the International Human Genome Sequencing Consortium reported a draft sequence of the euchromatic portion of the human genome. Since then, the international collaboration has worked to convert this draft into a genome sequence with high accuracy and nearly complete coverage. Here, we report the result of this finishing process. The current genome sequence (Build 35) contains 2.85 billion nucleotides interrupted by only 341 gaps. It covers ∼99% of the euchromatic genome and is accurate to an error rate of ∼1 event per 100,000 bases. Many of the remaining euchromatic gaps are associated with segmental duplications and will require focused work with new methods. The near-complete sequence, the first for a vertebrate, greatly improves the precision of biological analyses of the human genome including studies of gene number, birth and death. Notably, the human enome seems to encode only 20,000-25,000 protein-coding genes. The genome sequence reported here should serve as a firm foundation for biomedical research in the decades ahead
- …