1 research outputs found
Recommended from our members
Enhanced lower entropy bounds with application to constructive learning
In this paper the authors prove two new lower bounds for the number-of-bits required by neural networks for classification problems defined by m examples from R{sup n}. Because they are obtained in a constructive way, they can be used for designing a constructive algorithm. These results rely on techniques used for determining tight upper bounds, which start by upper bounding the space with an n-dimensional ball. Very recently, a better upper bound has been detailed by showing that the volume of the ball can always be replaced by the volume of the intersection of two balls. A first lower bound for the case of integer weights in the range [{minus}p,p] has been detailed: it is based on computing the logarithm of the quotient between the volume of the ball containing all the examples (rough approximation) and the maximum volume of a polyhedron. A first improvement over that bound will come from a tighter upper bound of the maximum volume of the polyhedron by two n-dimensional cones. An even tighter bound will be obtained by upper bounding the space by the intersection of two balls