We use belief-propagation techniques to study the equilibrium behavior of a
bipartite spin-glass, with interactions between two sets of N and P=αN spins. Each spin has a finite degree, i.e.\ number of interaction partners
in the opposite set; an equivalent view is then of a system of N neurons
storing P diluted patterns. We show that in a large part of the parameter
space of noise, dilution and storage load, delimited by a critical surface, the
network behaves as an extensive parallel processor, retrieving all P patterns
{\it in parallel} without falling into spurious states due to pattern
cross-talk and typical of the structural glassiness built into the network. Our
approach allows us to consider effects beyond those studied in replica theory
so far, including pattern asymmetry and heterogeneous dilution. Parallel
extensive retrieval is more robust for homogeneous degree distributions, and is
not disrupted by biases in the distributions of the spin-glass links