A novel sub-optimal low-complexity equalization and turbo-iterative decoding scheme based on running the sum-product algorithm on an aggressively pruned tree is proposed in this paper for use in a multiple transmit and receive antenna (MIMO) system operating over severe frequency-selective fading inter-symbol interference (ISI) channels. The receiver deals with the issue of signal processing complexity which with a full-search equalization grows with power-law. The sum-product algorithm is applied to the pruned tree which is constructed by two main operations, a sphere list detection and a threshold-based tree search algorithms. At a particular node of the tree, only a number of most probable branches in the tree of hypothetical symbols are expanded and included in the list of candidates; at a particular tree-section, all but some of most probable candidatesare pruned. This pruned tree takes the soft input and generates the soft output, and is utilized in the turbo-iterative manner with the decoder of the low-density parity check code. We oobtained the approximated error probability using the pair-wise error calculation averaged over the fading ensemble, and use it to boundour simulation results. Our current simulation results are obtained for MIMO systems up to four transmit and four receive antennas, using 4-QAM symbols. They indicate the proposed receiverperforms extremely well. The proposed transceiver system is ideal for a system of higher spectral efficiency with even larger signal constellations. Adopting Hassbi-Vikalo's framework, we provide a method which enables a quick evaluation of the signal processing complexity required in the proposed algorithm at a given set of system parameters
