Semiconductor detector readout chips with pulse processing electronics have made possible spectroscopic X-ray imaging, bringing an improvement in the overall image quality and, in the case of medical imaging, a reduction in the X-ray dose delivered to the patient. In this contribution we review the state of the art in semiconductor-detector readout ASICs for spectroscopic X-ray imaging with emphasis on hybrid pixel detector technology. We discuss how some of the key challenges of the technology (such as dealing with high fluxes, maintaining spectral fidelity, power consumption density) are addressed by the various ASICs. In order to understand the fundamental limits of the technology, the physics of the interaction of radiation with the semiconductor detector and the process of signal induction in the input electrodes of the readout circuit are described. Simulations of the process of signal induction are presented that reveal the importance of making use of the small pixel effect to minimize the impact of the slow motion of holes and hole trapping in the induced signal in high-Z sensor materials. This can contribute to preserve fidelity in the measured spectrum with relatively short values of the shaper peaking time. Simulations also show, on the other hand, the distortion in the energy spectrum due to charge sharing and fluorescence photons when the pixel pitch is decreased. However, using recent measurements from the Medipix3 ASIC, we demonstrate that the spectroscopic information contained in the incoming photon beam can be recovered by the implementation in hardware of an algorithm whereby the signal from a single photon is reconstructed and allocated to the pixel with the largest deposition