The total anisotropy of a diffuse background composed of two or more sources, such as the Fermi-Large Area Telescope (LAT)-measured gamma-ray background, is set by the anisotropy of each source population and the contribution of each population to the total intensity. The total anisotropy as a function of energy (the anisotropy energy spectrum) will modulate as the relative contributions of the sources change, implying that the anisotropy energy spectrum also encodes the intensity spectrum of each source class. We develop techniques, applicable to any such diffuse background, for unraveling the intensity spectrum of each component source population given a measurement of the total intensity spectrum and the total anisotropy energy spectrum, without introducing a priori assumptions about the spectra of the source classes. We demonstrate the potential of these methods by applying them to example scenarios for the composition of the Fermi-LAT gamma-ray background consistent with current data and feasible within 10 yr of observation