Icy dust grains play a key role in the chemistry of the interstellar medium. The cumulative outcome of recent observations, laboratory studies, and astrochemical models indicates that solid-phase reaction mechanisms may dominate the formation of complex organic molecules such as amino acids and sugars in space. Consequently, the composition and structure of the icy grain mantle may significantly influence solid-phase reaction pathways. In this work, we present a new experimental setup capable of studying astrochemical ice analogs in both the TeraHertz (THz), or far-Infrared (far-IR), region (0.3–7.5 THz; 10–250 cm^(−1)) and the mid-IR (400–4000 cm^(−1)). The instruments are capable of performing a variety of spectroscopic studies that can provide especially relevant laboratory data to support astronomical observations from telescopes such as Herschel, SOFIA, and ALMA. Experimental spectra of astrochemical ice analogs of water and carbon dioxide in pure, mixed, and layered ices were collected at different temperatures under high vacuum conditions with the goal of investigating the structure of the ice. We tentatively observe a new feature in both amorphous solid water and crystalline water at 33 cm^(−1) (1 THz). In addition, our studies of mixed and layered ices show how it is possible to identify the location of carbon dioxide as it segregates within the ice by observing its effect on the THz spectrum of water ice. The THz spectra of mixed and layered ices are further analyzed by fitting their spectral features to those of pure amorphous solid water and crystalline water ice to quantify the effects of temperature changes on structure. From the results of this work, it appears that THz spectroscopy is potentially well suited to study thermal transformations within the ice