The next generation of electronic devices faces the challenge of adequately containing and controlling extremely high charge densities within structures of nanometer dimensions. Atomic-scale transistors must be thin and be able to control extremely high charge densities (>10e13/cm^2). Silicon devices typically have two-dimensional electron gas (2DEG) densities around 10e12/cm^2. Nitride-based devices can sustain densities an order of magnitude higher. The "complex oxides" have recently emerged as an attractive materials system to support these developments. The demonstration of a 2DEG at the SrTiO3/LaAlO3 interface has triggered an avalanche of research, including the unprecedentedly high density of 3x10e14/cm^2 at SrTiO3/GdTiO3 and SrTiO3/SmTiO3 interfaces. Metal-insulator (Mott) transitions that are inherent to some of these complex oxides could offer even greater prospects for enhanced functionality or novel device concepts.The materials and heterostructures that have been explored to date are clearly only a small subset of the vast number of materials combinations that could lead to interesting phenomena. In this work we use first-principles methods to build greater understanding of the interface phenomena, so that searches can be better informed and more focused. We also develop a set of criteria that the materials and their heterostructures should satisfy to develop a high-performance 2DEG-based device. We focus in particular on the band alignment, calculating it for a variety of different potential materials. Next, we study GdTiO3/SrTiO3/GdTiO3 heterostructures in depth, where each interface contributes excess electrons into the SrTiO3. We calculate the 2DEG formation for a superlattice containing six layers of SrTiO3, and compare with angle-resolved photoemission spectroscopy results. Together, the experimental and theoretical results conclusively show that the 2DEG results from the interface itself, and does not originate from a secondary source such as oxygen vacancies. These heterostructures also exhibit a metal-to-insulator transition as the SrTiO3 layer thickness decreases, which could possibly be used as a "Mott field effect transistor" - the system is very close to a metal-to-insulator transition, and modulating a small fraction of the electron density would lead to switching between the metallic and insulating phases. The mechanism behind this transition is unraveled, and we construct a bulk model of the transition based on the surprising observation that SrTiO3 itself can become a Mott insulator when doped with an extremely high density of electrons.Building on our study of the SrTiO3/GdTiO3 interfaces, we investigate the electronic structure of GdTiO3 in detail - our calculated band gap differs markedly from past experimental values, but is consistent with recent photoluminescence measurements. We find that the presence of small hole polarons leads to a feature in the optical absorption spectrum which was previously interpreted to be the band gap. Since small hole polarons are present in all the rare-earth titanates, not only GdTiO3, the values of the band gaps (also based on optical absorption measurements) across the series will likely have to be revised. Lastly, to understand the formation of small hole polarons in the rare-earth titanates, we study point defects and impurities in GdTiO3. We also investigate how defects may impact the behavior of GdTiO3 in electronic devices