Near-eye display systems, which project digital information directly into the human visual system, are expected to revolutionize the interface between digital information and physical world. However, the image quality of most near-eye displays is still far inferior to that of direct-view displays. Both light engine and imaging optics of near-eye display systems play important roles to the degraded image quality. In addition, near-eye displays also suffer from a relatively low optical efficiency, which severely limits the device operation time. Such an efficiency loss originates from both light engines and projection processes. This dissertation is devoted to addressing these two critical issues from the entire system perspective. In Chapter 2, we propose useful design guidelines for the miniature light-emitting diode (mLED) backlit liquid crystal displays (LCDs) to mitigate halo artifacts. After developing a high dynamic range (HDR) light engine in Chapter 3, we establish a systematic image quality evaluation model for virtual reality (VR) devices and analyze the requirements for light engines. Our guidelines for mLED backlit LCDs have been widely practiced in direct-view displays. Similarly, the newly established criteria for light engines will shed new light to guide future VR display development. To improve the optical efficiency of near eye displays, we must optimize each component. For the light engine, we focus on color-converted micro-LED microdisplays. We fabricate a pixelated cholesteric liquid crystal film on top of a pixelated QD array to recycle the leaked blue light, which in turn doubles the optical efficiency and widens the color gamut. In Chapter 5, we tailor the radiation pattern of the light engine to match the etendue of the imaging systems, as a result, the power loss in the projection process is greatly reduced. The system efficiency is enhanced by over one-third for both organic light-emitting diode (OLED) displays and LCDs while maintaining indistinguishable image nonuniformity. In Chapter 6, we briefly summarize our major accomplishments
