Micro-LED displays are now emerging rapidly towards commercialization. This novel compound semiconductor display technology has the potential to interface very effectively to advanced electronics, in particular Si CMOS, to create new forms of display which also interrogate and communicate with their environment in sophisticated ways

Dawson, Martin D.

Griffiths, Alexander D.

Gu, Erdan

Herrnsdorf, Johannes

McKendry, Jonathan

Strain, Michael J.

Xie, Enyuan

English

University of Strathclyde Institutional Repository

Micro-LEDs for Technological Convergence between Displays, Optical Communications, and Sensing and Imaging Systems Martin D. Dawson*, Johannes Herrnsdorf*, Enyuan Xie*, Erdan Gu*, Jonathan McKendry*, Alexander D. Griffiths*, and Michael J. Strain* *Institute of Photonics, Department of Physics, University of Strathclyde, Technology and Innovation Centre, 99 George Street, Glasgow, G1 1RD, UK  Abstract Micro-LED displays are now emerging rapidly towards commercialization. This novel compound semiconductor display technology has the potential to interface very effectively to advanced electronics, in particular Si CMOS, to create new forms of display which also interrogate and communicate with their environment in sophisticated ways. Author Keywords micro-LED, highly integrated semiconductor information displays, visible light communications. 1. Objective and Background Micro light-emitting diode (Micro-LED) technology, based principally on the capabilities of gallium nitride inorganic semiconductor epi-structures, is emerging very rapidly to commercial maturity to provide a new generation of robust and high-performance displays [1]. These devices, which comprise high-density formats of individual LED pixels with dimensions of a few microns to tens of microns, are of the self-illumination type with high-brightness, broad viewing angle, and very rapid (ns) response time. Demonstrator capabilities spanning wearables, to augmented (AR) and mixed reality (MR) systems, through to large screen TVs and displays, are currently being trialled by innovative new businesses and multi-national corporations. In parallel with these developments, the lighting industry - in its move to gallium nitride-based solid state lighting - is beginning to embrace new operating models (such as Lighting-as-a-Service) which will embed additional functionality such as optical wireless communications via Light Fidelity (LiFi) [2]. The developments underway posit a move from first-generation, so-called ‘smart lighting’, towards what might be called digital lighting. Micro-LED technology is also setting performance benchmarks in LiFi [3,4], and the exciting prospect is emerging of combining micro-LED based capabilities for displays with lighting and communications, potentially also involving sensing, ranging and imaging functions. Thus micro-LEDs are poised to offer new convergences between hitherto largely disparate areas of technology and application [5].   Figure 1. Typical layout of an active-matrix GaN LED array.  2. Results Micro-LED technology fits naturally into high-pixel density 1-D and 2-D emitter array formats to provide electronic visual display capability. Digital interfacing is commonly achieved through flip-ship integration with a digitally addressed active-matrix driver chip [6] as illustrated in Figure 1 and shown photographically in Figure 2, but alternative approaches such as monolithic integration [7] or transfer-printing [8] have been demonstrated. Furthermore, the detailed physics of micro-sized pixels offers enhanced modulation bandwidths into the GHz range per pixel [3], more than two orders of magnitude higher than that typical of conventional broad area LEDs. These factors can be combined in new forms of spatially multiplexed or spatially modulated LiFi communications, such as space-shift keying (SSK) or multiple input multiple output (MIMO), to enhance data communications channel capacity [3]. In this format, the display function embodies the spatial registration/distribution of information in a communications link, rather than necessarily embodying direct-view images of a scene. However, the frame rate or image refresh rate is so fast for micro-LEDs that the displays could operate multi-modally, to implement display and spatially modulated communications functions in parallel.   Figure 2. Close-up of a flip-chip integrated GaN on CMOS active matrix micro-LED array. High refresh rate binary mask patterns can be generated with complementary metal-oxide semiconductor (CMOS) interfaced micro-LED displays [6], at frame rates which may exceed 1 MHz, and at a brightness suitable for projection [9] as demonstrated in Figure 3. An overview of devices used in different technology demonstrators to date is given in Table 1, and ongoing development efforts focus on improving array density, frame rates, and optimized digital interfacing with digital signal processing systems. Binary illumination patterns projected at a frame rate beyond visual flicker recognition can carry a data signal, which can be received by high-speed cameras. Such a scheme is illustrated in Figure 4, where the data patterns are updated at half the receiving camera frame rate. Such a micro-LED projector based camera communications scheme is accessible to state-of-the-art mobile phone camera receivers offering frame rates of 960 fps [14], and the data signal can in principle be overlaid with image or video content running at a much lower frame rate.  Table 1. Overview of CMOS-integrated micro-LED arrays, which have been used for technology demonstrators towards applications including displays, LiFi, fluorescence lifetime imaging (FLIM), structured visible light positioning (VLP), optical camera communications (OCC), and 3D ranging and imaging. Year Dimension Application highlights Ref 2008 4×16 FLIM [6] 2009 8×8 FLIM [10] 2012 16×16 LiFi, VLP, OCC, 3D ranging [8,11,14,15] 2015 10×40 Display, VLP, 3D imaging [9] 2015 6×6 LiFi [12] 2017 1×16 LiFi [13]   Figure 3. Projection of structured light patterns onto a screen by a 16×16 active-matrix array of GaN micro-LEDs.   Figure 4. Schematic of optical camera communications using structured illumination from a micro-LED projector.   The checkerboard-like high-frame rate patterns projected by these systems can also provide unique digital signatures to each location in the projected frame, which can be used to implement location, tracking and navigation functions [15]. Thus the projected output of such a display can be used e.g. to locate and track moving objects in the space around it, potentially including viewers of the display, while also displaying the image or video information required of a conventional display. Figure 5 shows a tracking result obtained by this method. Two retro-reflectors were placed in the projection area at different positions, corresponding to different coordinates in the grid defined by the projected LED pixels. A photodiode was placed next to the micro-LED array to receive the light returned from the scene. As a time-series of appropriate patterns [15] is projected, the photodiode receives a unique fingerprint signal from each reflector, as shown by the two traces in Figure 5, thus revealing the location of each of them. Compared with other visible light positioning methods, the use of micro-LED display projection requires minimal computational resources, provides straight-forward combination with space-division multiple access LiFi, and is readily scalable through the choice of the projection format.   Figure 5. Optical signal received from two retro-reflectors placed in a tracking grid defined by LED array projection. The first reflector (blue trace) was at grid position (4,6), and the second reflector (yellow trace) at (14,7).  An additional variant of these digital lighting systems uses LED projections from multiple spatial directions (usually four) in conjunction with a camera to implement photometric stereo imaging [16]. Given the current size scaling of micro-LED displays and the prospective ability to incorporate light sensing functions through front plane or other forms of integration, it is possible in future that the projected output from a micro-LED display (e.g. from its four corners) can also act to image the environment around it. In this configuration, the micro-LEDs are able to provide 3D photometric stereo imaging, visible light positioning through received signal strength, and a LiFi data signal all at the same time. As illustrated in Figure 6, photometric stereo imaging relies on illumination from different directions, where the camera resolves the differences in surface shading according to each illumination angle. By modulating the corner LEDs with suitable orthogonal multiple access carrier signals, photometric stereo-imaging can be made robust to background illumination from other light sources or the display itself [17].  The different operating modes of micro-LED arrays are matched to the properties of suited detector technology. LiFi and visible light positioning is often optimized for photo-diodes or avalanche photodiodes, which offer a high bandwidth and a high dynamic range. Optical camera communications and photometric stereo-imaging are usually designed to work optimally with CMOS image sensors. A recent detector technology that offers particularly exciting prospects are silicon single photon avalanche diodes (SPADs). These devices are fabricated by standard CMOS processes and have a high sensitivity, enabling low light level systems to operate significantly closer to the shot noise limit than with other types of detectors [18]. Furthermore, the individual photon counts by these devices have an exquisite time resolution on the order of 100 ps.   Figure 6. Photometric stereo 3D imaging utilizes illumination from different directions.  The first active-matrix GaN micro-LED arrays [6] were specifically designed to operate in conjunction with SPAD detectors, with the target application at the time being fluorescence lifetime imaging. Recent progress in the development of digitally interfaced high density SPAD arrays [19] has reinvigorated the interest in structured illumination systems operating at the photon level, extending the application spectrum to 3D ranging [8], low size, weight and power long distance communications [20], and temporally encoded multi-spectral imaging [21]. 3. Impact Micro-LED technology is currently being seen largely as a new form of display technology fitting established device formats and market areas, albeit one which can achieve entirely new levels of performance. This view underestimates the likely impact being offered by the compatibility of this technology with sophisticated control electronics, especially Si CMOS. CMOS-enabled active-matrix control readily accesses the high modulation bandwidths of GaN micro-LED pixels, which are well beyond the switching speeds of other display technologies. Multi-modal operation of displays is now in prospect, where the display can act as both a communications and image representation medium and is also able to sense and interact with its environment in a variety of ways. We will review these exciting emerging capabilities and technological convergences and speculate on how they will develop. 4. Acknowledgments We acknowledge support from the Engineering and Physical Sciences research Council (EPSRC) under grants EP/K00042X/1, EP/M01326X/1, and EP/S001751/1. Data are available online at https://doi.org/10.15129/a8d15d82-e8c0-42f1-8333-4d2adae8abd9. 5. References [1] Dawson MD and Neil MAA (Eds). "Special Cluster Issue: Micro-pixellated LEDs for Science and Instrumentation” J. Phys D: Applied Physics, Vol. 41 (9), 2008. [2] Haas H, Yin L, Wang Y, and Chen C, “What is LiFi?”, J. Lightwave Technol., Vol 34 (6), 1533 , 2016. [3] Rajbhandari S et al., "A Review of Gallium Nitride LEDs for Multi-Gb/s Visible Light Data Communications", Semiconductor Science and Technology, Vol. 32, 023001, 2017. [4] O’Brien D, Rajbhandari S, and Chun H, “Transmitter and receiver technologies for optical wireless”, Phil. Trans. R. Soc. A, Vol. 378, 20190182, 2020. [5] Griffiths AD, Herrnsdorf J, McKendry JJD, Strain MJ, and Dawson MD, “Gallium nitride micro-light-emitting diode structured light sources for multi-modal optical wireless communications systems”, Phil. Trans. R. Soc. A, Vol. 378, 20190185, 2020. [6] Rae BR et al., “CMOS driven micro-pixel LEDs integrated with single photon avalanche diodes for time resolved fluorescence measurements”, J. Phys. D: Appl. Phys. 41, 094011, 2008. [7] Liu ZJ, Huang T, Ma J, Liu C, and Lau KM, “Monolithic Integration of AlGaN/GaN HEMT on LED by MOCVD”, IEEE Electron Dev. Lett., 35, 330, 2014. [8] Carreira JFC et al., “Direct integration of micro-LEDs and a SPAD detector on a silicon CMOS chip for data communications and time-of-flight ranging”, Opt Express 28, 6909, 2020. [9] Herrnsdorf J et al., “Active-matrix GaN micro light-emitting diode display with unprecedented brightness”, IEEE Trans. Electron Dev. 62, 1918, 2015. [10] McKendry JJD et al., “Individually addressable AlInGaN micro-LED arrays with CMOS control and subnanosecond output pulses”, IEEE Photon. Technol. Lett. 21, 811, 2009. [11] McKendry JJD et al., “Visible-light communications using a CMOS-controlled micro-light-emitting-diode array”, J. Lightwave Technol. 30, 61, 2012. [12] Rajbhandari S et al., “High-Speed Integrated Visible Light Communication System: Device Constraints and Design Considerations”, IEEE J. Sel. Areas Commun. 33, 1750, 2015. [13] Jalajakumari AVN., “High-Speed Integrated Digital to Light Converter for Short Range Visible Light Communication”, IEEE Photon. Technol. Lett. 29, 118, 2017. [14] Griffiths A et al., “Scalable visible light communications with a micro-LED array projector and high-speed smartphone camera”, Opt. Express 27, 15585, 2019.  [15] Herrnsdorf J, Dawson MD, and Strain MJ, “Positioning and data broadcasting using illumination pattern sequences displayed by LED arrays”, IEEE Trans. Comm., 66, 5582, 2018. [16] Le Francois E, et al., “Top-down illumination photometric stereo imaging using light-emitting diodes and a mobile device”, Paper JTu3A.106, Frontiers in Optics, 102nd OSA Annual Meeting (Washington DC, USA, September 2019). [17] Herrnsdorf J et al., “LED-based Photometric Stereo-Imaging Employing Frequency-Division Multiple Access”, IEEE Photonics Conference, Reston VA, USA, 2018. [18] Steindl B, Hofbauer M, Schneider-Hornstein K, Brandl P, Zimmermann H, “Single-Photon Avalanche Photodiode Based Fiber Optic Receiver for Up to 200 Mb/s”, IEEE J. Sel. Top. Quantum Electron. 24, 3801308, 2018. [19] Hutchings SW et al., “A Reconfigurable 3-D-Stacked SPAD Imager With In-Pixel Histogramming for Flash LIDAR or High-Speed Time-of-Flight Imaging”, IEEE J. Solid-State Circuits, 54, 2947, 2019. [20] Griffiths AD, Herrnsdorf J, Strain MJ, Dawson MD, “High-sensitivity inter-satellite optical communications using LEDs and single photon receivers”, 11th International Conference on Advances in Satellite and Space Communications, Valencia, Spain, 2019. [21] Griffiths AD et al., “Multispectral time-of-flight imaging using light-emitting diodes”, Opt. Express 27, 35485, 2019.     

Micro-LEDs for technological convergence between displays, optical communications, and sensing and imaging systems

Crossref

https://strathprints.strath.ac.uk/72120/1/Dawson_etal_DW2020_Micro_LEDs_for_technological_convergence_between_displays.pdf

Micro-LEDs for technological convergence between displays, optical communications, and sensing and imaging systems

Abstract

Similar works

Full text

Available Versions

University of Strathclyde Institutional Repository

Crossref