Advances on CMOS image sensors

This paper offers an introduction to the technological advances of image sensors designed using complementary metal–oxide–semiconductor (CMOS) processes along the last decades. We review some of those technological advances and examine potential disruptive growth directions for CMOS image sensors and proposed ways to achieve them. Those advances include breakthroughs on image quality such as resolution, capture speed, light sensitivity and color detection and advances on the computational imaging. The current trend is to push the innovation efforts even further as the market requires higher resolution, higher speed, lower power consumption and, mainly, lower cost sensors. Although CMOS image sensors are currently used in several different applications from consumer to defense to medical diagnosis, product differentiation is becoming both a requirement and a difficult goal for any image sensor manufacturer. The unique properties of CMOS process allows the integration of several signal processing techniques and are driving the impressive advancement of the computational imaging. With this paper, we offer a very comprehensive review of methods, techniques, designs and fabrication of CMOS image sensors that have impacted or might will impact the images sensor applications and markets

On evolution of CMOS image sensors

CMOS Image Sensors have become the principal technology in majority of digital cameras. They started replacing the film and Charge Coupled Devices in the last decade with the promise of lower cost, lower power requirement, higher integration and the potential of focal plane processing. However, the principal factor behind their success has been the ability to utilise the shrinkage in CMOS technology to make smaller pixels, and thereby have more resolution without increasing the cost. With the market of image sensors exploding courtesy their inte- gration with communication and computation devices, technology developers improved the CMOS processes to have better optical performance. Nevertheless, the promises of focal plane processing as well as on-chip integration have not been fulfilled. The market is still being pushed by the desire of having higher number of pixels and better image quality, however, differentiation is being difficult for any image sensor manufacturer. In the paper, we will explore potential disruptive growth directions for CMOS Image sensors and ways to achieve the same

A high speed Tri-Vision system for automotive applications

Purpose: Cameras are excellent ways of non-invasively monitoring the interior and exterior of vehicles. In particular, high speed stereovision and multivision systems are important for transport applications such as driver eye tracking or collision avoidance. This paper addresses the synchronisation problem which arises when multivision camera systems are used to capture the high speed motion common in such applications. Methods: An experimental, high-speed tri-vision camera system intended for real-time driver eye-blink and saccade measurement was designed, developed, implemented and tested using prototype, ultra-high dynamic range, automotive-grade image sensors specifically developed by E2V (formerly Atmel) Grenoble SA as part of the European FP6 project – sensation (advanced sensor development for attention stress, vigilance and sleep/wakefulness monitoring). Results : The developed system can sustain frame rates of 59.8 Hz at the full stereovision resolution of 1280 × 480 but this can reach 750 Hz when a 10 k pixel Region of Interest (ROI) is used, with a maximum global shutter speed of 1/48000 s and a shutter efficiency of 99.7%. The data can be reliably transmitted uncompressed over standard copper Camera-Link® cables over 5 metres. The synchronisation error between the left and right stereo images is less than 100 ps and this has been verified both electrically and optically. Synchronisation is automatically established at boot-up and maintained during resolution changes. A third camera in the set can be configured independently. The dynamic range of the 10bit sensors exceeds 123 dB with a spectral sensitivity extending well into the infra-red range. Conclusion: The system was subjected to a comprehensive testing protocol, which confirms that the salient requirements for the driver monitoring application are adequately met and in some respects, exceeded. The synchronisation technique presented may also benefit several other automotive stereovision applications including near and far-field obstacle detection and collision avoidance, road condition monitoring and others.Partially funded by the EU FP6 through the IST-507231 SENSATION project.peer-reviewe

National MEMS Technology Roadmap - Markets, Applications and Devices

MEMS teknologiaa on jo pitkään käytetty lukuisien eri laitteiden valmistamiseen. Osa näistä laitteista on ollut markkinoilla jo useita vuosia, kun taas osa on vasta kehitysvaiheessa. Jotta tutkimus ja kehitystyötä osattaisiin jatkossa kohdistaa oikeille painopistealueille, on tärkeää tietää mihin suuntaan kehitys on menossa. Tämä työ on osa kansallista MEMS teknologioiden tiekartta -projektia ja sen tavoitteena oli selvittää MEMS laitteiden kehityksen suuntaa. Työ toteutettiin laajana kirjallisuustutkimuksena. Lisäksi tulosten tueksi haastateltiin asiantuntijoita Suomen MEMS teollisuudesta. Työssä tarkasteltiin lukuisia jo markkinoilta löytyviä ja vasta kehitteillä olevia MEMS laitteita ja analysoitiin niitä sekä teknisestä että kaupallisesta näkökulmasta. Tutkimuksen perusteella kävi ilmi, että MEMS markkinat ovat pitkään muodostuneet vakiintuneista laitteista kuten mustesuihkupäistä, kiihtyvyysantureista, paineantureista sekä RF suotimista. Lisäksi mikrofonit, gyroskoopit ja optiset laitteet ovat olleet kaupallisesti saatavilla jo pitkään. Markkinat ovat hiljattain alkaneet tehdä tilaa myös uusille MEMS laitteille, joita tulee ulos nopeaa vauhtia. Viimeisimpänä markkinoille tulleita laitteita ovat erilaiset mikrofluidistiikka laitteet, mikrobolometrit sekä yhdistelmäanturit. Pian kaupallisesti saatavia laitteita ovat magnetometrit, automaattitarkennuslaitteet sekä MEMS oskillaattorit. Näiden laitteiden lisäksi kehitteillä on monia uusia MEMS laitteita, jotka saattavat tarjota merkittäviä mahdollisuuksia tulevaisuudessa. Kehitteillä olevia laitteita ovat erilaiset lääketieteelliset laitteet, atomikellot, mikrojäähdyttimet, mikrokaiuttimet, energiantuottolaitteet sekä RFID-laitteet. Kaikki kehitteillä olevista laitteista eivät välttämättä tule menestymään kaupallisesti, mutta jatkuva tutkimustyö osoittaa, että monilla MEMS laitteilla on potentiaalia useissa eri sovelluksissa. Markkinanäkökulmasta tarkasteltuna suurin potentiaali piilee kuluttajaelektroniikka markkinoilla. Muita tulevaisuuden kannalta potentiaalisia markkinoita ovat lääketieteelliset ja teollisuusmarkkinat. Tutkimus osoitti että MEMS laitteiden tutkimukseen ja kehitykseen liittyy monia potentiaalisia painopistealueita tulevaisuudessa. Käyttömahdollisuuksien parantamiseksi monet jo vakiintuneet laitteet kaipaavat vielä parannuksia. Toisaalta, jo olemassa olevia laitteita voidaan hyödyntää uusissa sovelluksissa. Lisäksi monet uusista ja kehitteillä olevista MEMS laitteista vaativat vielä kehitystyötä.MEMS technology has long been applied to the fabrication of various devices from which some have already been in use for several years, whereas others are still under development. In order to find future focus areas in research and development activities in the industry, it is important to know where the development is going. This thesis was conducted as a part of National MEMS technology roadmap, and it aimed for determining the evolution of MEMS devices. The work was conducted as an extensive literature review. In addition, experts from the Finnish MEMS industry were interviewed in order obtain a broader insight to the results. In this thesis various existing and emerging MEMS devices were reviewed and analyzed from technological and commercial perspectives. The study showed that the MEMS market has long been composed of established devices, such as inkjet print-heads, pressure sensors, accelerometers and RF filters. Also gyroscopes, microphones and optical MEMS devices have already been on the market for a long time. Lately, many new devices have started to find their place in the markets. The most recently introduced commercial devices include microfluidic devices, micro bolometers, and combo sensors. There are also a few devices including magnetometers, MEMS oscillators, and auto-focus devices that are currently crossing the gap from R&D to commercialization. In addition to the already available devices, many new MEMS devices are under development, and might offer significant opportunities in the future. These emerging devices include various bioMEMS devices, atomic clocks, micro-coolers, micro speakers, power MEMS devices, and RFID devices. All of the emerging devices might not find commercial success, but the constant stream shows, that there are numerous applications, where MEMS devices could be applied in. From a market point of view, the greatest potential in the future lies in consumer electronics market. Other highly potential markets include medical and industrial markets. The results of the thesis indicate that there are many potential focus areas in the future related to MEMS devices, including improvements of the existing devices in order to gain better utilization, application of the existing devices in new areas, and development work among the emerging devices

Optimum Selection of DNN Model and Framework for Edge Inference

This paper describes a methodology to select the optimum combination of deep neuralnetwork and software framework for visual inference on embedded systems. As a first step, benchmarkingis required. In particular, we have benchmarked six popular network models running on four deep learningframeworks implemented on a low-cost embedded platform. Three key performance metrics have beenmeasured and compared with the resulting 24 combinations: accuracy, throughput, and power consumption.Then, application-level specifications come into play. We propose a figure of merit enabling the evaluationof each network/framework pair in terms of relative importance of the aforementioned metrics for a targetedapplication. We prove through numerical analysis and meaningful graphical representations that only areduced subset of the combinations must actually be considered for real deployment. Our approach can beextended to other networks, frameworks, and performance parameters, thus supporting system-level designdecisions in the ever-changing ecosystem of embedded deep learning technology.Ministerio de Economía y Competitividad (TEC2015-66878-C3-1-R)Junta de Andalucía (TIC 2338-2013)European Union Horizon 2020 (Grant 765866