Sensory navigation device for blind people

[EN] This paper presents a new Electronic Travel Aid (ETA) 'Acoustic Prototype' which is especially suited to facilitate the navigation of visually impaired users. The device consists of a set of 3-Dimensional Complementary Metal Oxide Semiconductor (3-D CMOS) image sensors based on the three-dimensional integration and Complementary Metal-Oxide Semiconductor (CMOS) processing techniques implemented into a pair of glasses, stereo headphones as well as a Field-Programmable Gate Array (FPGA) used as processing unit. The device is intended to be used as a complementary device to navigation through both open known and unknown environments. The FPGA and the 3D-CMOS image sensor electronics control object detection. Distance measurement is achieved by using chip-integrated technology based on the Multiple Short Time Integration method. The processed information of the object distance is presented to the user via acoustic sounds through stereophonic headphones. The user interprets the information as an acoustic image of the surrounding environment. The Acoustic Prototype transforms the surface of the objects of the real environment into acoustical sounds. The method used is similar to a bat's acoustic orientation. Having good hearing ability, with few weeks training the users are able to perceive not only the presence of an object but also the object form (that is, if the object is round, if it has corners, if it is a car or a box, if it is a cardboard object or if it is an iron or cement object, a tree, a person, a static or moving object). The information is continuously delivered to the user in a few nanoseconds until the device is shut down, helping the end user to perceive the information in real time.The first author would like to acknowledge that this research was funded through the FP6 European project CASBLiP number 027063 and Project number 2062 of the Programa de Apoyo a la Investigacion y Desarrollo 2011 from the Universitat Politecnica de Valencia.Dunai, L.; Peris Fajarnes, G.; Lluna Gil, E.; Defez Garcia, B. (2013). Sensory navigation device for blind people. Journal of Navigation. 66(3):346-362. doi:10.1017/S0373463312000574S34636266

Real time occupant detection in high dynamic range environments

The aim of this thesis is to explore strategies for real-time image segmentation of non-rigid objects in a spatio-temporal domain with a stationary camera within an optical high dynamic range environment. Camera, illumination and segmentation techniques are discussed for image processing in environments which are characterized by large intensity fluctuations and hence a high optical dynamic range (HDR), in particular for vehicle interior surveillance. Since the introduction of the airbag in 1981 numberless lives were saved and bad injuries were avoided. But in recent years the airbag has frequently been in the headlines due to the increasing number of injuries caused by it. To avoid these injuries a new generation of ’smart airbags’ has been designed which shows the ability to inflate in multiple steps and with different volumes. In order to determine the optimal inflation mode for a crash it is necessary to consider information about the interior situation and the occupants of the vehicle. This thesis presents a real-time visual occupant detection and classification system for advanced airbag deployment, utilizing a custom CMOS camera and motion based image segmentation algorithms for embedded systems under adverse illumination conditions. A novel illumination method is presented which combines a set of images flashed with different radiant intensities, which significantly simplifies image segmentation in HDR environments. With a constant exposure time for the imager a single image can be produced with a compressed dynamic range and a simultaneously reduced offset. This makes it possible to capture a vehicle interior under adverse light conditions without using high dynamic range cameras and without losing image detail. The expansion of this active illumination experiment leads to a novel shadow detection and removal technique that produces a shadow-free scene by simulating an artificial infinite illuminant plane over the held of view. Finally a shadowless image without loss of texture details is obtained without any region extraction phase. Furthermore, a texture based segmentation approach for stationary cam-eras is presented which is neither effected by sudden illumination changes nor by shadow effects

Solid-State Imaging in Standard CMOS Processes

The main aim of this work is to investigate the real CMOS imaging possibilities of standard (not CMOS imaging enhanced) 0.5µm and 0.35µm CMOS processes available for in-house fabrication at the Fraunhofer IMS by performing an extensive study of standard available photodetector structures, mainly based on reverse biased p-n junctions and/or metal-oxide-semiconductor capacitors (MOS-C). Moreover, novel concepts of photodetector pixel structures and readout circuits are proposed, modelled, simulated, fabricated, and characterised, that should achieve an improvement in performance as well as new application developments in the area of CMOS imaging systems. The latter, undergoing as a small amount of changes (extra masks, thermal steps, ion implantations, etc.) as possible within the standard CMOS processes mentioned. In this sense, in Chapter 1 a brief review of the fundamentals of silicon oriented photodetection and electronic devices physics is given, some of the basic postulates of which are directly applied to the case of the 0.5µm standard CMOS process available at the Fraunhofer IMS, whose photodetection possibilities are investigated in detail in Chapter 2. Chapter 3 deals with different pixel configuration possibilities to be fabricated in the 0.5µm process. As a potential solution to overcome some of the problems encountered in Chapter 3, in Chapter 4, the possibilities of using separated photoactive and readout regions in a mixed silicon-on-insulator (SOI) based high-voltage CMOS process developed for automotive industry applications are discussed. Moreover, the same 30V thin-film SOI CMOS process is proposed for direct (not using a scintillator material) X-ray scientific CMOS imaging applications, as it is explained in Chapter 5. In Chapter 6, the photodetection possibilities of the recently developed 0.35µm standard CMOS process available at the Fraunhofer IMS are investigated, as well as different pixel configurations possible to be fabricated in this process. Finally, a discussion is carried out regarding the results obtained throughout the enlisted chapters, and new lines of investigation are attempted to be opened based on some of the results obtained in the present investigation.Das Hauptthema dieser Dissertation ist die Untersuchung vorhandener und neuer Photodetektoren für die CMOS-Bildsensorik, insbesondere in Bezug auf die technologischen und optoelektronischen Eigenschaften sowie das Rauschen. Durch die gründliche Charakterisierung vorhandener und neuer Photodetektoren und deren Ausleseschaltungen, hergestellt in den 0,5µm, 0,35µm und 1,0µm SOI Standard CMOS Prozessen, wurde das vorhandene Wissen vertieft und erweitert. Auf dieser Basis wurden neue Bauelemente für die 2D- und 3D-Bildsensorik entworfen und bereits vorhandene optimiert. Zum besseren Verständnis der Bauelementenstruktur, und um die Anzahl von Fertigungsdurchläufen für neuartige Photodetektorbauelementen zu reduzieren, wurde die Prozess- und Bauelemente-Simulationsumgebung (ISE-Synopsys) TCAD genutzt. Ziel dabei war es, neu entwickelte Bauelemente vor einer Fertigung bezüglich ihrer Schlüsselspezifikationen zu optimieren und qualitativ bewerten zu können. Ferner wurden passende Ausleseschaltungen für die Photodetektorbauelemente entwickelt. Somit konnte ein System aus Photodetektor und Ausleseschaltung optimal auf eine Anwendung abgestimmt werden. Um die Entwurfsicherheit von CMOS Photodetektoren und Ausleseschaltungen zu erhöhen, wurde zur Modellierung von Photodetektorbauelementen mit den Softwarepaketen MAPLE und Fortran eine Entwicklungsumgebung aufgebaut. Auf Basis der so gewonnenen Spezifikationen der Photodetektorstrukturen konnten die gesamten Pixel mit der Schaltungsentwicklungsumgebung CADENCE simuliert werden. Anschließend wurden diese Detektoren und Ausleseschaltungen in den 0,5µm und 0,35µm Standard CMOS Prozessen sowie in dem 1,0µm SOI CMOS Prozess hergestellt und getestet. Um das Signal-Rausch-Verhältnis (SNR) zu verbessern, wurden verschiedenen Pixelkonfigurationen mit voneinander getrennten Photoaktiv- und Auslesegebieten untersucht, wie z.B. in den „Photogate“- oder „Buried-Photodiode“ Aktiv Pixelsensoren. Es wurde allerdings gezeigt, dass die Ladungskopplung, die für die Auslese bei solchen Pixel notwendig ist, bei keiner Kombination der Photodetektoren und der Auslesegebiete im 0,5µm Standard CMOS Prozess funktioniert. Anschließend wurden Untersuchungen von unterschiedlichen Pixelalternativen mit innovativen Ausleseprinzipien durchgeführt. Eines der neu entwickelten Ausleseprinzipien wurde am „Charge-Injection-Photogate-Pixel“ getestet. Durch dieses Ausleseprinzip entsteht eine große interne Verstärkung. Danach wurde dieses Ausleseprinzip auf einer SOI-Struktur angewendet. Dabei werden alle Vorteile eines Hochspannungs- und Hochtemperaturprozesses zusammen mit den Möglichkeiten genutzt, die ein Standard CMOS Prozess zur Integration der Ausleseelektronik auf dem selben Chip erlaubt. Im Rahmen dieser Dissertation wurde darüber hinaus die Möglichkeit untersucht, die Indium-Zinn-Oxid (ITO) Schichten im 0,5µm Standard CMOS Prozess als Gate-Material einzusetzen. Dies bewirkt einen höheren Quantumwirkungsgrad im sichtbaren Wellenlängenbereich, wenn es bei Photogate-Pixel angewandt wird. Im Nah-Infrarot Wellenlängenbereich jedoch bringt es keine Vorteile gegenüber dem standardmäßig verwendeten Polysilizium. Anschließend wurden verschiedene Pixelstrukturen im 0,35µm Standard CMOS Prozess entworfen und simuliert. Dabei wurde gezeigt, dass zumindest theoretisch das Ladungskopplungsprinzip bei den auf „Photogates“ und auf „Buried-Photodioden“ basierenden Strukturen funktioniert