Virtual pan-tilt-zoom for a wide-area-video surveillance system

Thesis (M. Eng.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2008.Includes bibliographical references (p. 95).Advancements in the CMOS Image Sensor have enabled very high-performance, high resolution imaging systems to be built at relatively low cost. The availability of high-pixel count video imaging systems that can cover a wide field-of-view enables a surveillance technique called Virtual Pan-Tilt-Zoom. Virtual Pan-Tilt-Zoom provides the same functional properties as a mechanical pan-tilt-zoom setup, but it does not suffer from the physical limitations presented by a mechanical setup. A video system using Virtual Pan-Tilt-Zoom would have immediate continuous access to a high pixel-count image representing a wide coverage area, and it would enable a user to "virtually" pan, tilt, and zoom around the coverage area by reading out only the relevant image data associated with a Region of Interest that is dynamically defined by the user. This paper will examine the various camera electronics readout architectures that are possible to support the Virtual Pan-Tilt-Zoom function. Then, this project will examine and implement a specific implementation of the readout architecture for a high-resolution video camera system developed at MIT Lincoln Laboratory. The Multi-Aperture Sparse Imager Video System (MASIV) developed at MIT Lincoln Laboratory incorporates CMOS imagers to create an 880 Mega pixel image, and was used as the platform to implement the camera electronics for Virtual Pan-Tilt-Zoom functionality.by Richard Sinn.M.Eng

Digital photography and the dynamics of technology innovation

Thesis (S.M.)--Massachusetts Institute of Technology, System Design & Management Program, 2002.Includes bibliographical references (leaf 96).Companies heavily and successfully invested in traditional technologies (defenders) often find it difficult to make the transitions to new disruptive technologies, in spite of technological competence and clear opportunity to do so. The core competencies that enabled the firm to excel under the old paradigms become core rigidities when faced with the need to address technological discontinuities. Products like digital still cameras, DSCs, represent the convergence of multiple rapidly changing technologies in electronics, optics, computers, networks, and software. The emergence and adoption of digital still photography both accompanies and defines a new paradigm in the sharing of images as it attempts to both emulate and replace the previous modalities while creating new market-expanding opportunities. The emergence of digital still photography has been predicted and promised for several decades. Indeed, it has already managed to replace silver halide altogether in certain market segments previously relied upon by conventional photography firms, and is at present extending beyond the early adopter stage in the broader consumer market. It is a current example of innovation and technological discontinuity, and one that has enough history to permit analysis. It poses a real potential disruptive threat to the incumbent players, some of which have succumbed while others apparently succeeded. This thesis studies the relationships between the development of the composite technologies in digital photography, the environment in which they operate, the emergence of dominant designs, market diffusion, and the strategies for success employed by leading participants. In the process of studying patterns of entry and exit firms and a detailed look at their products, evidence of a dominant design and support in this industry for the Abernathy and Utterback model of industrial innovation is uncovered. Also revealed is a second wave of innovation in the DSC industry that is firmly established and suggests the onset of a Christiansen-style disruptive dynamic. By studying this specific technological discontinuity in the context of the broader patterns, lessons in adapting to technological change in general are learned.by J. Peter Zelten.S.M

Empowering Low-Cost CMOS Cameras by Image Processing to Reach Comparable Results with Costly CCDs

Despite the huge research effort to improve the performance of the complementary metal oxide semiconductor (CMOS) image sensors, charge-coupled devices (CCDs) still dominate the cell biology-related conventional fluorescence microscopic imaging market where low or ultra-low noise imaging is required. A detailed comparison of the sensor specifications and performance is usually not provided by the manufacturers which leads the end users not to go out of the habitude and choose a CCD camera instead of a CMOS one. However, depending on the application, CMOS cameras, when empowered by image processing algorithms, can become cost-efficient solutions for conventional fluorescence microscopy. In this paper, we introduce an application-based comparative study between the default CCD camera of an inverted microscope (Nikon Ti-S Eclipse) and a custom-designed CMOS camera and apply efficient image processing algorithms to improve the performance of CMOS cameras. Quantum micro-bead samples (emitting fluorescence light at different intensity levels), breast cancer diagnostic tissue cell samples, and Caco-2 cell samples are imaged by both CMOS and CCD cameras. The results are provided to show the reliability of CMOS camera processed images and finally to be of assistance when scientists select their cameras for desired applications

CMOS Active Pixel Sensors for Digital Cameras: Current State-of-the-Art

Image sensors play a vital role in many image sensing and capture applications. Among the various types of image sensors, complementary metal oxide semiconductor (CMOS) based active pixel sensors (APS), which are characterized by reduced pixel size, give fast readouts and reduced noise. APS are used in many applications such as mobile cameras, digital cameras, Webcams, and many consumer, commercial and scientific applications. With these developments and applications, CMOS APS designs are challenging the old and mature technology of charged couple device (CCD) sensors. With the continuous improvements of APS architecture, pixel designs, along with the development of nanometer CMOS fabrications technologies, APS are optimized for optical sensing. In addition, APS offers very low-power and low-voltage operations and is suitable for monolithic integration, thus allowing manufacturers to integrate more functionality on the array and building low-cost camera-on-a-chip. In this thesis, I explore the current state-of-the-art of CMOS APS by examining various types of APS. I show design and simulation results of one of the most commonly used APS in consumer applications, i.e. photodiode based APS. We also present an approach for technology scaling of the devices in photodiode APS to present CMOS technologies. Finally, I present the most modern CMOS APS technologies by reviewing different design models. The design of the photodiode APS is implemented using commercial CAD tools

Star Imager For Nanosatellite Applications

This research examines the feasibility of Commercial-off-the-shelf Complementary Metal-Oxide-Semiconductor image sensors for use on nanosatellites as a star imager. An emphasis is placed on method selection and implementation of the star imager algorithm: Centroiding, Identification and Attitude Determination. The star imager algorithm makes use of the Lost-in-Space condition to provide attitude knowledge for each image. Flat Field, Checker Board and Point Spread Function calibration methods were employed to characterize the star imager. Finally, feasibility testing of the star imager is accomplished through simulations and night sky images

Low-power CMOS digital-pixel Imagers for high-speed uncooled PbSe IR applications

This PhD dissertation describes the research and development of a new low-cost medium wavelength infrared MWIR monolithic imager technology for high-speed uncooled industrial applications. It takes the baton on the latest technological advances in the field of vapour phase deposition (VPD) PbSe-based medium wavelength IR (MWIR) detection accomplished by the industrial partner NIT S.L., adding fundamental knowledge on the investigation of novel VLSI analog and mixed-signal design techniques at circuit and system levels for the development of the readout integrated device attached to the detector. The work supports on the hypothesis that, by the use of the preceding design techniques, current standard inexpensive CMOS technologies fulfill all operational requirements of the VPD PbSe detector in terms of connectivity, reliability, functionality and scalability to integrate the device. The resulting monolithic PbSe-CMOS camera must consume very low power, operate at kHz frequencies, exhibit good uniformity and fit the CMOS read-out active pixels in the compact pitch of the focal plane, all while addressing the particular characteristics of the MWIR detector: high dark-to-signal ratios, large input parasitic capacitance values and remarkable mismatching in PbSe integration. In order to achieve these demands, this thesis proposes null inter-pixel crosstalk vision sensor architectures based on a digital-only focal plane array (FPA) of configurable pixel sensors. Each digital pixel sensor (DPS) cell is equipped with fast communication modules, self-biasing, offset cancellation, analog-to-digital converter (ADC) and fixed pattern noise (FPN) correction. In-pixel power consumption is minimized by the use of comprehensive MOSFET subthreshold operation. The main aim is to potentiate the integration of PbSe-based infra-red (IR)-image sensing technologies so as to widen its use, not only in distinct scenarios, but also at different stages of PbSe-CMOS integration maturity. For this purpose, we posit to investigate a comprehensive set of functional blocks distributed in two parallel approaches: • Frame-based “Smart” MWIR imaging based on new DPS circuit topologies with gain and offset FPN correction capabilities. This research line exploits the detector pitch to offer fully-digital programmability at pixel level and complete functionality with input parasitic capacitance compensation and internal frame memory. • Frame-free “Compact”-pitch MWIR vision based on a novel DPS lossless analog integrator and configurable temporal difference, combined with asynchronous communication protocols inside the focal plane. This strategy is conceived to allow extensive pitch compaction and readout speed increase by the suppression of in-pixel digital filtering, and the use of dynamic bandwidth allocation in each pixel of the FPA. In order make the electrical validation of first prototypes independent of the expensive PbSe deposition processes at wafer level, investigation is extended as well to the development of affordable sensor emulation strategies and integrated test platforms specifically oriented to image read-out integrated circuits. DPS cells, imagers and test chips have been fabricated and characterized in standard 0.15μm 1P6M, 0.35μm 2P4M and 2.5μm 2P1M CMOS technologies, all as part of research projects with industrial partnership. The research has led to the first high-speed uncooled frame-based IR quantum imager monolithically fabricated in a standard VLSI CMOS technology, and has given rise to the Tachyon series [1], a new line of commercial IR cameras used in real-time industrial, environmental and transportation control systems. The frame-free architectures investigated in this work represent a firm step forward to push further pixel pitch and system bandwidth up to the limits imposed by the evolving PbSe detector in future generations of the device.La present tesi doctoral descriu la recerca i el desenvolupament d'una nova tecnologia monolítica d'imatgeria infraroja de longitud d'ona mitja (MWIR), no refrigerada i de baix cost, per a usos industrials d'alta velocitat. El treball pren el relleu dels últims avenços assolits pel soci industrial NIT S.L. en el camp dels detectors MWIR de PbSe depositats en fase vapor (VPD), afegint-hi coneixement fonamental en la investigació de noves tècniques de disseny de circuits VLSI analògics i mixtes pel desenvolupament del dispositiu integrat de lectura unit al detector pixelat. Es parteix de la hipòtesi que, mitjançant l'ús de les esmentades tècniques de disseny, les tecnologies CMOS estàndard satisfan tots els requeriments operacionals del detector VPD PbSe respecte a connectivitat, fiabilitat, funcionalitat i escalabilitat per integrar de forma econòmica el dispositiu. La càmera PbSe-CMOS resultant ha de consumir molt baixa potència, operar a freqüències de kHz, exhibir bona uniformitat, i encabir els píxels actius CMOS de lectura en el pitch compacte del pla focal de la imatge, tot atenent a les particulars característiques del detector: altes relacions de corrent d'obscuritat a senyal, elevats valors de capacitat paràsita a l'entrada i dispersions importants en el procés de fabricació. Amb la finalitat de complir amb els requisits previs, es proposen arquitectures de sensors de visió de molt baix acoblament interpíxel basades en l'ús d'una matriu de pla focal (FPA) de píxels actius exclusivament digitals. Cada píxel sensor digital (DPS) està equipat amb mòduls de comunicació d'alta velocitat, autopolarització, cancel·lació de l'offset, conversió analògica-digital (ADC) i correcció del soroll de patró fixe (FPN). El consum en cada cel·la es minimitza fent un ús exhaustiu del MOSFET operant en subllindar. L'objectiu últim és potenciar la integració de les tecnologies de sensat d'imatge infraroja (IR) basades en PbSe per expandir-ne el seu ús, no només a diferents escenaris, sinó també en diferents estadis de maduresa de la integració PbSe-CMOS. En aquest sentit, es proposa investigar un conjunt complet de blocs funcionals distribuïts en dos enfocs paral·lels: - Dispositius d'imatgeria MWIR "Smart" basats en frames utilitzant noves topologies de circuit DPS amb correcció de l'FPN en guany i offset. Aquesta línia de recerca exprimeix el pitch del detector per oferir una programabilitat completament digital a nivell de píxel i plena funcionalitat amb compensació de la capacitat paràsita d'entrada i memòria interna de fotograma. - Dispositius de visió MWIR "Compact"-pitch "frame-free" en base a un novedós esquema d'integració analògica en el DPS i diferenciació temporal configurable, combinats amb protocols de comunicació asíncrons dins del pla focal. Aquesta estratègia es concep per permetre una alta compactació del pitch i un increment de la velocitat de lectura, mitjançant la supressió del filtrat digital intern i l'assignació dinàmica de l'ample de banda a cada píxel de l'FPA. Per tal d'independitzar la validació elèctrica dels primers prototips respecte a costosos processos de deposició del PbSe sensor a nivell d'oblia, la recerca s'amplia també al desenvolupament de noves estratègies d'emulació del detector d'IR i plataformes de test integrades especialment orientades a circuits integrats de lectura d'imatge. Cel·les DPS, dispositius d'imatge i xips de test s'han fabricat i caracteritzat, respectivament, en tecnologies CMOS estàndard 0.15 micres 1P6M, 0.35 micres 2P4M i 2.5 micres 2P1M, tots dins el marc de projectes de recerca amb socis industrials. Aquest treball ha conduït a la fabricació del primer dispositiu quàntic d'imatgeria IR d'alta velocitat, no refrigerat, basat en frames, i monolíticament fabricat en tecnologia VLSI CMOS estàndard, i ha donat lloc a Tachyon, una nova línia de càmeres IR comercials emprades en sistemes de control industrial, mediambiental i de transport en temps real.Postprint (published version