This paper presents a QCIF HDR imager where
visual information is simultaneously captured and adaptively
compressed by means of an in-pixel tone mapping scheme [1].
The tone mapping curve (TMC) is calculated from a nonlinear
histogram of the previous image, which serves as
a probability indicator of the distribution of illuminations
within the present frame. The chip produces 7-bit/pixel
images that can map illuminations from 311 10-6lux to 5875
lux in a single frame in a way that each pixel decides when to
stop observing photocurrent integration –with extreme values
captured at 8s and 20 s respectively. Pixels use a 3x3 m2
Nwell-Psubstrate photodiode and an autozeroing technique
for establishing the reset voltage, which cancels most of
the o set contributions created by the analog processing
circuitry. Measured sensitivity is 5.79 V
lux s . Dark current
e ects in the final image are attenuated by an automatic
programming of the DAC top voltage. The chip has been
designed in the 0.35 m OPTO technology from AMS.FEDER TEC2009-1181

Liñán Cembrano, Gustavo

Rodríguez Vázquez, Ángel Benito

Vargas Sierra, Sonia

idUS. Depósito de Investigación Universidad de Sevilla

P28 A QCIF 145dB Imager For Focal PlaneProcessor Chips Using a Tone MappingTechnique in Standard 0.35µm CMOSTechnologyS. Vargas-Sierra, G. Lin˜a´n-Cembrano, A. Rodrı´guez-Va´zquezInstituto de Microelectro´nica de Sevilla (IMSE-CNM), CSIC and Universidad de SevillaAvda. Ame´rico Vespucio s/n, 41092 Sevilla, SpainEmail: sonia@imse-cnm.csic.es, linan@imse-cnm.csic.es, angel@imse-cnm.csic.esAbstract—This paper presents a QCIF HDR imager wherevisual information is simultaneously captured and adaptivelycompressed by means of an in-pixel tone mapping scheme [1].The tone mapping curve (TMC) is calculated from a non-linear histogram of the previous image, which serves asa probability indicator of the distribution of illuminationswithin the present frame. The chip produces 7-bit/pixelimages that can map illuminations from 311×10-6lux to 5875lux in a single frame in a way that each pixel decides when tostop observing photocurrent integration –with extreme valuescaptured at 8s and 20µs respectively. Pixels use a 3x3µm2Nwell-Psubstrate photodiode and an autozeroing techniquefor establishing the reset voltage, which cancels most ofthe offset contributions created by the analog processingcircuitry. Measured sensitivity is 5.79 Vlux·s . Dark currenteffects in the final image are attenuated by an automaticprogramming of the DAC top voltage. The chip has beendesigned in the 0.35µm OPTO technology from AMS.I. IntroductionHigh Dynamic Range (HDR) imagers usually codifyilluminations in the scene non-adaptively, using either longbit-words per pixel –e.g. mantissa exponent [2]– obtainedfrom the combination of images captured at different expo-sures [3], or a fixed compressive function –e.g. logarithmicapproach [4]– among many other possibilities [5]. Thesenon-adaptive approaches usually lead, non-exclusively, toeither high computational costs for the post-processingof the images –in the long bit-words case– or the lossof details and lack of contrast –e.g. log. sensors– due tothe fixed compression. In order to avoid these drawbacks,the proposed system produces an adaptive compression ofilluminations using only 7-bit per pixel.II. Tone Mapping Algorithm for HDR OperationThe key point in the operation of the imager is thecombination of measuring the crossing time between areference signal, Vre f , and the Vph(Ipix, t) voltage –forpixels working in the photocurrent integration mode– andramping up very fast the analog reference at the endof the exposure to allow for poorly illuminated pixelsto intersect Vre f . In either the case, the crossing eventmakes the pixel to get its value from the current statusof a 7-bit globally distributed signal TMC[6:0] whosenon-linear temporal evolution is calculated from a tone-mapping algorithm [1], [6]. This calculation employs anauxiliar image, named Time Stamp image (TS), which isalso generated on chip. TS image information is providedby one out of every four pixels in a 2×2 neighborhood–see Fig. 3(a), and acts as an indicator of the distributionof illuminations in the scene [1]. The generation of theTS image follows the same principle as that of the Tone-Mapped image, as shown in Fig. 1. During photocurrentintegration, when the pixel voltage Vph intersects Vre f , thepixel samples both the status of the 4-bit Time Stampbus TSC[3:0] and the value of the Tone Mapping busTMC[6:0]. The exposure time is divided into 16 non-linearly distributed windows, each of them having a dif-ferent TS value. The duration of every temporal windowhas been selected so that they are compressed towardsthe higher illumination bands, mimicking the natural com-pression (1/Ipix) of the intersection time expressed in(1), and optimized taken into account the distributionof luminances in public HDR image data bases1. In thelast temporal window, which lasts for 1.28ms, the analogreference signal evolves linearly from its previous constantvalue (Vbot = 1V), to a programmable value Vtop in 128steps. Pixels crossing Vre f during this interval sample thevalue 0 in their time stamp information.Tcross(Ipix,Vre f ) =CpixIpix(Vrst − Vre f ) (1)1http://www.cs.ucf.edu/∼reinhard/cdrom/hdr.html.TMC<6:0>TSC<3:0>VrefVph High illuminated pixelVph Low illuminated pixel5678...12345678 0Fig. 1: Signals Involved in HDR Image Acquisition.VphVbufnRSTVref Digital ControlEVALROWREADVcompSRAM7 bits + 4 bitsSHOLDP1Fig. 2: TS Pixel Block Diagram.The temporal evolution of TMC[6:0] is created from thehistogram of the TS image in the previous frame (this iswhy we consider the information in TS as an indicator ofprobability rather than as an exact evaluation, and so itmay fail when the exposure time is too long as comparedto the rate of changes in the image). TMC[6:0] varieslinearly within each temporal window, spanning over anumber of LSBs which is proportional2 to the weight ofthis temporal window in the histogram of the TS image.Just for illustration purposes, if this histogram shows thathalf of the pixels crossed Vre f during temporal window #3,the TMC[6:0] curve will span over 64 codes during thistemporal window. Finally, since the duration of temporalwindows is non-linearly distributed in time, the obtainedprofile for the TM curve is non-linear in time as well (orpiece-wise linear to be more precise). More details aboutthe generation of these curves and the duration of temporalwindows is provided in [1].III. PixelsAs mentioned previously, pixels have been arranged intwo categories:• TS Pixels: including both TMC and TSC samplingcircuitry.• Basic Pixels: including only TMC sampling circuitry.A block level schematic of a TS pixel is shown in Fig. 2.The sensor, a 3 × 3µm2 Nwell/Psubs diode3, works inphotocurrent integration mode. It uses an auto-zeroingtechnique to establish the reset voltage through the com-bined action of a buffer –which in operation isolates thephotodiode capacitor from comparator’s kickback noise–,an analog comparator (where Vre f=Vrst during reset phase)and a PMOS feedback switch P1. Additionally, digitalcircuitry is included to control read and write operationsof the SRAM cells. Signal ROW controls the externalwrite of data row by row –for evaluation and initializationpurposes–, signal EVAL activates internal write operationand signal READ enables external readouts –which areobviously synchronized with the ROW signal. TS pixelscontain 7(TMC)+4(TSC)=11 bits of SRAM, whereas Ba-sic Pixels (BP) do only include 7(TMC) SRAM modules.2This is the most straightforward way of assigning codes to temporalwindows. In our system implementation we can choose among differentoptions including minimum threshold to activate a temporal window, log-arithmic distribution of codes depending on the histogram, enhancementof different bands (priority to dark or priority to bright), etc.3Aperture in metal structures over the diode is 9.75 × 7.30µm2. Dueto this, carriers created within this area can also contribute to thephotogenerated current by reaching the photodiode through diffusion,increasing the effective fill-factor.Pixels are physically arranged as shown in Fig. 3(a).Notice that each TS pixel takes, conceptually, some areafrom its 3 BP neighbors which is used to allocate the4 SRAM modules for TSC storage. Indeed, all pixelshave 8(7TMC+1TSC) SRAM blocks. TSC modules aregrouped in the middle of the 2×2 arrangement –as shownin Fig. 3(a)– and controlled by signals produced in the TSpixel only. The layout of a group of 2×2 pixels is shownin Fig. 3(b). Observe that we have grouped the SRAMmodules in the central vertical region, sharing globalcontrol, digital power and ground lines. This increasesthe attainable pitch and reduces the noise from digitalswitching in the analog blocks.A. Auto-zeroing TechniqueA crucial issue in the operation of the imager is the us-age of an auto-zeroing technique to cancel out most offsetcontributions from the two amplifiers in the pixel. Duringreset phase, the voltage Vrst is applied to the Vre f inputin Fig. 2, and transmitted to the photodiode’s integratingcapacitor through the negative feedback loop created bythe two amplifiers and the reset switch. If we consider thatamplifiers can be efficiently modeled to this purpose bytheir input-referred offset voltage VOx and a finite DC gainAx –where x = B for the Buffer and C for the Comparator–,one finds after simple calculations –which include Taylor’sseries expansion and neglecting second order error terms–that the reset value is approximately established to:Vphrst  (1 + C)−1 · [(1 + B) · (Vrst + VOC) − VOB] (2)where C = 1/AC , and B = 1/AB.Thus, the effective differential voltage applied at compara-tor’s input during operation –including the feedthroughcontribution VFT introduced by the reset switch– is:Ve f f ≈ Vre f − (Vrst − IpixCpix ∆t) − (1 − B)VFT +  (3) = CVrst − B IpixCpix ∆t + C(1 − C)[VOC − (1 − B)VOB] (4)Clearly, most of errors –except the feedthrough, whichis the main error contribution at the end– vanish asthe amplifiers gain is sufficiently high. This, in practice,is translated into a small residual contribution due tothe impossibility of designing very large gain low-poweramplifiers (each amplifier consumes 50nA) within suchsmall area. The pixel design has been made under the 3sigma constraint for all added non-idealities.Basic PixelBasic PixelBasic PixelTS Pixel(a) Pixel Group Organization. (b) Pixel Group Layout.Fig. 3: Pixels Group.VrefiPOWER_ONΦVdac(a) Vre f Distribution CircuitryVref<147:0>VdacnRSTΦVrstVbotPOWER_ON(b) Vre f Distribution SignalsFig. 4: Vre f Distribution Scheme.IV. Chip-Level Additional FunctionalitiesA. Analog Reference GenerationA dynamic biasing mechanism (in Fig. 4) has beendeveloped in order to transmit the Vre f signal to the array.As shown in Fig. 4(b), Vre f must drop very quickly fromVrst to its constant value during most of the exposure(Vbot = 1V), and, in the last window, move from Vbot = 1Vto Vtop in 128 steps to perform a kind of single ramp A-to-D conversion of pixels not crossing Vre f previously. Everyrow is provided with an analog buffer that receives Vre f ,from an on-chip DAC, and drives all the correspondingnodes in its row. Clearly, there will be slight differencesin the final voltage reached by each row due to offset,and other non-idealities. The next step is to switch-off theamplifiers and short-circuit all Vre fi nodes –Fig. 4(a)– tothe DAC’s output. This forces all nodes to reach the samefinal voltage in a shorter time than only using one driver atthe output of the DAC (due to RC effects in wires drivingthe signal to the different points in the array).B. Dark Signal Contribution AttenuationDark current effects are specially noticeable in darkpixels, that may look very noisy in long-exposure shots.In order to attenuate the visual degradation producedby this undesired contribution, we have experimentallymeasured average dark signal contribution IDC and stan-dard deviation σ(IDC) for different exposure times andoperating temperatures. These measurements allow us todiminish the visual effect of dark current in pixels crossingVre f during the last temporal window simply by loweringVtop as shown in Fig. 5, where Idark = IDC + 3σ(IDC).It is worth mentioning that the optimum Vtop level isautomatically generated by the FPGA controlling the chipusing exposure time, DC measurements and the input froman on-chip PTAP sensor.V. Chip ArchitectureThe architecture of the chip is shown in Fig. 6(a), withits core array of 148 × 180 pixels (QCIF + 2 dummyrows and columns on each side). Pixels functionality issupported by additional periphery blocks. An 8-bit DACgenerates the reset voltage Vrst during reset, the fixedvoltage Vbot during the exposition time and finally the128 levels ramp signal from Vbot to Vtop during the lasttemporal window. 148 buffers (one per row) enhance thedynamics of distributing Vre f to the array. Digital controlsignals also employ per-row distributed digital buffers(including clock-tree generation). TSC[3:0] and TMC[6:0]are generated by a Code Generator in gray format. Thiscoding reduces switching at the pixel level to only oneSRAM module at a time (instead of 7) for Basic Pixelsand 2 (instead of 11) for TS Pixels. Read and writeoperations from the array are accomplished by a bank ofsense amplifiers. Image is retrieved row by row and storedin a read buffer (1 row) which outputs images through ahigh-speed 36-bit bus (4 TMC codes + 2 TSC codes at atime –eq. to 43MBytes/s). Fig. 6(b) shows a microscope’scapture.VI. Experimental ResultsDue to paper size limits, we only present here4 a com-parison of images captured from 3 commercial systemsand our chip (see Fig. 7). The Sony Cybershot DSC-W80 [7] –which includes an enhanced sensitivity CCDsensor (S uper HADTM CCD), the Iphone4 camera –whichallows HDR Mode [8] (since iOS 4.2) by using a com-bination of 3 pictures, and the Photonfocus MV-D752E-40-U2-12 [9], which employs the Lin-Log technology. No-ticeably, despite using only half of the codes (128 vs. 256)for image representation, our approach produces an imagewhich is –visually– competitive with the other approaches.Fig. 8 shows the results for the cumulative application offour Sobel filters (-45, 0, 45, 90) –using matlab– over theimages in Fig. 7 in order to illustrate how informationis kept when capturing a HDR scene. Clearly, images inFig. 8(a), (b), exhibit some lack of details near –or within–the bulb, whereas Fig. 8(c) losses the information in thelow contrast printed characters. The results from our chipshow little lower details in the bulb area than those in (c)but much better performance in the low-contrast areas.Besides our sensor also exhibit lower noise than thosein Fig. 8(b), and (c), being only slightly outperformed–in terms of noise– by the results in Fig. 8(a). Table Isummarizes the most important characteristics of the chip.VII. Conclusions and Future WorkWe have presented an imager that automatically adaptsto compress the HDR scene in a 7-bit format usinga Tone-Mapping algorithm with information from theprevious frame. Pixels include auto-zeroing and in-pixelSRAM storage which allows for long exposure shots.4More results will be presented at the conference.Fig. 5: Dark Signal Contribution Mitigation Scheme.TS BPBP BPTS BPBP BPTS BPBP BPTS BPBP BPTS BPBP BPTS BPBP BP(a) Chip Block Diagram(b) Microscope CaptureFig. 6: Chip Block Diagram and Microscope Capture.An automatic dark signal contribution mitigation schemehas been implemented to enhance the visual quality indark areas. Global analog reference to the pixels is dy-namically distributed to allow for low-power, fast, andprecise operation. We are currently working in a megapixelresolution imager using a 130nm 3D technology with pitchestimations below 7µm and fill factor near 100% using BSIapproach.AcknowledgmentThis work is partially funded by TEC2009-11812,CENIT ADAPTA, ONR Grant N000141110312, andFEDER 2007-2013.References[1] S. Vargas-Sierra, G. Lin˜a´n Cembrano, and A. Rodrı´guez-Va´zquez,“High-dynamic range tone-mapping algorithm for focal plane pro-cessors,” in SPIE Microtechnologies, 2011., April 2011.(a) Cybershot (b) Iphone 4(c) LinLog (d) This workFig. 7: Comparison with commercial cameras.(a) Cybershot (b) Iphone 4(c) LinLog (d) This workFig. 8: Processed Images Comparison.TABLE I: Chip CharacteristicsCharacteristic ValueTechnology 3.3V 0.35µm 2P4M AMS OPTOImage Size 180(H)x148(V) (QCIF+dummies)Pitch 33µmPhotodiode NW-Psub, Aperture 9.75×7.3µm2Fill Factor 0.8% (Diode) 6.5% (Aperture)Full Well Capacity 267 ke−Exposure Time 20µs to 8sImage coding 7 bitsSensitivity 5.79V·lux−1·s−1Average Dark Signal 10.8mV·s−1Maximum Dynamic Range 145dB (from 311µlx to 5875lx)Fastest Image Download Time 666µsFastest Operation Power Consumption 562mW@511fps[2] A. Belenky, A. Fish, A. Spivak, and O. Yadid-Pecht, “Global shuttercmos image sensor with wide dynamic range,” Circuits and SystemsII: Express Briefs, IEEE Transactions on, vol. 54, no. 12, pp. 1032–1036, dec. 2007.[3] M. Mase, S. Kawahito, M. Sasaki, Y. Wakamori, and M. Furuta,“A wide dynamic range cmos image sensor with multiple exposure-time signal outputs and 12-bit column-parallel cyclic a/d converters,”Solid-State Circuits, IEEE Journal of, vol. 40, no. 12, pp. 2787 –2795, dec. 2005.[4] H.-Y. Cheng, B. Choubey, and S. Collins, “An integrating widedynamic-range image sensor with a logarithmic response,” ElectronDevices, IEEE Transactions on, vol. 56, no. 11, pp. 2423 –2428,nov. 2009.[5] A. Spivak, A. Belenky, A. Fish, and O. Yadid-Pecht, “Wide-dynamic-range cmos image sensors - comparative performance analysis,”Electron Devices, IEEE Transactions on, vol. 56, no. 11, pp. 2446–2461, nov. 2009.[6] E. Reinhard, G. Ward, S. Pattanaik, and P. Debevec, High DynamicRange Imaging: Acquisition, Display, and Image-Based Lighting.Elsevier / Morgan Kaufmann, 2006.[7] [Online]. Available: http://news.sel.sony.com/assets/Cyber-shot 2007/specs/DSC-W80.pdf[8] [Online]. Available: http://manuals.info.apple.com/en US/iphone user guide.pdf[9] [Online]. Available: http://www.photonfocus.com/html/eng/products/products.php?prodId=55

A QCIF 145dB imager for focal plane processor chips using a tone mapping technique in standard 0.35μm CMOS technology

https://idus.us.es/xmlui/bitstream/11441/79331/1/A%20QCIF%20145dB.pdf

A QCIF 145dB imager for focal plane processor chips using a tone mapping technique in standard 0.35μm CMOS technology

Abstract

Similar works

Full text

Available Versions

idUS. Depósito de Investigación Universidad de Sevilla