Evaluation of the color image and video processing chain and visual quality management for consumer systems

With the advent of novel digital display technologies, color processing is increasingly becoming a key aspect in consumer video applications. Today’s state-of-the-art displays require sophisticated color and image reproduction techniques in order to achieve larger screen size, higher luminance and higher resolution than ever before. However, from color science perspective, there are clearly opportunities for improvement in the color reproduction capabilities of various emerging and conventional display technologies. This research seeks to identify potential areas for improvement in color processing in a video processing chain. As part of this research, various processes involved in a typical video processing chain in consumer video applications were reviewed. Several published color and contrast enhancement algorithms were evaluated, and a novel algorithm was developed to enhance color and contrast in images and videos in an effective and coordinated manner. Further, a psychophysical technique was developed and implemented for performing visual evaluation of color image and consumer video quality. Based on the performance analysis and visual experiments involving various algorithms, guidelines were proposed for the development of an effective color and contrast enhancement method for images and video applications. It is hoped that the knowledge gained from this research will help build a better understanding of color processing and color quality management methods in consumer video

유기발광 다이오드 표시장치를 장착한 이동형 시스템의 전력 공급 최적화

학위논문 (박사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2012. 8. 장래혁.오늘날 스마트폰, 태블릿 PC 와 같은 휴대용 전자기기는 고성능의 중앙처리장치 (CPU), 대용량 메모리, 대형 화면, 고속의 무선 인터페이스 등을 탑재함에따라 전 력 소모량이 급속히 증가하여 그 전력 소모는 이미 소형의 랩탑 컴퓨터 수준에 이르고 있다. 성능과 전력 소모량의 측면에서 휴대용 전자기기와 랩탑 컴퓨터 사 이의 구분이 점차 사라지고 있음에도 배터리 및 전력 변환 회로는 기존의 설계 원칙들만을 따라 설계되고 있는 실정이다. 삼성전자의 갤럭시 탭 및 Apple 사의 iPad 등 스마트폰 및 태블릿 PC의 경우 1-cell 직렬 리튬 이온 전지를 사용하는 반 면, 랩탑 컴퓨터의 경우는 제조사에 따라 3-cell 에서 5-cell 직렬 등으로 설계되고 있다. 이는 배터리 출력 전압을 다르게 함으로써 전력 변환 효율에 영향을 준다. 전력 변환 회로의 효율 및 배터리의 수명은 입출력 전압/전류를 비롯한 동작 환경의 영향을 받는다. 휴대용 전자기기에 사용되는 각종 전자부품은 전력 소모를 줄이기 위한 다양한 기능들을 구현하고 있으며, 중앙처리장치의 동적 전압/주파 수 조절 기법 등 공급전압의 변화를 수반하는 기법 역시 다양하게 적용되고 있다. 이는 각 장치의 공급 전압 및 전류의 변화로 인한 전력 변환 회로의 효율의 변화 를 초래한다. 따라서 중앙처리장치, 디스플레이 등 주요 전력 소비 장치의 전력 절감 기법을 개발할 때에는 개별 장치의 전력 소비를 줄이는 것과 동시에 개별 장 치의 동작 행태에 대한 정확한 분석에 기반하여 배터리, 전력 변환회로의 설계가 함께이루어져야 한다. 선행 연구를 통해 배터리의 특성을 고려한 배터리 구성의 최적화 기법이 제안되었다 [1]. 중앙처리장치의 동적 전압/주파수 제어 기법에 이어 유기발광다이오드(OLED) 기반 디스플레이의 동적 구동회로 공급 전압 기법이 제안되었다 [2]. 유기발광다 이오드 디스플레이는 전력 소모 및 시야각 등 기존 액정 표시장치에 비해 여러 우수한 특성으로 인해 주목받고 있는 차세대 디스플레이 장치이다. 유기발광다 이오드 디스플레이의 적은 전력 소모량에도 불구하고 화면의 대형화 및 해상도의 고밀도화에 따라 시스템 전력 소모에서 여전히 큰 비중을 차지하고 있다. 유기발 광다이오드 디스플레이의 동적 구동회로 공급 전압 기법(OLED DVS)는 색상의 변화의 기초한 기존의 유기발광다이오드 디스플레이 전력 절감 기법과는 달리 최 소한의 이미지 왜곡만을 수반하여 대부분의 사진, 동영상 등에 적용가능한 전력 절감 기법이다. 해당 기법은 공급 전압의 변화시킬 필요가 있으며, 이를 시스템에 올바르게 통합시키기 위해서는 전력 변환 회로 및 배터리 구성에 미치는 영향을 고려해야 한다. 본 논문에서는 유기발광다이오드 디스플레이의 전력 소모와 함께 전체 시스 템 효율에 미치는 영향을 함께 고려하여 시스템을 최적화한다. 배터리 구성 역 시 기존의 설계 표준 대신 체계적인 시스템 분석에 기반한 최적화가 시도되었다. 공급전압이 조절 가능한 유기발광다이오드 디스플레이 하드웨어 및 제어기 시스 템-온-칩 (System-on-a-chip, SoC) 가 제작되었고, 그 동작 특성이 분석되었다. 기존 스마트폰 및 태블릿 PC 개발용 플랫폼의 전력 변환 효율 및 동작 특성 역시 분석 되었다. 유기발광다이오드 디스플레이의 동적 구동회로 공급 전압 기법의 동작 특성 및 스마트폰 플랫폼의 동작 특성, 배터리 특성에 대한 분석을 기반으로 시스 템 수준에서의 전력 변환 효율이 최적화되었다.Modern mobile devices such as smartphone or tablet PC are typically equipped a high-performance CPU, memory, wireless interface, and display. As a result, their power consumption is as high as a small-size laptop computer. The boundary between the mobile devices and laptop computer is becoming unclear from the perspective of the performance and power. However, their battery and related power conversion architecture are only designed according to the legacy design so far. Smartphone and tablet PCs from major vendors such as iPad from Apple or Galaxy-tab from Samsung uses 1-cell Li-ion battery. The laptop PC typically has 3-cell Li-ion battery. The output voltage of the battery affect system-level power conversion efficiency. Furthermore, traditional power conversion architecture in the mobile computing system is designed only considering the fixed condition where the system-level low-power techniques such as DVFS are becoming mandatory. Such a low-power techniques applied to the major components result in not only load demand fluctuation but also supply voltage changing. It has an effect on the battery lifetime as well as the system-level power delivery efficiency. The efficiency is affected by the operating condition including input voltage, output voltage, and output current. We should consider the operating condition of the major power consumer such as a display to enhance the system-level power delivery efficiency. Therefore, we need to design the system not only from the perspective of the power consumption but also energy storage design. The optimization of battery setup considering battery characteristics was presented in [1]. Beside the DVFS of microprocessor, a power saving technique based on the supply voltage scaling of the OLED driver circuit was recently introduced [2]. An organic light emitting diode (OLED) is a promising display device which has a lot of advantages compared with conventional LCD, but it still consumes significant amount of power consumption due to the size and resolution increasing. The OLED dynamic voltage scaling (OLED DVS) technique is the first OLED display power saving technique that induces only minimal color change to accommodate display of natural images where the existing OLED low-power techniques are based on the color change. The OLED DVS incurs supply voltage change. Therefore we need to consider the system-level power delivery efficiency and battery setup to properly integrate the DVS-enabled OLED display to the system. In this dissertation, we not only optimize the power consumption of the OLED display but also consider its effect on the whole system power efficiency. We perform the optimization of the battery setup by a systematic method instead of the legacy design rule. At first, we develop an algorithm for the OLED DVS for the still images and a histogram-based online method for the image sequence with a hardware board and a SoC. We characterize the behavior of the OLED DVS. Next, we analyze the characteristics of the smartphone and tablet-PC platforms by using the development platforms. We profile the power consumption of each components in the smartphone and power conversion efficiency of the boost converter which is used in the tablet-PC for the display devices. We optimize not only the power consuming components or the conversion system but also the energy storage system based on the battery model and system-level power delivery efficiency analysis.1 Introduction 1.1 Supply Voltage Scaling for OLED Display 1.2 Power Conversion Efficiency in MobileSystems 1.3 Research Motivation 2 Related Work 2.1 Low-Power Techniques for Display Devices 2.1.1 Light Source Control-Based Approaches 2.1.2 User Behavior-Based Approaches 2.1.3 Low-Power Techniques for Controller and Framebuffer 2.1.4 Pre-ChargingforOLED 2.1.5 ColorRemapping 2.2 Battery discharging efficiency aware low-power techniques 2.2.1 Parallel Connection 2.2.2 Constant-Current Regulator-Based Architecture 2.3 System-level power analysis techniques 3 Preliminary 38 3.1 Organic Light Emitting Diode (OLED) Display 3.1.1 OLED Cell Architecture 3.1.2 OLED Panel Architecture 3.1.3 OLED Driver Circuits 3.2 Effect of VDD scaling on driver circuits 3.2.1 VDD scaling for AM drivers 3.2.2 VDD scaling for PWM drivers 4 Supply Voltage Scaling and Image Compensation of OLED displays 4.1 Image quality and power models of OLED panels 4.2 OLED display characterization 4.3 VDD scaling and image compensation 5 OLED DVS implementation 5.1 Hardware prototype implementation 5.2 OLED DVS System-on-Chip implementation 5.3 Optimization of OLED DVS SoC 5.4 VDD transition overhead 6 Power conversion efficiency and delivery architecture in mobile Systems 6.1 Power conversion efficiency model of switching-Mode DC–DC converters 6.2 Power conversion efficiency model of linear regulator power loss model 6.3 Rate Capacity Effect of Li-ion Batteries 7 Power conversion efficiency-aware battery setup optimization with DVS- enabled OLED display 7.1 System-level power efficiency model 7.2 Power conversion efficiency analysis of smartphone platform 7.3 Power conversion efficiency for OLED power supply 7.4 Li-ion battery model 7.4.1 Battery model parameter extraction 7.5 Battery setup optimization 8 Experiments 8.1 Simulation result for OLED display with AM driver 8.2 Measurement result for OLED display with PWM driver 8.3 Design space exploration of battery setup with OLED displays 9 Conclusion 10 Future WorkDocto

Appearance-based image splitting for HDR display systems

High dynamic range displays that incorporate two optically-coupled image planes have recently been developed. This dual image plane design requires that a given HDR input image be split into two complementary standard dynamic range components that drive the coupled systems, therefore there existing image splitting issue. In this research, two types of HDR display systems (hardcopy and softcopy HDR display) are constructed to facilitate the study of HDR image splitting algorithm for building HDR displays. A new HDR image splitting algorithm which incorporates iCAM06 image appearance model is proposed, seeking to create displayed HDR images that can provide better image quality. The new algorithm has potential to improve image details perception, colorfulness and better gamut utilization. Finally, the performance of the new iCAM06-based HDR image splitting algorithm is evaluated and compared with widely spread luminance square root algorithm through psychophysical studies

Gamut extension algorithm development and evaluation for the mapping of standard image content to wide-gamut displays

Wide-gamut display technology has provided an excellent opportunity to produce visually pleasing images, more so than in the past. However, through several studies, including Laird and Heynderick, 2008, it was shown that linearly mapping the standard sRGB content to the gamut boundary of a given wide-gamut display may not result in optimal results. Therefore, several algorithms were developed and evaluated for observer preference, including both linear and sigmoidal expansion algorithms, in an effort to define a single, versatile gamut expansion algorithm (GEA) that can be applied to current display technology and produce the most preferable images for observers. The outcome provided preference results from two displays, both of which resulted in large scene dependencies. However, the sigmoidal GEAs (SGEA) were competitive with the linear GEAs (LGEA), and in many cases, resulted in more pleasing reproductions. The SGEAs provide an excellent baseline, in which, with minor improvements, could be key to producing more impressive images on a wide-gamut display

Tone mapping for high dynamic range images

Tone mapping is an essential step for the reproduction of "nice looking" images. It provides the mapping between the luminances of the original scene to the output device's display values. When the dynamic range of the captured scene is smaller or larger than that of the display device, tone mapping expands or compresses the luminance ratios. We address the problem of tone mapping high dynamic range (HDR) images to standard displays (CRT, LCD) and to HDR displays. With standard displays, the dynamic range of the captured HDR scene must be compressed significantly, which can induce a loss of contrast resulting in a loss of detail visibility. Local tone mapping operators can be used in addition to the global compression to increase the local contrast and thus improve detail visibility, but this tends to create artifacts. We developed a local tone mapping method that solves the problems generally encountered by local tone mapping algorithms. Namely, it does not create halo artifacts, nor graying-out of low contrast areas, and provides good color rendition. We then investigated specifically the rendition of color and confirmed that local tone mapping algorithms must be applied to the luminance channel only. We showed that the correlation between luminance and chrominance plays a role in the appearance of the final image but a perfect decorrelation is not necessary. Recently developed HDR monitors enable the display of HDR images with hardly any compression of their dynamic range. The arrival of these displays on the market create the need for new tone mapping algorithms. In particular, legacy images that were mapped to SDR displays must be re-rendered to HDR displays, taking best advantage of the increase in dynamic range. This operation can be seen as the reverse of the tone mapping to SDR. We propose a piecewise linear tone scale function that enhances the brightness of specular highlights so that the sensation of naturalness is improved. Our tone scale algorithm is based on the segmentation of the image into its diffuse and specular components as well as on the range of display luminance that is allocated to the specular component and the diffuse component, respectively. We performed a psychovisual experiment to validate the benefit of our tone scale. The results showed that, with HDR displays, allocating more luminance range to the specular component than what was allocated in the image rendered to SDR displays provides more natural looking images

Flat panel display signal processing

Televisions (TVs) have shown considerable technological progress since their introduction almost a century ago. Starting out as small, dim and monochrome screens in wooden cabinets, TVs have evolved to large, bright and colorful displays in plastic boxes. It took until the turn of the century, however, for the TV to become like a ‘picture on the wall’. This happened when the bulky Cathode Ray Tube (CRT) was replaced with thin and light-weight Flat Panel Displays (FPDs), such as Liquid Crystal Displays (LCDs) or Plasma Display Panels (PDPs). However, the TV system and transmission formats are still strongly coupled to the CRT technology, whereas FPDs use very different principles to convert the electronic video signal to visible images. These differences result in image artifacts that the CRT never had, but at the same time provide opportunities to improve FPD image quality beyond that of the CRT. This thesis presents an analysis of the properties of flat panel displays, their relation to image quality, and video signal processing algorithms to improve the quality of the displayed images. To analyze different types of displays, the display signal chain is described using basic principles common to all displays. The main function of a display is to create visible images (light) from an electronic signal (video), requiring display chain functions like opto-electronic effect, spatial and temporal addressing and reconstruction, and color synthesis. The properties of these functions are used to describe CRT, LCDs, and PDPs, showing that these displays perform the same functions, using different implementations. These differences have a number of consequences, that are further investigated in this thesis. Spatial and temporal aspects, corresponding to ‘static’ and ‘dynamic’ resolution respectively, are covered in detail. Moreover, video signal processing is an essential part of the display signal chain for FPDs, because the display format will in general no longer match the source format. In this thesis, it is investigated how specific FPD properties, especially related to spatial and temporal addressing and reconstruction, affect the video signal processing chain. A model of the display signal chain is presented, and applied to analyze FPD spatial properties in relation to static resolution. In particular, the effect of the color subpixels, that enable color image reproduction in FPDs, is analyzed. The perceived display resolution is strongly influenced by the color subpixel arrangement. When taken into account in the signal chain, this improves the perceived resolution on FPDs, which clearly outperform CRTs in this respect. The cause and effect of this improvement, also for alternative subpixel arrangements, is studied using the display signal model. However, the resolution increase cannot be achieved without video processing. This processing is efficiently combined with image scaling, which is always required in the FPD display signal chain, resulting in an algorithm called ‘subpixel image scaling’. A comparison of the effects of subpixel scaling on several subpixel arrangements shows that the largest increase in perceived resolution is found for two-dimensional subpixel arrangements. FPDs outperform CRTs with respect to static resolution, but not with respect to ‘dynamic resolution’, i.e. the perceived resolution of moving images. Life-like reproduction of moving images is an important requirement for a TV display, but the temporal properties of FPDs cause artifacts in moving images (‘motion artifacts’), that are not found in CRTs. A model of the temporal aspects of the display signal chain is used to analyze dynamic resolution and motion artifacts on several display types, in particular LCD and PDP. Furthermore, video signal processing algorithms are developed that can reduce motion artifacts and increase the dynamic resolution. The occurrence of motion artifacts is explained by the fact that the human visual system tracks moving objects. This converts temporal effects on the display into perceived spatial effects, that can appear in very different ways. The analysis shows how addressing mismatches in the chain cause motion-dependent misalignment of image data, e.g. resulting in the ‘dynamic false contour’ artifact in PDPs. Also, non-ideal temporal reconstruction results in ‘motion blur’, i.e. a loss of sharpness of moving images, which is typical for LCDs. The relation between motion blur, dynamic resolution, and temporal properties of LCDs is analyzed using the display signal model in the temporal (frequency) domain. The concepts of temporal aperture, motion aperture and temporal display bandwidth are introduced, which enable characterization of motion blur in a simple and direct way. This is applied to compare several motion blur reduction methods, based on modified display design and driving. This thesis further describes the development of several video processing algorithms that can reduce motion artifacts. It is shown that the motion of objects in the image plays an essential role in these algorithms, i.e. they require motion estimation and compensation techniques. In LCDs, video processing for motion artifact reduction involves a compensation for the temporal reconstruction characteristics of the display, leading to the ‘motion compensated inverse filtering’ algorithm. The display chain model is used to analyze this algorithm, and several methods to increase its performance are presented. In PDPs, motion artifact reduction can be achieved with ‘motion compensated subfield generation’, for which an advanced algorithm is presented