A PCA-based super-resolution algorithm for short image sequences

Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works. C. Miravet, and F. B. Rodríguez, "A PCA-based super-resolution algorithm for short image sequences", 17th IEEE International Conference on Image Processing (ICIP), Hong Kong, China, 2010, pp. 2025 - 2028In this paper, we present a novel, learning-based, two-step super-resolution (SR) algorithm well suited to solve the specially demanding problem of obtaining SR estimates from short image sequences. The first step, devoted to increase the sampling rate of the incoming images, is performed by fitting linear combinations of functions generated from principal components (PC) to reproduce locally the sparse projected image data, and using these models to estimate image values at nodes of the high-resolution grid. PCs were obtained from local image patches sampled at sub-pixel level, which were generated in turn from a database of high-resolution images by application of a physically realistic observation model. Continuity between local image models is enforced by minimizing an adequate functional in the space of model coefficients. The second step, dealing with restoration, is performed by a linear filter with coefficients learned to restore residual interpolation artifacts in addition to low-resolution blurring, providing an effective coupling between both steps of the method. Results on a demanding five-image scanned sequence of graphics and text are presented, showing the excellent performance of the proposed method compared to several state-of-the-art two-step and Bayesian Maximum a Posteriori SR algorithms.This work was supported by the Spanish Ministry of Education and Science under TIN 2007-65989 and CAM S-SEM-0255- 2006, and by COINCIDENTE project DN8644, RESTAURA

Robust super-resolution for petite image sequences

This paper proposes a new, learning-based, two-step super-resolution (SR) algorithm well suited to solve the specially demanding problem of obtaining SR estimates from petite image sequences. The first step, devoted to increase the sampling rate of the incoming images, is performed by fitting linear combinations of functions generated from principal components (PC) to reproduce locally the sparse projected image data, and using these models to estimate image values at nodes of the high-resolution grid. PCs were obtained from local image patches sampled at sub-pixel level, which were generated in turn from a database of high-resolution images by application of a physically realistic observation model. Continuity between local image models is enforced by minimizing an adequate functional in the space of model coefficients. The second step, dealing with restoration, is performed by a linear filter with coefficients learned to restore residual interpolation artifacts in addition to low-resolution blurring, providing an effective coupling between both steps of the method. Results on a demanding five-image scanned sequence of graphics and text are presented, showing the excellent performance of the proposed method compared to several state-of-the-art two-step and Bayesian Maximum a Posteriori SR algorithms

Superresolution imaging: A survey of current techniques

Cristóbal, G., Gil, E., Šroubek, F., Flusser, J., Miravet, C., Rodríguez, F. B., “Superresolution imaging: A survey of current techniques”, Proceedings of SPIE - The International Society for Optical Engineering, 7074, 2008. Copyright 2008. Society of Photo Optical Instrumentation Engineers. One print or electronic copy may be made for personal use only. Systematic reproduction and distribution, duplication of any material in this paper for a fee or for commercial purposes, or modification of the content of the paper are prohibited.Imaging plays a key role in many diverse areas of application, such as astronomy, remote sensing, microscopy, and tomography. Owing to imperfections of measuring devices (e.g., optical degradations, limited size of sensors) and instability of the observed scene (e.g., object motion, media turbulence), acquired images can be indistinct, noisy, and may exhibit insufficient spatial and temporal resolution. In particular, several external effects blur images. Techniques for recovering the original image include blind deconvolution (to remove blur) and superresolution (SR). The stability of these methods depends on having more than one image of the same frame. Differences between images are necessary to provide new information, but they can be almost unperceivable. State-of-the-art SR techniques achieve remarkable results in resolution enhancement by estimating the subpixel shifts between images, but they lack any apparatus for calculating the blurs. In this paper, after introducing a review of current SR techniques we describe two recently developed SR methods by the authors. First, we introduce a variational method that minimizes a regularized energy function with respect to the high resolution image and blurs. In this way we establish a unifying way to simultaneously estimate the blurs and the high resolution image. By estimating blurs we automatically estimate shifts with subpixel accuracy, which is inherent for good SR performance. Second, an innovative learning-based algorithm using a neural architecture for SR is described. Comparative experiments on real data illustrate the robustness and utilization of both methods.This research has been partially supported by the following grants: TEC2007-67025/TCM, TEC2006-28009-E, BFI-2003-07276, TIN-2004-04363-C03-03 by the Spanish Ministry of Science and Innovation, and by PROFIT projects FIT-070000-2003-475 and FIT-330100-2004-91. Also, this work has been partially supported by the Czech Ministry of Education under the project No. 1M0572 (Research Center DAR) and by the Czech Science Foundation under the project No. GACR 102/08/1593 and the CSIC-CAS bilateral project 2006CZ002

Towards Real-Time Hyperspectral Multi-Image Super-Resolution Reconstruction Applied to Histological Samples

publishedVersio

不完全な観測画像列を用いた超解像再構成

本論文は，観測画像が数多く取得できない場合における超解像再構成に関して検討した 結果をまとめたものである． 常に解像度の向上を求められる画像計測において，得られた観測画像をソフトウェア的に 高解像度化できる超解像再構成技術は，撮像素子の高密度化技術のようなハードウェア的な 高解像度化手段と併用できることもあり，非常に有益である．しかし，超解像再構成を可能に するためには，観測画像の画素数の総合計（既知数：方程式数）が目的となる倍率の超解像 画像の画素数（未知数）と同じか，多くなければならない．即ち，同一箇所における同一観測 条件の取得画像を数多く必要とする．しかし，衛星リモートセンシングや定常ではない観測対 象のモニタリングのように，同一地点における同一条件の観測画像が数多く取得できないため に，高解像度化を必要としても超解像再構成技術が適用できない場合も数多く存在する． そこで本論文では，観測画像列が不完全な，即ち観測画像数が足りない場合の超解像再 構成法を提案する．それは，超解像再構成の前処理として予備補間処理法を導入することで ある．予備補間処理法とは，不完全な観測画像列を連続空間上に再配置した後に，観測過 程のPSF（点像分布関数：Point spread function）を考慮した2 次元不等間隔補間で画素数を 増やす一種の正則化により，観測画像の全画素数が必要数の半分以下であっても超解像を 可能にする方法である．この予備補間処理法を既存の反復逆投影法（ IBP 法： Iterative backward projection）に適用した改良IBP 法を提案する．IBP 法は，X 線CT 画像の再構成法 として実績があり，アルゴリズムが簡単で逆問題解法として収束性の良い方法として知られて いる． また， IBP 法の収束条件と正則化パラメータである逆投影カーネル（ BPK： Back-projection kernel）との関係についても詳細に検討し，予備補間処理の誤差に起因する 発散を抑制し，強制的に収束条件を満足させるWiener 型BPKを提案した．また，その設計法 についても新たに示した． 上記の方法を用いて，サンプル画像を用いたシミュレーションと実験室内で取得した可視お よび熱赤外画像を用いた実験により，予備補間処理法の効果とWiener 型BPK の有効性を確 認した．その結果，必要とする観測画像列の半数程度においても全て揃った場合と同等な超 解像性能を達成した． 同様に非線形逆問題解法である階層型ニューラルネットワーク法の，不完全な観測画像列 における超解像再構成への適用可能性も検討した．階層型ニューラルネットワークはぼけ画 像の復元やパターン認識などに広く使用されているが，超解像再構成に利用された例はそれ ほど多くない．本論文では，入力ベクトルと出力ベクトルの要素数に対してPSF の拡がりを考慮した，ベクターマッピング・ニューラルネットワーク法（VMNN 法：Vector mapping neural network）にすることで，不完全な観測画像列を用いた超解像再構成を可能にした． サンプル画像を用いたシミュレーションと実験室内で取得した可視および熱赤外画像を用 いた実験の結果，必要とする観測画像列の半分以上においては，VMNN 法の超解像性能は 改良IBP 法には及ばないが，観測画像の枚数が1/4 程度になると改良IBP 法よりも良い超解 像性能を示すことが明らかになった．学位記番号：工博甲37