New metric products, movies and 3D models from old stereopairs and their application to the in situ palaeontological site of Ambrona

[ES] Este artículo está basado en la información del siguiente proyecto:● LDGP_mem_006-1: "[S_Ambrona_Insitu] Levantamiento fotogramétrico del yacimiento paleontológico “Museo in situ” de Ambrona (Soria)", http://hdl.handle.net/10810/7353● LDGP_mem_006-1: "[S_Ambrona_Insitu] Levantamiento fotogramétrico del yacimiento paleontológico “Museo in situ” de Ambrona (Soria)", http://hdl.handle.net/10810/7353[EN] This paper is based on the information gathered in the following project:[EN] 3D modelling tools from photographic pictures have experienced significant improvements in the last years. One of the most outstanding changes is the spread of the photogrammetric systems based on algorithms referred to as Structure from Motion (SfM) in contrast with the traditional stereoscopic pairs. Nevertheless, the availability of important collections of stereoscopic registers collected during past decades invites us to explore the possibilities for re-using these photographs in order to generate new multimedia products, especially due to the fact that many of the documented elements have been largely altered or even disappeared. This article analyses an example of application to the re-use of a collection of photographs from the palaeontological site of Ambrona (Soria, Spain). More specifically, different pieces of software based on Structure from Motion (SfM) algorithms for the generation of 3D models with photographic textures are tested and some derived products such as orthoimages, video or applications of Augmented Reality (AR) are presented.[ES] Las herramientas de modelado 3D a partir de imágenes fotográficas han experimentado avances muy significativos en los últimos años. Uno de los más destacados corresponde a la generalización de los sistemas fotogramétricos basados en los algoritmos denominados Structure from Motion (SfM) sobre los proyectos de documentación tradicional basados en pares estereoscópicos. La existencia de importantes colecciones de registros estereoscópicos realizados durante las décadas anteriores invita a explorar las posibilidades de reutilización de estos registros para la obtención de productos multimedia actuales, máxime cuando algunos de los elementos documentados han sufrido grandes modificaciones o incluso desaparecido. En el presente artículo se analiza la reutilización de colecciones fotográficas de yacimientos paleontológicos mediante un ejemplo centrado en el yacimiento de Ambrona (Soria, España). En concreto, se contrastan varios programas basados en los algoritmos denominados Structure from Motion (SfM) para la generación del modelo 3D con textura y otros productos derivados como ortoimágenes, vídeos o aplicaciones de Realidad Aumentada (RA)

A blind stereoscopic image quality evaluator with segmented stacked autoencoders considering the whole visual perception route

Most of the current blind stereoscopic image quality assessment (SIQA) algorithms cannot show reliable accuracy. One reason is that they do not have the deep architectures and the other reason is that they are designed on the relatively weak biological basis, compared with findings on human visual system (HVS). In this paper, we propose a Deep Edge and COlor Signal INtegrity Evaluator (DECOSINE) based on the whole visual perception route from eyes to the frontal lobe, and especially focus on edge and color signal processing in retinal ganglion cells (RGC) and lateral geniculate nucleus (LGN). Furthermore, to model the complex and deep structure of the visual cortex, Segmented Stacked Auto-encoder (S-SAE) is used, which has not utilized for SIQA before. The utilization of the S-SAE complements weakness of deep learning-based SIQA metrics that require a very long training time. Experiments are conducted on popular SIQA databases, and the superiority of DECOSINE in terms of prediction accuracy and monotonicity is proved. The experimental results show that our model about the whole visual perception route and utilization of S-SAE are effective for SIQA

Sparse representation based stereoscopic image quality assessment accounting for perceptual cognitive process

In this paper, we propose a sparse representation based Reduced-Reference Image Quality Assessment (RR-IQA) index for stereoscopic images from the following two perspectives: 1) Human visual system (HVS) always tries to infer the meaningful information and reduces uncertainty from the visual stimuli, and the entropy of primitive (EoP) can well describe this visual cognitive progress when perceiving natural images. 2) Ocular dominance (also known as binocularity) which represents the interaction between two eyes is quantified by the sparse representation coefficients. Inspired by previous research, the perception and understanding of an image is considered as an active inference process determined by the level of “surprise”, which can be described by EoP. Therefore, the primitives learnt from natural images can be utilized to evaluate the visual information by computing entropy. Meanwhile, considering the binocularity in stereo image quality assessment, a feasible way is proposed to characterize this binocular process according to the sparse representation coefficients of each view. Experimental results on LIVE 3D image databases and MCL database further demonstrate that the proposed algorithm achieves high consistency with subjective evaluation

Quality assessment metric of stereo images considering cyclopean integration and visual saliency

In recent years, there has been great progress in the wider use of three-dimensional (3D) technologies. With increasing sources of 3D content, a useful tool is needed to evaluate the perceived quality of the 3D videos/images. This paper puts forward a framework to evaluate the quality of stereoscopic images contaminated by possible symmetric or asymmetric distortions. Human visual system (HVS) studies reveal that binocular combination models and visual saliency are the two key factors for the stereoscopic image quality assessment (SIQA) metric. Therefore inspired by such findings in HVS, this paper proposes a novel saliency map in SIQA metric for the cyclopean image called “cyclopean saliency”, which avoids complex calculations and produces good results in detecting saliency regions. Moreover, experimental results show that our metric significantly outperforms conventional 2D quality metrics and yields higher correlations with human subjective judgment than the state-of-art SIQA metrics. 3D saliency performance is also compared with “cyclopean saliency” in SIQA. It is noticed that the proposed metric is applicable to both symmetric and asymmetric distortions. It can thus be concluded that the proposed SIQA metric can provide an effective evaluation tool to assess stereoscopic image quality

Quality index for stereoscopic images by jointly evaluating cyclopean amplitude and cyclopean phase

With widespread applications of three-dimensional (3-D) technology, measuring quality of experience for 3-D multimedia content plays an increasingly important role. In this paper, we propose a full reference stereo image quality assessment (SIQA) framework which focuses on the innovation of binocular visual properties and applications of low-level features. On one hand, based on the fact that human visual system understands an image mainly according to its low-level features, local phase and local amplitude extracted from phase congruency measurement are employed as primary features. Considering the less prominent performance of amplitude in IQA, visual saliency is applied into the modification on amplitude. On the other hand, by fully considering binocular rivalry phenomena, we create the cyclopean amplitude map and cyclopean phase map. With this method, both image features and binocular visual properties are mutually combined with each other. Meanwhile, a novel binocular modulation function in spatial domain is also adopted into the overall quality prediction of amplitude and phase. Extensive experiments demonstrate that the proposed framework achieves higher consistency with subjective tests than relevant SIQA metrics

Blind assessment for stereo images considering binocular characteristics and deep perception map based on deep belief network

© 2018 Elsevier Inc. In recent years, blind image quality assessment in the field of 2D image/video has gained the popularity, but its applications in 3D image/video are to be generalized. In this paper, we propose an effective blind metric evaluating stereo images via deep belief network (DBN). This method is based on wavelet transform with both 2D features from monocular images respectively as image content description and 3D features from a novel depth perception map (DPM) as depth perception description. In particular, the DPM is introduced to quantify longitudinal depth information to align with human stereo visual perception. More specifically, the 2D features are local histogram of oriented gradient (HoG) features from high frequency wavelet coefficients and global statistical features including magnitude, variance and entropy. Meanwhile, the global statistical features from the DPM are characterized as 3D features. Subsequently, considering binocular characteristics, an effective binocular weight model based on multiscale energy estimation of the left and right images is adopted to obtain the content quality. In the training and testing stages, three DBN models for the three types features separately are used to get the final score. Experimental results demonstrate that the proposed stereo image quality evaluation model has high superiority over existing methods and achieve higher consistency with subjective quality assessments

Determining Conjugate Points of An Aerial Photograph Stereopairs Using Separate Channel Mean Value Technique

In  the  development  of  digital  photogrammetric  system,  automatic image matching process play an important role. The automatic image matching is  used  in  finding  the  conjugate  points  of  an  aerial  photograph  stereopair automatically.  This  matching  technique  gives  quite  significant  contribution especially  in  the  development  of  3D  photogrammetry  in  an  attempt  to  get  the exact and precise topographic information during the stereo restitution. There are two image matching methods that have been so far developed, i.e. the area based system  for  gray  level  environment  and  the  feature  based  system  for  natural feature  environment.  This  research  is  trying  to  implement  the  area  based matching  with  normalized  cross  correlation  technique  to  get  the  correlation coefficient between the spectral value of the left image and its pair on the right. Based  on  the  previous  researches,  the  use  of  color  image  could  increase  the quality  of  matching.  One  of  the  color  image  matching  technique  is  known  as Separate Channel Mean Value. In order to be able to see the performance of the technique, a number of sampling areas with various different characteristics have been  chosen,  i.e.  the  heterogeneous,  homogeneous,  texture,  shadow,   and contrast. The  result  shows  the  highest  similarity  measure  is  obtained  on  heterogeneous sample area at size of all reference and search image, i.e. (11 pixels x 11 pixels) and   (23  pixels  x  23  pixels).  In  these  area  the  correlation  coefficient  reached more than 0.7 and the highest percentage of similarity measure is obtained. The average of total similarity  measure of conjugate images in the sampling image area  only  reach  about  41.43  %  of  success.  Therefore,  this  technique  has  a weakness and some treatment to overcome the problems is still needed