493 research outputs found

    No-reference Stereoscopic Image Quality Assessment Using Natural Scene Statistics

    Get PDF
    We present two contributions in this work: (i) a bivariate generalized Gaussian distribution (BGGD) model for the joint distribution of luminance and disparity subband coefficients of natural stereoscopic scenes and (ii) a no-reference (NR) stereo image quality assessment algorithm based on the BGGD model. We first empirically show that a BGGD accurately models the joint distribution of luminance and disparity subband coefficients. We then show that the model parameters form good discriminatory features for NR quality assessment. Additionally, we rely on the previously established result that luminance and disparity subband coefficients of natural stereo scenes are correlated, and show that correlation also forms a good feature for NR quality assessment. These features are computed for both the left and right luminance-disparity pairs in the stereo image and consolidated into one feature vector per stereo pair. This feature set and the stereo pair׳s difference mean opinion score (DMOS) (labels) are used for supervised learning with a support vector machine (SVM). Support vector regression is used to estimate the perceptual quality of a test stereo image pair. The performance of the algorithm is evaluated over popular databases and shown to be competitive with the state-of-the-art no-reference quality assessment algorithms. Further, the strength of the proposed algorithm is demonstrated by its consistently good performance over both symmetric and asymmetric distortion types. Our algorithm is called Stereo QUality Evaluator (StereoQUE)

    Binocular Rivalry Oriented Predictive Auto-Encoding Network for Blind Stereoscopic Image Quality Measurement

    Full text link
    Stereoscopic image quality measurement (SIQM) has become increasingly important for guiding stereo image processing and commutation systems due to the widespread usage of 3D contents. Compared with conventional methods which are relied on hand-crafted features, deep learning oriented measurements have achieved remarkable performance in recent years. However, most existing deep SIQM evaluators are not specifically built for stereoscopic contents and consider little prior domain knowledge of the 3D human visual system (HVS) in network design. In this paper, we develop a Predictive Auto-encoDing Network (PAD-Net) for blind/No-Reference stereoscopic image quality measurement. In the first stage, inspired by the predictive coding theory that the cognition system tries to match bottom-up visual signal with top-down predictions, we adopt the encoder-decoder architecture to reconstruct the distorted inputs. Besides, motivated by the binocular rivalry phenomenon, we leverage the likelihood and prior maps generated from the predictive coding process in the Siamese framework for assisting SIQM. In the second stage, quality regression network is applied to the fusion image for acquiring the perceptual quality prediction. The performance of PAD-Net has been extensively evaluated on three benchmark databases and the superiority has been well validated on both symmetrically and asymmetrically distorted stereoscopic images under various distortion types

    Blind assessment for stereo images considering binocular characteristics and deep perception map based on deep belief network

    Get PDF
    © 2018 Elsevier Inc. In recent years, blind image quality assessment in the field of 2D image/video has gained the popularity, but its applications in 3D image/video are to be generalized. In this paper, we propose an effective blind metric evaluating stereo images via deep belief network (DBN). This method is based on wavelet transform with both 2D features from monocular images respectively as image content description and 3D features from a novel depth perception map (DPM) as depth perception description. In particular, the DPM is introduced to quantify longitudinal depth information to align with human stereo visual perception. More specifically, the 2D features are local histogram of oriented gradient (HoG) features from high frequency wavelet coefficients and global statistical features including magnitude, variance and entropy. Meanwhile, the global statistical features from the DPM are characterized as 3D features. Subsequently, considering binocular characteristics, an effective binocular weight model based on multiscale energy estimation of the left and right images is adopted to obtain the content quality. In the training and testing stages, three DBN models for the three types features separately are used to get the final score. Experimental results demonstrate that the proposed stereo image quality evaluation model has high superiority over existing methods and achieve higher consistency with subjective quality assessments

    A blind stereoscopic image quality evaluator with segmented stacked autoencoders considering the whole visual perception route

    Get PDF
    Most of the current blind stereoscopic image quality assessment (SIQA) algorithms cannot show reliable accuracy. One reason is that they do not have the deep architectures and the other reason is that they are designed on the relatively weak biological basis, compared with findings on human visual system (HVS). In this paper, we propose a Deep Edge and COlor Signal INtegrity Evaluator (DECOSINE) based on the whole visual perception route from eyes to the frontal lobe, and especially focus on edge and color signal processing in retinal ganglion cells (RGC) and lateral geniculate nucleus (LGN). Furthermore, to model the complex and deep structure of the visual cortex, Segmented Stacked Auto-encoder (S-SAE) is used, which has not utilized for SIQA before. The utilization of the S-SAE complements weakness of deep learning-based SIQA metrics that require a very long training time. Experiments are conducted on popular SIQA databases, and the superiority of DECOSINE in terms of prediction accuracy and monotonicity is proved. The experimental results show that our model about the whole visual perception route and utilization of S-SAE are effective for SIQA

    Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery

    Get PDF
    One of the main challenges for computer-assisted surgery (CAS) is to determine the intra-opera- tive morphology and motion of soft-tissues. This information is prerequisite to the registration of multi-modal patient-specific data for enhancing the surgeon’s navigation capabilites by observ- ing beyond exposed tissue surfaces and for providing intelligent control of robotic-assisted in- struments. In minimally invasive surgery (MIS), optical techniques are an increasingly attractive approach for in vivo 3D reconstruction of the soft-tissue surface geometry. This paper reviews the state-of-the-art methods for optical intra-operative 3D reconstruction in laparoscopic surgery and discusses the technical challenges and future perspectives towards clinical translation. With the recent paradigm shift of surgical practice towards MIS and new developments in 3D opti- cal imaging, this is a timely discussion about technologies that could facilitate complex CAS procedures in dynamic and deformable anatomical regions

    Velocimetry-based pressure information for spray analysis – novel experimental, processing and evaluation strategies

    Get PDF
    In der vorliegenden Arbeit wurde der Spraytransport von komplexen Benzindirekteinspritzungssprays (GDI) mittels auf Geschwindigkeitmessung basierter Druckauswertung untersucht. Für diesen Zweck wurden neue Versuchs-, Verarbeitungs- und Auswertestrategien eingeführt, um eine Druckauswertung der Spray-induzierten Strömung zu befähigen und deren Möglichkeiten auszuweiten. Dies umfasst unter anderem ein statistisches Verfahren auf Basis der Unsteady Reynolds-Averaged Navier-Stokes (URANS) Gleichungen und Ensemble-Mittelung, welche die Druckauswertung transienter, statistisch stationärer Strömungen mittels konventioneller Particle Image Velocimetry (PIV) ermöglicht. Darüber hinaus wurde eine neuartige Technik namens Dual-Plane-Stereo-Astigmatismus (DPSA) entwickelt, die die Auswertung momentaner Druckfelder und damit die Analyse einzelner Einspritzereignisse unter Verwendung eines stereoskopischen Aufbaus und einer einzigen Lichtquelle ermöglicht. Abschließend wurde die Methode der Physics-Informed Neural Networks (PINNs) erfolgreich aus dem Bereich des Deep Learnings in die experimentelle Strömungsmechanik und Spray-Analyse übertragen. Das PINN-Verfahren weitet die Möglichkeiten der bisherigen auf Geschwindigkeitsmessung basierenden Druckauswertung aus und ermöglicht die Auswertung von bislang nicht auswertbaren Strömungsbereichen, sowohl in Raum und Zeit. Unter Verwendung der beschriebenen Methoden wurde die Wechselwirkung zwischen Spray und Umgebungsgasströmung für unterschiedliche Betriebsbedingungen und Sprayauslegungen untersucht. Es zeigte sich, dass der Impulsaustausch mit höherem Einspritzdruck, Gasdichte, Kraftstofftemperatur, größerer Relativgeschwindigkeit, Spray-Gas-Grenzfläche, Sprayexpansion und stärkerer Zerstäubung bzw. Flash-Boiling zunimmt. Als eine wesentliche Erkenntnis wurde festgestellt, dass die Ablenkung von Sprays bzw. das Phänomen der Strahl-zu-Strahl-Wechselwirkung und Spraykontraktion auf einen Nettoimpuls zurückzuführen ist, der auf einzelne Spraykeulen infolge von induzierten Druckkräften wirkt. In diesem Zusammenhang wurde das Vorhandensein eines Niederdruckgebiets im Zentrum von Mehrlochsprays experimentell bestätigt. Es wurde aufgezeigt, dass das Ausmaß der Strahl-zu-Strahl-Wechselwirkung und der Spraykontraktion durch eine enge Spritzlochanordnung und -ausrichtung, eine starke Zerstäubung und ein erhöhtes Tropfen-Folgeverhalten begünstigt wird

    Stereoscopic depth increases intersubject correlations of brain networks

    Get PDF
    Three-dimensionalmovies presented via stereoscopic displays have becomemore popular in recent years aiming at a more engaging viewing experience. However, neurocognitive processes associated with the perception of stereoscopic depth in complex and dynamic visual stimuli remain understudied. Here, we investigate the influence of stereoscopic depth on both neurophysiology and subjective experience. Using multivariate statistical learning methods, we compare the brain activity of subjects when freely watching the same movies in 2D and in 3D. Subjective reports indicate that 3D movies are more strongly experienced than 2D movies. On the neural level, we observe significantly higher intersubject correlations of cortical networks when subjects are watching 3D movies relative to the same movies in 2D. We demonstrate that increases in intersubject correlations of brain networks can serve as neurophysiologicalmarker for stereoscopic depth and for the strength of the viewing experience

    Production and Assessment of Usefulness of Interactive 2-D and Stereoscopic 3-D Videos as Tools for Anatomic Dissection Preparation and Examination Review

    Get PDF
    Laboratory is an integral part of a gross anatomy course in which students have their first in–depth dissection experience and explore structure-function relationships. Students arrive in the course that requires acquisition of a large vocabulary and visual imagery with scant prior knowledge. Even with extensive preparation on their part, the task is so difficult that students rely heavily on help from peers, teaching assistants, and instructors to gain the best from laboratory time. In recognition of the complexity of the learning task and the limitation on the amount of help available, this research was conducted to explore the value of educational tools that could enhance learning, make time in the laboratory more profitable, and decrease dependency on peers, teaching assistants, and instructors. Because anatomy is a highly visually based discipline, it was reasoned that interactive high definition videos with verbal descriptions of dissections would enhance the learning process. High definition videos of dissections were produced in 2–D and stereoscopic 3–D formats and compared with the standard dissection guide as tools for laboratory preparation. Stereoscopic 3–D format was included because of the hypothesis that the depth it provides might help students more readily grasp the relationships of structures to each other. Timing, duration, and tools provided to interact with the various formats varied with the experiment. The videos consisted of short presentations (10–14 minutes) of dissection steps or reviews of relationships of structures and were self–paced so they could be viewed more than once. Questions to encourage interaction with the materials were integrated into the videos and supplied with the Guide. Depending on the experiment, data collected included performance on paper and practical examinations, dissection quality, and frequency of requests for help in addition to surveys designed to assess ease of use and acceptance of the various presentation modes. Results presented in the thesis indicate that videos were superior to the guide in helping students prepare for dissection and develop understanding of the assigned body structures and their relationships. With the reservation that mode of 3–D delivery may play a role, 2–D videos were usually rated more positively than 3–D videos in student opinions. Both types of videos improved performance on various assessments and received more positive feedback when compared to the laboratory manual. This research confirmed the basic hypothesis that videos are effective tools for use in anatomy education and that they are worthy of significant investment of resources to help overcome some of the challenges facing anatomy educators
    corecore