No-Reference Quality Assessment for 360-degree Images by Analysis of Multi-frequency Information and Local-global Naturalness

360-degree/omnidirectional images (OIs) have achieved remarkable attentions due to the increasing applications of virtual reality (VR). Compared to conventional 2D images, OIs can provide more immersive experience to consumers, benefitting from the higher resolution and plentiful field of views (FoVs). Moreover, observing OIs is usually in the head mounted display (HMD) without references. Therefore, an efficient blind quality assessment method, which is specifically designed for 360-degree images, is urgently desired. In this paper, motivated by the characteristics of the human visual system (HVS) and the viewing process of VR visual contents, we propose a novel and effective no-reference omnidirectional image quality assessment (NR OIQA) algorithm by Multi-Frequency Information and Local-Global Naturalness (MFILGN). Specifically, inspired by the frequency-dependent property of visual cortex, we first decompose the projected equirectangular projection (ERP) maps into wavelet subbands. Then, the entropy intensities of low and high frequency subbands are exploited to measure the multi-frequency information of OIs. Besides, except for considering the global naturalness of ERP maps, owing to the browsed FoVs, we extract the natural scene statistics features from each viewport image as the measure of local naturalness. With the proposed multi-frequency information measurement and local-global naturalness measurement, we utilize support vector regression as the final image quality regressor to train the quality evaluation model from visual quality-related features to human ratings. To our knowledge, the proposed model is the first no-reference quality assessment method for 360-degreee images that combines multi-frequency information and image naturalness. Experimental results on two publicly available OIQA databases demonstrate that our proposed MFILGN outperforms state-of-the-art approaches

Learning-based Satisfied User Ratio Prediction for Symmetrically and Asymmetrically Compressed Stereoscopic Images

The file attached to this record is the author's final peer reviewed version.The Satisfied User Ratio (SUR) for a given distortion level is the fraction of subjects that cannot perceive a quality difference between the original image and its compressed version. By predicting the SUR, one can determine the highest distortion level which allows to save bit rate while guaranteeing a good visual quality. We propose the first method to predict the SUR for symmetrically and asymmetrically compressed stereoscopic images. Unlike SUR prediction techniques for 2D images and videos, our method exploits the properties of binocular vision. We first extract features that characterize image quality and image content. Then, we use gradient boosting decision trees to reduce the number of features and train a regression model that learns a mapping function from the features to the SUR values. Experimental results on the SIAT-JSSI and SIAT-JASI datasets show high SUR prediction accuracy for H.265 All-Intra and JPEG2000 symmetrically and asymmetrically compressed stereoscopic images

Vision-based pavement marking detection and condition assessment : a case study

Pavement markings constitute an effective way of conveying regulations and guidance to drivers. They constitute the most fundamental way to communicate with road users, thus, greatly contributing to ensuring safety and order on roads. However, due to the increasingly extensive traffic demand, pavement markings are subject to a series of deterioration issues (e.g., wear and tear). Markings in poor condition typically manifest as being blurred or even missing in certain places. The need for proper maintenance strategies on roadway markings, such as repainting, can only be determined based on a comprehensive understanding of their as-is worn condition. Given the fact that an efficient, automated and accurate approach to collect such condition information is lacking in practice, this study proposes a vision-based framework for pavement marking detection and condition assessment. A hybrid feature detector and a threshold-based method were used for line marking identification and classification. For each identified line marking, its worn/blurred severity level was then quantified in terms of worn percentage at a pixel level. The damage estimation results were compared to manual measurements for evaluation, indicating that the proposed method is capable of providing indicative knowledge about the as-is condition of pavement markings. This paper demonstrates the promising potential of computer vision in the infrastructure sector, in terms of implementing a wider range of managerial operations for roadway management

Understanding perceived quality through visual representations

The formatting of images can be considered as an optimization problem, whose cost function is a quality assessment algorithm. There is a trade-off between bit budget per pixel and quality. To maximize the quality and minimize the bit budget, we need to measure the perceived quality. In this thesis, we focus on understanding perceived quality through visual representations that are based on visual system characteristics and color perception mechanisms. Specifically, we use the contrast sensitivity mechanisms in retinal ganglion cells and the suppression mechanisms in cortical neurons. We utilize color difference equations and color name distances to mimic pixel-wise color perception and a bio-inspired model to formulate center surround effects. Based on these formulations, we introduce two novel image quality estimators PerSIM and CSV, and a new image quality-assistance method BLeSS. We combine our findings from visual system and color perception with data-driven methods to generate visual representations and measure their quality. The majority of existing data-driven methods require subjective scores or degraded images. In contrast, we follow an unsupervised approach that only utilizes generic images. We introduce a novel unsupervised image quality estimator UNIQUE, and extend it with multiple models and layers to obtain MS-UNIQUE and DMS-UNIQUE. In addition to introducing quality estimators, we analyze the role of spatial pooling and boosting in image quality assessment.Ph.D

Unified no-reference quality assessment of singly and multiply distorted stereoscopic images

A challenging problem in the no-reference quality assessment of multiply distorted stereoscopic images (MDSIs) is to simulate the monocular and binocular visual properties under a mixed type of distortions. Due to the joint effects of multiple distortions in MDSIs, the underlying monocular and binocular visual mechanisms have different manifestations with those of singly distorted stereoscopic images (SDSIs). This paper presents a unified no-reference quality evaluator for SDSIs and MDSIs by learning monocular and binocular local visual primitives (MB-LVPs). The main idea is to learn MB-LVPs to characterize the local receptive field properties of the visual cortex in response to SDSIs and MDSIs. Furthermore, we also consider that the learning of primitives should be performed in a task-driven manner. For this, two penalty terms including reconstruction error and quality inconsistency are jointly minimized within a supervised dictionary learning framework, generating a set of quality-oriented MB-LVPs for each single and multiple distortion modality. Given an input stereoscopic image, feature encoding is performed using the learned MB-LVPs as codebooks, resulting in the corresponding monocular and binocular responses. Finally, responses across all the modalities are fused with probabilistic weights which are determined by the modality-specific sparse reconstruction errors, yielding the final monocular and binocular features for quality regression. The superiority of our method has been verified on several SDSI and MDSI databases

Ra: The Sun for Science and Humanity

To guide the development of the Ra Strategic Framework, we defined scientific and applications objectives. For our primary areas of scientific interest, we choose the corona, the solar wind, the Sun's effect on the Earth, and solar theory and model development. For secondary areas of scientific interest, we selected sunspots, the solar constant, the Sun's gravitational field, helioseismology and the galactic cosmic rays. We stress the importance of stereoscopic imaging, observations at high spatial, spectral, and temporal resolutions, as well as of long duration measurements. Further exploration of the Sun's polar regions is also important, as shown already by the Ulysses mission. From an applications perspective, we adopted three broad objectives that would derive complementary inputs for the Strategic Framework. These were to identify and investigate: possible application spin-offs from science missions, possible solar-terrestrial missions dedicated to a particular application, and possible future applications that require technology development. The Sun can be viewed as both a source of resources and of threats. Our principal applications focus was that of threat mitigation, by examining ways to improve solar threat monitoring and early warning systems. We compared these objectives to the mission objectives of past, current, and planned international solar missions. Past missions (1962-1980) seem to have been focused on improvement of scientific knowledge, using multiple instrument spacecraft. A ten year gap followed this period, during which the results from previous missions were analyzed and solar study programmes were prepared in international organizations. Current missions (1990-1996) focus on particular topics such as the corona, solar flares, and coronal mass ejections. In planned missions, Sun/Earth interactions and environmental effects of solar activity are becoming more important. The corona is the centre of interest of almost all planned missions. It seems that no international long-term strategy has yet been adopted. For these plans the number of necessary future missions can be reduced and the onboard instrumentation can be optimized by performing a comparative analysis. The study of the corona must be done from different observing locations, orbits closer to the Sun, and by different means. The Cluster mission replacement is in progress; however, if the replacement is not implemented, the ISTP programme will fade after 1998. Furthermore, the physics of the Sun's interior should be emphasized more in the Mid- and Far-Term programmes. Finally, more emphasis should be placed on monitoring space weather and forecasting Sun/Earth interactions

Applications of Tethers in Space: Workshop Proceedings, Volume 1

The complete documentation of the workshop including all addresses, panel reports, charts, and summaries are presented. This volume presents all the reports on the fundamentals of applications of tethers in space. These applications include electrodynamic interactions, transportation, gravity utilization, constellations, technology and test, and science applications

Remote Sensing and the Earth

A text book on remote sensing, as part of the earth resources Skylab programs, is presented. The fundamentals of remote sensing and its application to agriculture, land use, geology, water and marine resources, and environmental monitoring are summarized

Analytical Ultrasonics in Materials Research and Testing

Research results in analytical ultrasonics for characterizing structural materials from metals and ceramics to composites are presented. General topics covered by the conference included: status and advances in analytical ultrasonics for characterizing material microstructures and mechanical properties; status and prospects for ultrasonic measurements of microdamage, degradation, and underlying morphological factors; status and problems in precision measurements of frequency-dependent velocity and attenuation for materials analysis; procedures and requirements for automated, digital signal acquisition, processing, analysis, and interpretation; incentives for analytical ultrasonics in materials research and materials processing, testing, and inspection; and examples of progress in ultrasonics for interrelating microstructure, mechanical properites, and dynamic response