Saliency in Augmented Reality

With the rapid development of multimedia technology, Augmented Reality (AR) has become a promising next-generation mobile platform. The primary theory underlying AR is human visual confusion, which allows users to perceive the real-world scenes and augmented contents (virtual-world scenes) simultaneously by superimposing them together. To achieve good Quality of Experience (QoE), it is important to understand the interaction between two scenarios, and harmoniously display AR contents. However, studies on how this superimposition will influence the human visual attention are lacking. Therefore, in this paper, we mainly analyze the interaction effect between background (BG) scenes and AR contents, and study the saliency prediction problem in AR. Specifically, we first construct a Saliency in AR Dataset (SARD), which contains 450 BG images, 450 AR images, as well as 1350 superimposed images generated by superimposing BG and AR images in pair with three mixing levels. A large-scale eye-tracking experiment among 60 subjects is conducted to collect eye movement data. To better predict the saliency in AR, we propose a vector quantized saliency prediction method and generalize it for AR saliency prediction. For comparison, three benchmark methods are proposed and evaluated together with our proposed method on our SARD. Experimental results demonstrate the superiority of our proposed method on both of the common saliency prediction problem and the AR saliency prediction problem over benchmark methods. Our data collection methodology, dataset, benchmark methods, and proposed saliency models will be publicly available to facilitate future research

Perceptual Quality Assessment of Omnidirectional Audio-visual Signals

Omnidirectional videos (ODVs) play an increasingly important role in the application fields of medical, education, advertising, tourism, etc. Assessing the quality of ODVs is significant for service-providers to improve the user's Quality of Experience (QoE). However, most existing quality assessment studies for ODVs only focus on the visual distortions of videos, while ignoring that the overall QoE also depends on the accompanying audio signals. In this paper, we first establish a large-scale audio-visual quality assessment dataset for omnidirectional videos, which includes 375 distorted omnidirectional audio-visual (A/V) sequences generated from 15 high-quality pristine omnidirectional A/V contents, and the corresponding perceptual audio-visual quality scores. Then, we design three baseline methods for full-reference omnidirectional audio-visual quality assessment (OAVQA), which combine existing state-of-the-art single-mode audio and video QA models via multimodal fusion strategies. We validate the effectiveness of the A/V multimodal fusion method for OAVQA on our dataset, which provides a new benchmark for omnidirectional QoE evaluation. Our dataset is available at https://github.com/iamazxl/OAVQA.Comment: 12 pages, 5 figures, to be published in CICAI202

Spatial-temporal heterogeneity and built environment nonlinearity in inconsiderate parking of dockless bike-sharing

Although previous studies have shed light on the travel behaviour of dockless bike-sharing (DBS) users, little research focused on their inconsiderate parking behavior. Unlike the travelling behavior, the choice of parking location is closely linked to the different built environments surrounding the parking locations. Therefore, to improve the efficiency of governance, it is vital to explore the parking patterns and heterogeneous influences of the built environment on inconsiderate parking and formulate targeted measures. This paper measures the coordinates of prohibited parking areas in the field to identify inconsiderate parking. Based on big data from Mobike DBS and data on the built environment, the paper empirically analyzes the heterogeneous spatiotemporal distribution patterns of inconsiderate parking with clustering and decision trees. The influencing factors of inconsiderate parking and their nonlinear effects are further analyzed using random forest and partial dependence plots (PDP). The results show that there is significant spatiotemporal heterogeneity in inconsiderate parking, in which different clusters reflect various characteristics of the built environment. Furthermore, marginal effect analysis finds that influencing factors such as riding distance, catering service places, lifestyle services, sports and leisure places, and hotels and hostels have a strong effect on inconsiderate parking behavior, and show nonlinear effects with optimal allocation intervals. Therefore, targeted strategies should be carried out in terms of dynamic temporal adjustment, precise spatial layout, differential management according to time and zone, and cause-assisted administration. The paper's results provide important decision-making support for inconsiderate parking.</p

A thermoviscoelastic finite deformation constitutive model based on dual relaxation mechanisms for amorphous shape memory polymers

ABSTRACTThis paper proposes a new thermoviscoelastic finite deformation model incorporating dual relaxation mechanisms to predict the complete thermo-mechanical response of amorphous shape memory polymers. The model is underpinned by the detailed microscopic molecular mechanism and effectively reflects the current understanding of the glass transition phenomenon. Novel evolution rules are obtained from the model to characterize the viscous flow, and a new theory named an internal stress model is introduced and combined with the dual relaxation mechanisms to capture the stress recovery. The rationality of the constitutive model is verified as the theoretical results agree well with the experimental data. Moreover, the constitutive model is further simplified to facilitate engineering applications, and it can roughly capture the characteristics of shape memory polymers

Relation-Attention Networks for Remote Sensing Scene Classification

Remote sensing (RS) scene classification plays an important role in a wide range of RS applications. Recently, convolutional neural networks (CNNs) have been applied to the field of scene classification in RS images and achieved impressive performance. However, to classify RS scenes, most of the existing CNN methods either utilize the high-level features from the last convolutional layer of CNNs, missing much important information existing at the other levels, or directly fuse the features at different levels, bringing redundant and/or mutually exclusive information. Inspired by the attention mechanism of the human visual system, in this article, we explore a novel relation-attention model and design an end-to-end relation-attention network (RANet) to learn powerful feature representations of multiple levels to further improve the classification performance. First, we propose to extract convolutional features at different levels by pretrained CNNs. Second, a multiscale feature computation module is constructed to connect features at different levels and generate multiscale semantic features. Third, a novel relation-attention model is designed to focus on the critical features whilst avoiding the use of redundant and even distractive ones by exploiting the scale contextual information. Finally, the resulting relation-attention features are concatenated and fed into a softmax layer for the final classification. Experiments on four well-known RS scene classification datasets (UC-Merced, WHU-RS19, AID, and OPTIMAL-31) show that our method outperforms some state-of-the-art algorithms

Multi-level Feature Fusion Networks with Adaptive Channel Dimensionality Reduction for Remote Sensing Scene Classification

Scene classification in very high-resolution (VHR) remote sensing (RS) images is a challenging task due to the complex and diverse content of the images. Recently, convolution neural networks (CNNs) have been utilized to tackle this task. However, CNNs cannot fully meet the needs of scene classification due to clutters and small objects in VHR images. To handle these challenges, this letter presents a novel multilevel feature fusion (MLFF) network with adaptive channel dimensionality reduction for RS scene classification. Specifically, an adaptive method is designed for channel dimensionality reduction of high-dimensional features. Then, an MLFF module is introduced to fuse the features in an efficient way. Experiments on three widely used data sets show that our model outperforms several state-of-the-art methods in terms of both accuracy and stability

Chensinin-1b Alleviates DSS-Induced Inflammatory Bowel Disease by Inducing Macrophage Switching from the M1 to the M2 Phenotype

Inflammatory bowel disease (IBD) is a chronic relapsing inflammatory disorder with an increasing prevalence worldwide. Macrophage polarization is involved in the pathogenesis of IBD. Repolarization of macrophage has thus emerged as a novel therapeutic approach for managing IBD. Chensinin-1b, derived from the skin of Rana chensinensis, is a derivative of a native antimicrobial peptide (AMP). It shows anti-inflammatory effects in sepsis models and can potentially modulate macrophage polarization. The objective of this research was to study the role of chensinin-1b in macrophage polarization and dextran sulfate sodium (DSS)-induced colitis. RAW264.7 macrophages were polarized to the M1 phenotype using lipopolysaccharide (LPS) and simultaneously administered chensinin-1b at various concentrations. The ability of chenisnin-1b to reorient macrophage polarization was assessed by ELISA, qRT-PCR, and flow cytometry analysis. The addition of chensinin-1b significantly restrained the expression of M1-associated proinflammatory cytokines and surface markers, including TNF-α, IL-6, NO, and CD86, and exaggerated the expression of M2-associated anti-inflammatory cytokines and surface markers, including IL-10, TGF-β1, Arg-1, Fizz1, Chil3, and CD206. Mechanistically, via Western Blotting, we revealed that chensinin-1b induces macrophage polarization from the M1 to the M2 phenotype by inhibiting the phosphorylation of nuclear factor-kappa B (NF-κB) and mitogen-activated protein kinase (MAPK). In mouse models of colitis, intraperitoneal administration of chensinin-1b alleviated symptoms induced by DSS, including weight loss, elevated disease activity index (DAI) scores, colon shortening, colonic tissue damage, and splenomegaly. Consistent with our in vitro data, chensinin-1b induced significant decreases in the expression of M1 phenotype biomarkers and increases in the expression of M2 phenotype biomarkers in the mouse colitis model. Furthermore, chensinin-1b treatment repressesed NF-κB phosphorylation in vivo. Overall, our data showed that chensinin-1b attenuates IBD by repolarizing macrophages from the M1 to the M2 phenotype, suggesting its potential as a therapeutic candidate for IBD

Confusing Image Quality Assessment: Toward Better Augmented Reality Experience

International audienceWith the development of multimedia technology, Augmented Reality (AR) has become a promising next-generation mobile platform. The primary value of AR is to promote the fusion of digital contents and real-world environments, however, studies on how this fusion will influence the Quality of Experience (QoE) of these two components are lacking. To achieve better QoE of AR, whose two layers are influenced by each other, it is important to evaluate its perceptual quality first. In this paper, we consider AR technology as the superimposition of virtual scenes and real scenes, and introduce visual confusion as its basic theory. A more general problem is first proposed, which is evaluating the perceptual quality of superimposed images, i.e., confusing image quality assessment. A ConFusing Image Quality Assessment (CFIQA) database is established, which includes 600 reference images and 300 distorted images generated by mixing reference images in pairs. Then a subjective quality perception experiment is conducted towards attaining a better understanding of how humans perceive the confusing images. Based on the CFIQA database, several benchmark models and a specifically designed CFIQA model are proposed for solving this problem. Experimental results show that the proposed CFIQA model achieves state-of-the-art performance compared to other benchmark models. Moreover, an extended ARIQA study is further conducted based on the CFIQA study. We establish an ARIQA database to better simulate the real AR application scenarios, which contains 20 AR reference images, 20 background (BG) reference images, and 560 distorted images generated from AR and BG references, as well as the correspondingly collected subjective quality ratings. Three types of full-reference (FR) IQA benchmark variants are designed to study whether we should consider the visual confusion when designing corresponding IQA algorithms. An ARIQA metric is finally proposed for better evaluating the perceptual quality of AR images. Experimental results demonstrate the good generalization ability of the CFIQA model and the state-of-the-art performance of the ARIQA model. The databases, benchmark models, and proposed metrics are available at: https://github.com/DuanHuiyu/ARIQA

A dataset of eye movements for the children with autism spectrum disorder

International audienc