TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering

Despite thousands of researchers, engineers, and artists actively working on improving text-to-image generation models, systems often fail to produce images that accurately align with the text inputs. We introduce TIFA (Text-to-Image Faithfulness evaluation with question Answering), an automatic evaluation metric that measures the faithfulness of a generated image to its text input via visual question answering (VQA). Specifically, given a text input, we automatically generate several question-answer pairs using a language model. We calculate image faithfulness by checking whether existing VQA models can answer these questions using the generated image. TIFA is a reference-free metric that allows for fine-grained and interpretable evaluations of generated images. TIFA also has better correlations with human judgments than existing metrics. Based on this approach, we introduce TIFA v1.0, a benchmark consisting of 4K diverse text inputs and 25K questions across 12 categories (object, counting, etc.). We present a comprehensive evaluation of existing text-to-image models using TIFA v1.0 and highlight the limitations and challenges of current models. For instance, we find that current text-to-image models, despite doing well on color and material, still struggle in counting, spatial relations, and composing multiple objects. We hope our benchmark will help carefully measure the research progress in text-to-image synthesis and provide valuable insights for further research

Image Spectral Resolution Enhancement for Mapping Native Plant Species in a Typical Area of the Three-River Headwaters Region, China

Large-scale multispectral remote sensing data are often unavailable for some practical applications. Spectral resolution enhancement for large-scale multispectral remote sensing images by incorporating small-scale hyperspectral remote sensing images is an alternative way to generate remote sensing images with both large spatial range and high spectral resolution. This paper proposes an improved spectral resolution enhancement method (ISREM) using spectral matrix and weighting the spectral angle of the transformation matrix. ISREM is tested in a typical area of the Three-River Headwaters region (TRHR) to produce a synthetic hyperspectral image (HSI). Two existing spectral resolution enhancement methods, the color resolution improvement software package (CRISP) and spectral resolution enhancement method (SREM), are adopted to compare with ISREM. To further test the practicality of the synthetic HSIs generated by the ISREM, CRISP and SREM, they are used to estimate the coverage of native plant species (NPS) using support vector machines (SVM) and random forest (RF) regressions. The experimental results are as follows. (1) For the Pearson correlation coefficient between the synthetic HSI and original image, ISREM yielded the largest value of 0.9582, followed by CRISP and SREM with values of 0.9480 and 0.9514. For spectral similarity, the HSI generated by the ISREM was the closest to the original reference HSI in the spectral curve. It also showed the best cumulative performance with the use of the three quality evaluation indexes. (2) The identification accuracies of native plant species were 93.51%, 90.91%, 89.61% and 89.61% using generated HSIs and original multispectral image (MSI) within a threshold of 20%, respectively. Compared with original MSI, the synthetic HSI showed better ability to identify NPS in the study area, which further illustrated the effectiveness of the ISREM. (3) The ISREM can reduce the strict requirement of pure pixels and maintain the quality of synthetic HSI by spectral angle weighting. Hence, the proposed ISREM outperforms the existing CRISP and SREM methods in image spectral resolution enhancement of multispectral remote sensing images

Assessment of macro, trace and toxic element intake from rice: differences between cultivars, pigmented and non-pigmented rice

Abstract Pigmented and non-pigmented rice varieties (grown in different areas) were collected in China, Yunnan, to investigate the content of macro-, trace elements and potentially toxic elements (PTEs), and to assess the health risk associated with dietary intake. The order of elemental concentrations in rice was Mn > Zn > Fe > Cu > Se for trace elements, P > K > Mg > Ca > Na for macro elements, and Cr > As > Cd for PTEs. Rice with a high concentration of essential elements also associated with a high content of PTEs. In addition, higher content of Cr, Mn and Na were found in pigmented rice. The health risk assessment showed that the daily intake of all elements was below the tolerable limit (UL). Moreover the intake of Fe, Zn and Se was far from sufficient for the nutrient requirement. The PTEs in rice dominated the health risk. Of concern is that this rice consumption is likely to contribute to carcinogenic risks in the long term and that adults are at higher health risk from pigmented rice compared to non-pigmented rice. This study confirms that the lack of essential micronutrients in rice and the health risk associated with rice diets should remain a concern

Shape Adaptive Neighborhood Information-Based Semi-Supervised Learning for Hyperspectral Image Classification

Hyperspectral image (HSI) classification is an important research topic in detailed analysis of the Earth’s surface. However, the performance of the classification is often hampered by the high-dimensionality features and limited training samples of the HSIs which has fostered research about semi-supervised learning (SSL). In this paper, we propose a shape adaptive neighborhood information (SANI) based SSL (SANI-SSL) method that takes full advantage of the adaptive spatial information to select valuable unlabeled samples in order to improve the classification ability. The improvement of the classification mainly relies on two aspects: (1) the improvement of the feature discriminability, which is accomplished by exploiting spectral-spatial information, and (2) the improvement of the training samples’ representativeness which is accomplished by exploiting the SANI for both labeled and unlabeled samples. First, the SANI of labeled samples is extracted, and the breaking ties (BT) method is used in order to select valuable unlabeled samples from the labeled samples’ neighborhood. Second, the SANI of unlabeled samples are also used to find more valuable samples, with the classifier combination method being used as a strategy to ensure confidence and the adaptive interval method used as a strategy to ensure informativeness. The experimental comparison results tested on three benchmark HSI datasets have demonstrated the significantly superior performance of our proposed method

Differential effects of climatic and non-climatic factors on the distribution of vegetation phenology trends on the Tibetan plateau

The study of vegetation phenology changes is important because it is a sensitive indicator of climate change, affecting the exchange of carbon, energy and water fluxes between the land and the atmosphere. Previous studies have focused on the effects of climatic factors among environmental factors on vegetation phenology, thus the effects of non-climatic factors among environmental factors have not been well quantified. This study endeavors to scrutinize the spatiotemporal inconsistency in the start-of-season (SOS) and the end-of-season (EOS) on the Tibetan Plateau (TP) and to quantify the effects of environmental factors on phenology. To this end, the Moderate-resolution Imaging Spectroradiomater (MODIS) Normalized Difference Vegetation Index (NDVI) data from 2001 to 2018 and four common used methods were employed to extract SOS and EOS, and the site data was used to select the most appropriate phenology results. The Geodetector model was used to assess and measure the explanatory power of different environmental factors. The research results indicate that temperature exerts a more substantial impact on phenology than precipitation on TP. non-climatic factors such as longitude, latitude, and elevation are more influential in determining the distribution of phenological trends than climatic factors. Among these non-climatic factors, latitude has the most prominent effect on the trends of SOS. Furthermore, non-climatic factors exhibit a stronger effect on SOS, whereas EOS is more susceptible to climatic factors and less influenced by non-climatic factors. These discoveries bear great significance in comprehending the intricate outcomes of regional changes on vegetation phenology and enhancing phenology models

Monitoring grassland degradation and restoration using a novel climate use efficiency (NCUE) index in the Tibetan Plateau, China

Grassland degradation is one of the most pressing challenges in natural environment and anthropogenic society. However, there is yet no effective approach for monitoring the spatio-temporal pattern of large-scale grassland degradation. In particular, the research on grassland changes in the harsh natural environment such as the Qinghai-Tibet Plateau is still in its infancy due to complexity, and it is extremely difficult for humans to reach these remote areas. The annual changes in the grassland biomass might be the results of climate fluctuations or grassland degradation. To test the hypothesis, the impact of inter-annual climate fluctuations needs to be considered when monitoring the grassland degradation based on spatio-temporal change of grassland biomass. In this paper, we propose a Novel Climate Use Efficiency index (NCUE) by considering rainfall, temperature, sunlight time, wind speed, surface temperature, accumulated temperature, time lag effect, light, temperature and water suitability and their coordination climatic factors that mainly affect vegetation growth comprehensively, to monitor grassland change suitable for cold and dry climate characteristics of the Qinghai-Tibet Plateau, and to reduce the effect of inter-annual variability of grassland productivity caused by climate fluctuation. As a consequence, grassland degradation monitoring could be more accurate and objective than existing ecological indicators. Our experiments show that the slope of NCUE over 31 years from 1982 to 2012 is 0.0028, showing a recovery trend in grassland. Degradation and restoration of grassland exist at the same time, and their proportions are 20.49% and 23.89%, respectively. By comparing with in-situ measurements in 2013 and 2009, 68% consistency was achieved with our prediction, and the 70% consistency is achieved by comparing with the positive and negative change trend of accumulated NDVI during the growing season. Moreover, the comparative analysis of land use/cover changes (LUCC) from 1990 to 2010 shows 69% of consistency. The ratio of the area of grassland significantly degradation and recovered predicted by NCUE change trend is 1.41% and 1.43%, respectively. It occupies a very small area of the study area. Yet, that predicted by NDVI change trend is 42.17% and 31.90%, respectively, and about 70% of the area is detected as drastic changes. It shows that NDVI is sensitive to climate fluctuations, while NCUE reduces the impact of climate fluctuations, reflecting change of grassland being affected by human activities and long-term climate change. The novel NCUE has great potential and utility to minify the impact of climate fluctuation and reflect grassland changes over space and time quantitatively. Such ecological index provides a new understanding of spatial and temporal patterns of grassland degradation in the Three River Headwaters Region (TRHR) at the same time

Mapping Alpine Grassland Fraction Coverage Using Zhuhai-1 OHS Imagery in the Three River Headwaters Region, China

The widely spread alpine grassland ecosystem in the Three River Headwaters Region (TRHR) plays an essential ecological role in carbon sequestration and soil and water conservation. In this study, we test the latest high spatial resolution hyperspectral (Zhuhai-1 OHS) remote sensing imagery to examine different alpine grassland coverage levels using Multiple Endmember Spectral Mixture Analysis (MESMA). Our results suggest that the 3-endmember (3-EM) MESMA model can provide the highest image pixel unmixing percentage, with a percentage exceeding 97% and 96% for pixel scale and landscape scale, respectively. The overall accuracy shows that Zhuhai-1 OHS imagery obtained the highest overall accuracy (83.7%, k = 0.77) in the landscape scale, but in the pixel scale, it is not as good as Landsat 8 OLI imagery. Overall, we can conclude that the hyperspectral imagery combined 3-EM MESMA model performs better in both pixel scale and landscape scale alpine grassland coverage mapping, while the multispectral imagery with the 3-EM MESMA model can satisfy requirements of alpine grassland coverage mapping at the pixel scale. The approaches and workflow to mapping alpine grassland in this study can help monitor alpine grassland degradation; not only in the Qinghai–Tibetan Plateau (QTP), but also in other grassland ecosystems

An Estimation of the Available Spatial Intensity of Solar Energy in Urban Blocks in Wuhan, China

Urban form is an important factor affecting urban energy. However, the design of urban form and energy mostly belong to two separate disciplines and fields, and urban energy planning research rarely considers their mutual relationship. The available space intensity (ASI) of solar energy is formed on the basis of energy planning and urban design; the objective of this research is to evaluate the impact of urban form on the ASI of solar energy and to propose strategies for planning of the space that is available for solar energy so as to improve the efficiency of urban energy utilization and achieve sustainable urban development. Methodologically, this study firstly proposes a model to quantify the ASI of solar energy using three indicators: solar radiation intensity (SRI), solar installation intensity (SII), and solar generation intensity (SEGI). Then, we quantitatively calculate the solar ASI of nine types of typical urban blocks in a sub-center of Wuhan City, Nanhu. Correlation analysis and multiple linear regression analysis are then used to analyze the correlation between the form indicators and solar ASI, as well as the degree of influence. The results show that the differences in SRI, SII, and SEGI amongst the nine types of city blocks were as high as 114.61%, 162.50%, and 61.01%. The solar ASI was mainly affected by three form indicators: the building coverage ratio, the average building height, and the volume-to-area ratio. Reducing the building coverage ratio and increasing vertical development at the same time can effectively improve the ASI of solar energy. The results of this study and the established method provide an important reference and rapid calculation tool for urban energy planning and design, reducing the data and time usually required for solar analysis at the block scale

Inorganic–organic hybrid polymer with multiple redox for high-density data storage

Although organic multilevel resistance memories have attracted much attention for potential realization of the exponentially-increasing density of data storage, the ambiguous structure–property relationship and the unclear switching mechanism impeded further development of multilevel resistance memory devices. Therefore, it is very urgent to ingeniously design multilevel memory materials with a certain switching mechanism. In this contribution, we have employed a multi-redox (multiple barriers) polyoxometalate-based inorganic–organic hybrid polymer (whose effective carriers are electrically controllable) to realize a ternary resistance switching memory (multilevel memories). We do believe that the as-designed inorganic–organic polymer can integrate the multi-redox states of the POM and the processability of flexible polymers together. The as-fabricated multilevel memory devices exhibit rewriteable switching properties among three redox states by applying different RESET voltages, good endurance with distinct operation windows, and long retention. Our results could provide a new strategy to design controllable multilevel resistance memories with excellent performance.NRF (Natl Research Foundation, S’pore)MOE (Min. of Education, S’pore)Accepted versio