Fuzzy aesthetic semantics description and extraction for art image retrieval

AbstractMore and more digitized art images are accumulated and expanded in our daily life and techniques are needed to be established on how to organize and retrieve them. Though content-based image retrieval (CBIR) made great progress, current low-level visual information based retrieval technology in CBIR does not allow users to search images by high-level semantics for art image retrieval. We propose a fuzzy approach to describe and to extract the fuzzy aesthetic semantic feature of art images. Aiming to deal with the subjectivity and vagueness of human aesthetic perception, we utilize the linguistic variable to describe the image aesthetic semantics, so it becomes possible to depict images in linguistic expression such as ‘very action’. Furthermore, we apply neural network approach to model the process of human aesthetic perception and to extract the fuzzy aesthetic semantic feature vector. The art image retrieval system based on fuzzy aesthetic semantic feature makes users more naturally search desired images by linguistic expression. We report extensive empirical studies based on a 5000-image set, and experimental results demonstrate that the proposed approach achieves excellent performance in terms of retrieval accuracy

Автентифікація зображень на основі їх семантичної сегментації у нейронних мережах глибокого навчання з їх попереднім обробленням за методами фільтрації

In modern domestic and international pre-trial practice and legal proceedings, physical evidence used in the form of electronic documents or their digital images. The issue of authenticating such images is hampered by possibilities of using artificial intelligence based editors to fake images making it impossible or significantly complicate the search for changed areas by forensic experts. Research issue considered by the authors is the authenticity assessment of digital images based on the use of their pre-processing (filtering) methods and artificial intelligence technologies for further analysis and determination of edited areas. This research paper purpose is to develop information technology for finding editing zones based on a combination of imaging techniques and neural network models for use in image authenticity research. Research paper novelty is to develop a technology to combine several methods of preprocessing images (in particular, ELA and PCA) to create an input stream of a deep learning neural network and to assess effectiveness of identifying editing zones created by the digital painting editor (Inpainting). Efficiency of 10 editing zone detectors using combination of ELA and PCA methods with different models of neural networks for recognizing editing zones has been developed and investigated. The best results (probability of recognition 0.916) within the used computing resources were obtained by detector based on the EfficientNet model. IT Effectiveness and related software for assessing authenticity of images based on a combination of image pre-processing methods and models of artificial neural networks in semantic classification and segmentation mode has been developed and evaluated

Utilising semantic technologies for intelligent indexing and retrieval of digital images

The proliferation of digital media has led to a huge interest in classifying and indexing media objects for generic search and usage. In particular, we are witnessing colossal growth in digital image repositories that are difficult to navigate using free-text search mechanisms, which often return inaccurate matches as they in principle rely on statistical analysis of query keyword recurrence in the image annotation or surrounding text. In this paper we present a semantically-enabled image annotation and retrieval engine that is designed to satisfy the requirements of the commercial image collections market in terms of both accuracy and efficiency of the retrieval process. Our search engine relies on methodically structured ontologies for image annotation, thus allowing for more intelligent reasoning about the image content and subsequently obtaining a more accurate set of results and a richer set of alternatives matchmaking the original query. We also show how our well-analysed and designed domain ontology contributes to the implicit expansion of user queries as well as the exploitation of lexical databases for explicit semantic-based query expansion

Strategies for Searching Video Content with Text Queries or Video Examples

The large number of user-generated videos uploaded on to the Internet everyday has led to many commercial video search engines, which mainly rely on text metadata for search. However, metadata is often lacking for user-generated videos, thus these videos are unsearchable by current search engines. Therefore, content-based video retrieval (CBVR) tackles this metadata-scarcity problem by directly analyzing the visual and audio streams of each video. CBVR encompasses multiple research topics, including low-level feature design, feature fusion, semantic detector training and video search/reranking. We present novel strategies in these topics to enhance CBVR in both accuracy and speed under different query inputs, including pure textual queries and query by video examples. Our proposed strategies have been incorporated into our submission for the TRECVID 2014 Multimedia Event Detection evaluation, where our system outperformed other submissions in both text queries and video example queries, thus demonstrating the effectiveness of our proposed approaches

Image Semantics in the Description and Categorization of Journalistic Photographs

This paper reports a study on the description and categorization of images. The aim of the study was to evaluate existing indexing frameworks in the context of reportage photographs and to find out how the use of this particular image genre influences the results. The effect of different tasks on image description and categorization was also studied. Subjects performed keywording and free description tasks and the elicited terms were classified using the most extensive one of the reviewed frameworks. Differences were found in the terms used in constrained and unconstrained descriptions. Summarizing terms such as abstract concepts, themes, settings and emotions were used more frequently in keywording than in free description. Free descriptions included more terms referring to locations within the images, people and descriptive terms due to the narrative form the subjects used without prompting. The evaluated framework was found to lack some syntactic and semantic classes present in the data and modifications were suggested. According to the results of this study image categorization is based on high-level interpretive concepts, including affective and abstract themes. The results indicate that image genre influences categorization and keywording modifies and truncates natural image description

CHORUS Deliverable 2.1: State of the Art on Multimedia Search Engines

Based on the information provided by European projects and national initiatives related to multimedia search as well as domains experts that participated in the CHORUS Think-thanks and workshops, this document reports on the state of the art related to multimedia content search from, a technical, and socio-economic perspective. The technical perspective includes an up to date view on content based indexing and retrieval technologies, multimedia search in the context of mobile devices and peer-to-peer networks, and an overview of current evaluation and benchmark inititiatives to measure the performance of multimedia search engines. From a socio-economic perspective we inventorize the impact and legal consequences of these technical advances and point out future directions of research

Applying semantic web technologies to knowledge sharing in aerospace engineering

This paper details an integrated methodology to optimise Knowledge reuse and sharing, illustrated with a use case in the aeronautics domain. It uses Ontologies as a central modelling strategy for the Capture of Knowledge from legacy docu-ments via automated means, or directly in systems interfacing with Knowledge workers, via user-defined, web-based forms. The domain ontologies used for Knowledge Capture also guide the retrieval of the Knowledge extracted from the data using a Semantic Search System that provides support for multiple modalities during search. This approach has been applied and evaluated successfully within the aerospace domain, and is currently being extended for use in other domains on an increasingly large scale