Utilising information foraging theory for user interaction with image query auto-completion

Query Auto-completion (QAC) is a prominently used feature in search engines, where user interaction with such explicit feature is facilitated by the possible automatic suggestion of queries based on a prefix typed by the user. Existing QAC models have pursued a little on user interaction and cannot capture a user’s information need (IN) context. In this work, we devise a new task of QAC applied on an image for estimating patch (one of the key components of Information Foraging Theory) probabilities for query suggestion. Our work supports query completion by extending a user query prefix (one or two characters) to a complete query utilising a foraging-based probabilistic patch selection model. We present iBERT, to fine-tune the BERT (Bidirectional Encoder Representations from Transformers) model, which leverages combined textual-image queries for a solution to image QAC by computing probabilities of a large set of image patches. The reflected patch probabilities are used for selection while being agnostic to changing information need or contextual mechanisms. Experimental results show that query auto-completion using both natural language queries and images is more effective than using only language-level queries. Also, our fine-tuned iBERT model allows to efficiently rank patches in the image

Advances in Information Retrieval [electronic resource] : 42nd European Conference on IR Research, ECIR 2020, Lisbon, Portugal, April 14–17, 2020, Proceedings, Part I /

This two-volume set LNCS 12035 and 12036 constitutes the refereed proceedings of the 42nd European Conference on IR Research, ECIR 2020, held in Lisbon, Portugal, in April 2020.* The 55 full papers presented together with 8 reproducibility papers, 46 short papers, 10 demonstration papers, 12 invited CLEF papers, 7 doctoral consortium papers, 4 workshop papers, and 3 tutorials were carefully reviewed and selected from 457 submissions. They were organized in topical sections named: Part I: deep learning I; entities; evaluation; recommendation; information extraction; deep learning II; retrieval; multimedia; deep learning III; queries; IR – general; question answering, prediction, and bias; and deep learning IV. Part II: reproducibility papers; short papers; demonstration papers; CLEF organizers lab track; doctoral consortium papers; workshops; and tutorials. *Due to the COVID-19 pandemic, this conference was held virtually.Deep Learning I -- Seed-guided Deep Document Clustering -- Improving Knowledge Graph Embedding using Locally and Globally Attentive Relation Paths -- ReadNet: A Hierarchical Transformer Framework for Web Article Readability Analysis -- Variational Recurrent Sequence-to-Sequence Retrieval for Stepwise Illustration -- A Hierarchical Model for Data-to-Text Generation -- Entities -- Context-guided Learning to Rank Entities -- Graph-Embedding Empowered Entity Retrieval -- Learning Advanced Similarities and Training Features for Toponym Interlinking -- Patch-based Identification of Lexical Semantic Relations -- Joint Word and Entity Embeddings for Entity Retrieval from a Knowledge Graph -- Evaluation -- Evaluating the Effectiveness of the Standard Insights Extraction Pipeline for Bantu Languages -- Recommendation -- Axiomatic Analysis of Contact Recommendation Methods in Social Networks: an IR perspective -- Recommending Music Curators: A Neural Style-Aware Approach -- Joint Geographical and Temporal Modeling based on Matrix Factorization for Point-of-Interest Recommendation -- Semantic Modelling of Citation Contexts for Context-aware Citation Recommendation -- TransRev: Modeling Reviews as Translations from Users to Items -- Information Extraction -- Domain-independent Extraction of Scientific Concepts from Research Articles -- Leveraging Schema Labels to Enhance Dataset Search -- Moving from formal towards coherent concept analysis: why, when and how -- Beyond Modelling: Understanding Mental Disorders in Online Social Media -- Deep Learning II -- Learning based Methods for Code Runtime Complexity Prediction -- Inductive Document Network Embedding with Topic-Word Attention -- Multi-components system for automatic Arabic diacritization -- A Mixed Semantic Features Model for Chinese NER with Characters and Words -- VGCN-BERT: Augmenting BERT with Graph Embedding for Text Classification -- Retrieval -- A Computational Approach for Objectively Derived Systematic Review Search Strategies -- You Can Teach an Old Dog New Tricks: Rank Fusion applied to Coordination Level Matching for Ranking in Systematic Reviews -- Counterfactual Online Learning to Rank -- A Framework for Argument Retrieval: Ranking Argument Clusters by Frequency and Specificity -- Relevance Ranking based on Query-Aware Context Analysis -- Multimedia -- Multimodal Entity Linking for Tweets -- MEMIS: Multimodal Emergency Management Information System -- Interactive Learning for Multimedia at Large -- Visual Re-ranking via Adaptive Collaborative Hypergraph Learning for Image Retrieval -- Motion Words: A Text-like Representation of 3D Skeleton Sequences -- Deep Learning III -- Reinforced Rewards Framework for Text Style Transfer -- Recognizing Semantic Relations: Attention-Based Transformers vs. Recurrent Models -- Early detection of rumours on Twitter via stance transfer learning -- Learning to Rank Images with Cross-Modal Graph Convolutions -- Diagnosing BERT with Retrieval Heuristics -- Queries -- Generation of Synthetic Query Auto Completion Logs -- What Can Task Teach Us About Query Reformulations? -- A Regularised Intent Model for Discovering Multiple Intents in E-Commerce Tail Queries -- Utilising Information Foraging Theory for User Interaction with Image Query Auto-Completion -- Using Image Captions and Multitask Learning for Recommending Query Reformulations -- IR – general -- Curriculum Learning Strategies for IR: An Empirical Study on Conversation Response Ranking -- Accelerating Substructure Similarity Search for Formula Retrieval -- Quantum-like Structure in Multidimensional Relevance Judgements -- Question Answering, Prediction, and Bias -- Temporal Latent Space Modeling for Community Prediction -- KvGR: A Graph-Based Interface for Explorative Sequential Question Answering on Heterogeneous Information Sources -- Answering Event-Related Questions over Long-term News Article Archives -- bias goggles - Graph-based Computation of the Bias of Web Domains through the Eyes of Users -- Deep Learning IV -- Biconditional Generative Adversarial Networks for Multiview Learning with Missing Views -- Semantic Path-Based Learning for Review Volume Prediction -- An Attention Model of Customer Expectation to Improve Review Helpfulness Prediction.This two-volume set LNCS 12035 and 12036 constitutes the refereed proceedings of the 42nd European Conference on IR Research, ECIR 2020, held in Lisbon, Portugal, in April 2020.* The 55 full papers presented together with 8 reproducibility papers, 46 short papers, 10 demonstration papers, 12 invited CLEF papers, 7 doctoral consortium papers, 4 workshop papers, and 3 tutorials were carefully reviewed and selected from 457 submissions. They were organized in topical sections named: Part I: deep learning I; entities; evaluation; recommendation; information extraction; deep learning II; retrieval; multimedia; deep learning III; queries; IR – general; question answering, prediction, and bias; and deep learning IV. Part II: reproducibility papers; short papers; demonstration papers; CLEF organizers lab track; doctoral consortium papers; workshops; and tutorials. *Due to the COVID-19 pandemic, this conference was held virtually