Fast Data in the Era of Big Data: Twitter's Real-Time Related Query Suggestion Architecture

We present the architecture behind Twitter's real-time related query suggestion and spelling correction service. Although these tasks have received much attention in the web search literature, the Twitter context introduces a real-time "twist": after significant breaking news events, we aim to provide relevant results within minutes. This paper provides a case study illustrating the challenges of real-time data processing in the era of "big data". We tell the story of how our system was built twice: our first implementation was built on a typical Hadoop-based analytics stack, but was later replaced because it did not meet the latency requirements necessary to generate meaningful real-time results. The second implementation, which is the system deployed in production, is a custom in-memory processing engine specifically designed for the task. This experience taught us that the current typical usage of Hadoop as a "big data" platform, while great for experimentation, is not well suited to low-latency processing, and points the way to future work on data analytics platforms that can handle "big" as well as "fast" data

Analisis pada Fitur Autocomplete Suggestion dan Semantik pada Pencarian di Mesin Pencari Google

Perkembangan teknologi internet mendorong munculnya fitur dan inovasiterbaru untuk meningkatkan pengalaman dan kemudahan pengguna dalammenjelajahi dunia maya. Salah satu fitur tersebut adalah autocompletesuggestion. Fitur autocomplete suggestion merupakan fitur yangdiimplementasikan pada web browser dan mesin pencari yang memungkinkanweb browser atau mesin pencari untuk memberikan saran pencarian ketikabaru beberapa kata diketikan dalam kolom pencarian atau address bar.Makalah ini membahas analisis tentang fitur autocomplete suggestion dansemantik dari sistem pencarian pada salah satu mesin pencari terpopuler didunia, yaitu Google

A Novel Combined Term Suggestion Service for Domain-Specific Digital Libraries

Interactive query expansion can assist users during their query formulation process. We conducted a user study with over 4,000 unique visitors and four different design approaches for a search term suggestion service. As a basis for our evaluation we have implemented services which use three different vocabularies: (1) user search terms, (2) terms from a terminology service and (3) thesaurus terms. Additionally, we have created a new combined service which utilizes thesaurus term and terms from a domain-specific search term re-commender. Our results show that the thesaurus-based method clearly is used more often compared to the other single-method implementations. We interpret this as a strong indicator that term suggestion mechanisms should be domain-specific to be close to the user terminology. Our novel combined approach which interconnects a thesaurus service with additional statistical relations out-performed all other implementations. All our observations show that domain-specific vocabulary can support the user in finding alternative concepts and formulating queries.Comment: To be published in Proceedings of Theories and Practice in Digital Libraries (TPDL), 201

Spellchecking for Children in Web Search: A Natural Language Interface Case-Study

Given the more widespread nature of natural language interfaces, it is increasingly important to understand who are accessing those interfaces, and how those interfaces are being used. In this paper, we explore spellchecking in the context of web search with children as the target audience. In particular, via a literature review we show that, while widely used, popular search tools are ill-designed for children. We then use spellcheckers as a case study to highlight the need for an interdisciplinary approach that brings together natural language processing, education, human-computer interaction to address a known information retrieval problem: query misspelling. We conclude that it is imperative that those for whom the interfaces are designed have a voice in the design process

Instant-Fuzzy search using Phrase Indexing and Segmentation with Proximity Ranking

Quick search is an information-retrieval in which a system finds answers to a query instantly whenever a user types query character-by-character. Now a days, instant search is basically beneficial task for the user to get effective responses to the query when user typing a query in search engine. Fuzzy search used to improve user search familiarities by finding relevant answers with keywords similar to query keywords. We are using phrase inception value which is used to limit the answer set generated by instant fuzzy search. For that main challenge is that to improve the speed of performance as well as minimize answer set to retrieval of desired documents for the user query. At the same time, we also need better ranking functions that consider the proximity of keywords to compute relevance scores. In this paper, we study how to compute proximity information into ranking in instant-fuzzy search while achieving efficient time and space complexities. A phrase base indexing technique is used to overcome the space and time limitations of these solutions, we propose an approach that focuses on mutual phrases in the database. We study how to index these phrase threshold value and compare user threshold for effective answer set and develop an computational algorithm for workwise segmenting a query into phrases and computing these phrases using algorithm to find related answers to the user query