OCC: A Smart Reply System for Efficient In-App Communications

Smart reply systems have been developed for various messaging platforms. In this paper, we introduce Uber's smart reply system: one-click-chat (OCC), which is a key enhanced feature on top of the Uber in-app chat system. It enables driver-partners to quickly respond to rider messages using smart replies. The smart replies are dynamically selected according to conversation content using machine learning algorithms. Our system consists of two major components: intent detection and reply retrieval, which are very different from standard smart reply systems where the task is to directly predict a reply. It is designed specifically for mobile applications with short and non-canonical messages. Reply retrieval utilizes pairings between intent and reply based on their popularity in chat messages as derived from historical data. For intent detection, a set of embedding and classification techniques are experimented with, and we choose to deploy a solution using unsupervised distributed embedding and nearest-neighbor classifier. It has the advantage of only requiring a small amount of labeled training data, simplicity in developing and deploying to production, and fast inference during serving and hence highly scalable. At the same time, it performs comparably with deep learning architectures such as word-level convolutional neural network. Overall, the system achieves a high accuracy of 76% on intent detection. Currently, the system is deployed in production for English-speaking countries and 71% of in-app communications between riders and driver-partners adopted the smart replies to speedup the communication process.Comment: link to demo: https://www.youtube.com/watch?v=nOffUT7rS0A&t=32

Retrieval Enhancements for Task-Based Web Search

The task-based view of web search implies that retrieval should take the user perspective into account. Going beyond merely retrieving the most relevant result set for the current query, the retrieval system should aim to surface results that are actually useful to the task that motivated the query. This dissertation explores how retrieval systems can better understand and support their users’ tasks from three main angles: First, we study and quantify search engine user behavior during complex writing tasks, and how task success and behavior are associated in such settings. Second, we investigate search engine queries formulated as questions, and explore patterns in a large query log that may help search engines to better support this increasingly prevalent interaction pattern. Third, we propose a novel approach to reranking the search result lists produced by web search engines, taking into account retrieval axioms that formally specify properties of a good ranking.Die Task-basierte Sicht auf Websuche impliziert, dass die Benutzerperspektive berücksichtigt werden sollte. Über das bloße Abrufen der relevantesten Ergebnismenge für die aktuelle Anfrage hinaus, sollten Suchmaschinen Ergebnisse liefern, die tatsächlich für die Aufgabe (Task) nützlich sind, die diese Anfrage motiviert hat. Diese Dissertation untersucht, wie Retrieval-Systeme die Aufgaben ihrer Benutzer besser verstehen und unterstützen können, und leistet Forschungsbeiträge unter drei Hauptaspekten: Erstens untersuchen und quantifizieren wir das Verhalten von Suchmaschinenbenutzern während komplexer Schreibaufgaben, und wie Aufgabenerfolg und Verhalten in solchen Situationen zusammenhängen. Zweitens untersuchen wir Suchmaschinenanfragen, die als Fragen formuliert sind, und untersuchen ein Suchmaschinenlog mit fast einer Milliarde solcher Anfragen auf Muster, die Suchmaschinen dabei helfen können, diesen zunehmend verbreiteten Anfragentyp besser zu unterstützen. Drittens schlagen wir einen neuen Ansatz vor, um die von Web-Suchmaschinen erstellten Suchergebnislisten neu zu sortieren, wobei Retrieval-Axiome berücksichtigt werden, die die Eigenschaften eines guten Rankings formal beschreiben

Diversified query expansion

La diversification des résultats de recherche (DRR) vise à sélectionner divers documents à partir des résultats de recherche afin de couvrir autant d’intentions que possible. Dans les approches existantes, on suppose que les résultats initiaux sont suffisamment diversifiés et couvrent bien les aspects de la requête. Or, on observe souvent que les résultats initiaux n’arrivent pas à couvrir certains aspects. Dans cette thèse, nous proposons une nouvelle approche de DRR qui consiste à diversifier l’expansion de requête (DER) afin d’avoir une meilleure couverture des aspects. Les termes d’expansion sont sélectionnés à partir d’une ou de plusieurs ressource(s) suivant le principe de pertinence marginale maximale. Dans notre première contribution, nous proposons une méthode pour DER au niveau des termes où la similarité entre les termes est mesurée superficiellement à l’aide des ressources. Quand plusieurs ressources sont utilisées pour DER, elles ont été uniformément combinées dans la littérature, ce qui permet d’ignorer la contribution individuelle de chaque ressource par rapport à la requête. Dans la seconde contribution de cette thèse, nous proposons une nouvelle méthode de pondération de ressources selon la requête. Notre méthode utilise un ensemble de caractéristiques qui sont intégrées à un modèle de régression linéaire, et génère à partir de chaque ressource un nombre de termes d’expansion proportionnellement au poids de cette ressource. Les méthodes proposées pour DER se concentrent sur l’élimination de la redondance entre les termes d’expansion sans se soucier si les termes sélectionnés couvrent effectivement les différents aspects de la requête. Pour pallier à cet inconvénient, nous introduisons dans la troisième contribution de cette thèse une nouvelle méthode pour DER au niveau des aspects. Notre méthode est entraînée de façon supervisée selon le principe que les termes reliés doivent correspondre au même aspect. Cette méthode permet de sélectionner des termes d’expansion à un niveau sémantique latent afin de couvrir autant que possible différents aspects de la requête. De plus, cette méthode autorise l’intégration de plusieurs ressources afin de suggérer des termes d’expansion, et supporte l’intégration de plusieurs contraintes telles que la contrainte de dispersion. Nous évaluons nos méthodes à l’aide des données de ClueWeb09B et de trois collections de requêtes de TRECWeb track et montrons l’utilité de nos approches par rapport aux méthodes existantes.Search Result Diversification (SRD) aims to select diverse documents from the search results in order to cover as many search intents as possible. For the existing approaches, a prerequisite is that the initial retrieval results contain diverse documents and ensure a good coverage of the query aspects. In this thesis, we investigate a new approach to SRD by diversifying the query, namely diversified query expansion (DQE). Expansion terms are selected either from a single resource or from multiple resources following the Maximal Marginal Relevance principle. In the first contribution, we propose a new term-level DQE method in which word similarity is determined at the surface (term) level based on the resources. When different resources are used for the purpose of DQE, they are combined in a uniform way, thus totally ignoring the contribution differences among resources. In practice the usefulness of a resource greatly changes depending on the query. In the second contribution, we propose a new method of query level resource weighting for DQE. Our method is based on a set of features which are integrated into a linear regression model and generates for a resource a number of expansion candidates that is proportional to the weight of that resource. Existing DQE methods focus on removing the redundancy among selected expansion terms and no attention has been paid on how well the selected expansion terms can indeed cover the query aspects. Consequently, it is not clear how we can cope with the semantic relations between terms. To overcome this drawback, our third contribution in this thesis aims to introduce a novel method for aspect-level DQE which relies on an explicit modeling of query aspects based on embedding. Our method (called latent semantic aspect embedding) is trained in a supervised manner according to the principle that related terms should correspond to the same aspects. This method allows us to select expansion terms at a latent semantic level in order to cover as much as possible the aspects of a given query. In addition, this method also incorporates several different external resources to suggest potential expansion terms, and supports several constraints, such as the sparsity constraint. We evaluate our methods using ClueWeb09B dataset and three query sets from TRECWeb tracks, and show the usefulness of our proposed approaches compared to the state-of-the-art approaches

Temporal dynamics in information retrieval

The passage of time is unrelenting. Time is an omnipresent feature of our existence, serving as a context to frame change driven by events and phenomena in our personal lives and social constructs. Accordingly, various elements of time are woven throughout information itself, and information behaviours such as creation, seeking and utilisation. Time plays a central role in many aspects of information retrieval (IR). It can not only distinguish the interpretation of information, but also profoundly influence the intentions and expectations of users' information seeking activity. Many time-based patterns and trends - namely temporal dynamics - are evident in streams of information behaviour by individuals and crowds. A temporal dynamic refers to a periodic regularity, or, a one-off or irregular past, present or future of a particular element (e.g., word, topic or query popularity) - driven by predictable and unpredictable time-based events and phenomena. Several challenges and opportunities related to temporal dynamics are apparent throughout IR. This thesis explores temporal dynamics from the perspective of query popularity and meaning, and word use and relationships over time. More specifically, the thesis posits that temporal dynamics provide tacit meaning and structure of information and information seeking. As such, temporal dynamics are a ‘two-way street’ since they must be supported, but also conversely, can be exploited to improve time-aware IR effectiveness. Real-time temporal dynamics in information seeking must be supported for consistent user satisfaction over time. Uncertainty about what the user expects is a perennial problem for IR systems, further confounded by changes over time. To alleviate this issue, IR systems can: (i) assist the user to submit an effective query (e.g., error-free and descriptive), and (ii) better anticipate what the user is most likely to want in relevance ranking. I first explore methods to help users formulate queries through time-aware query auto-completion, which can suggest both recent and always popular queries. I propose and evaluate novel approaches for time-sensitive query auto-completion, and demonstrate state-of-the-art performance of up to 9.2% improvement above the hard baseline. Notably, I find results are reflected across diverse search scenarios in different languages, confirming the pervasive and language agnostic nature of temporal dynamics. Furthermore, I explore the impact of temporal dynamics on the motives behind users' information seeking, and thus how relevance itself is subject to temporal dynamics. I find that temporal dynamics have a dramatic impact on what users expect over time for a considerable proportion of queries. In particular, I find the most likely meaning of ambiguous queries is affected over short and long-term periods (e.g., hours to months) by several periodic and one-off event temporal dynamics. Additionally, I find that for event-driven multi-faceted queries, relevance can often be inferred by modelling the temporal dynamics of changes in related information. In addition to real-time temporal dynamics, previously observed temporal dynamics offer a complementary opportunity as a tacit dimension which can be exploited to inform more effective IR systems. IR approaches are typically based on methods which characterise the nature of information through the statistical distributions of words and phrases. In this thesis I look to model and exploit the temporal dimension of the collection, characterised by temporal dynamics, in these established IR approaches. I explore how the temporal dynamic similarity of word and phrase use in a collection can be exploited to infer temporal semantic relationships between the terms. I propose an approach to uncover a query topic's "chronotype" terms -- that is, its most distinctive and temporally interdependent terms, based on a mix of temporal and non-temporal evidence. I find exploiting chronotype terms in temporal query expansion leads to significantly improved retrieval performance in several time-based collections. Temporal dynamics provide both a challenge and an opportunity for IR systems. Overall, the findings presented in this thesis demonstrate that temporal dynamics can be used to derive tacit structure and meaning of information and information behaviour, which is then valuable for improving IR. Hence, time-aware IR systems which take temporal dynamics into account can better satisfy users consistently by anticipating changing user expectations, and maximising retrieval effectiveness over time

Hábitos de recuperación de información en motores de búsqueda sobre lectura, libro y bibliotecas en España (2004-2016)

Este estudio ha tenido como objetivo principal, determinar si los procesos y expresiones de búsqueda de información usados por los usuarios en motores de búsqueda, pueden considerarse como indicadores válidos para el análisis y estudio de los hábitos de lectura y posible interés en otros contenidos ofrecidos por las bibliotecas en España (como videojuegos o películas).Para ello se propone un modelo de análisis con el que caracterizar el lenguaje de búsqueda de información de los usuarios de internet que utilizan Google desde España como motor de búsqueda, durante el período 2004 - 2016, al recuperar información sobre la temática de el libro, la lectura y las bibliotecas, desde una perspectiva histórica. De esta forma, se pretende aportar otra dimensión de análisis a los estudios que hay sobre los hábitos lectores en general, y en España en particular.La investigación tiene distintas áreas de aplicación del análisis del lector online, como son el apoyo a la indización y la clasificación bibliotecaria, la evaluación de colecciones y evaluación de la biblioteca, los estudios de necesidades de usuarios, la evaluación de OPACs, la analítica digital de sedes web bibliotecarias o de entidades de la industria del libro como editoriales, librerías online, metabuscadores o páginas web de autores y aficionados a la literatura en general, márketing bibliotecario y promoción de la lectura, márketing editorial, altmetría y Cibermetría, y SEO (posicionamiento en buscadores).El análisis de los hábitos lectores tiene una larga tradición en el mundo offline, especialmente en España, donde el estudio de hábitos lectores es parte importante de la investigación estratégica en la industria del libro. Se han observado distintas metodologías, desde las encuestas y entrevistas a lectores y no lectores, el análisis de las ventas de los libros y la prensa, a los análisis de logs de préstamos en las bibliotecas. Al entrar la lectura en e-book, y en plena era de internet, la lectura en papel ha sufrido una transformación, donde los usuarios leen por internet, y buscan su lectura (ya sea online, en e-book y/o en papel) a través de internet, especialmente utilizando motores de búsqueda, de los que en España el más utilizado desde principios de siglo hasta al menos su segunda década, es el buscador Google. Es este cambio en las formas de localizar la lectura la que impulsa a investigar cómo se busca información sobre lectura en un buscador. Anteriormente se han investigado distintos aspectos de esas conductas con distintas técnicas, dentro del paradigma cognitivo, y especialmente dentro de la disciplina de Information Seeking, de difícil traducción al castellano. Tras consignar modelos de búsqueda por parte de los usuarios, como el modelo Berrypicking de Marcia Bates, el modelo de Ellis, el modelo de Marchionini, o el modelo de Information Search Process de Kulthau, entre otros, se han estudiado otros modificadores de las conductas de búsqueda, llegando a los estudios sobre User Search Behaviour (conductas de búsqueda de los usuarios en motores de búsqueda) especialmente en lo concerniente a desambiguación y expansión de búsquedas, análisis longitudinal de la búsqueda y de Query Intent, el Análisis de la Intención de Búsqueda. Es precísamente en la combinación de las últimas subdisciplinas hacia donde se ha orientado este estudio. Para la investigación, en 2010 se obtuvieron de Google Keywords Planner, el log de búsquedas del motor de búsqueda, más de 30.000 expresiones de búsqueda (denominadas también como frases de búsqueda, queries, keywords o palabras clave), relacionadas con el libro, la lectura y las bibliotecas, segmentando la búsqueda de palabras clave en lenguaje español y de búsquedas realizadas desde España. Posteriormente se extrajo de Google Trends la serie de datos histórica de 2004 a 2016, para conformar un dataset con el que realizar un análisis longitudinal. Las palabras clave fueron clasificadas en 27 facetas distintas de intención de búsqueda, contando también con aspectos modificadores y aspectos lingüísticos. Por tanto, no se clasificó en categorías mutuamente excluyentes, sino de forma que una expresión de búsqueda pudiera pertenecer a varias clases simultáneamente, por lo que se realizó un estudio del grado de co-ocurrencia entre las distintas facetas y los aspectos identificados. Posteriormente se dividió las palabras clave, previamente clasificadas, en una nueva dimensión de análisis, según si era atemporales (tenían una larga vida en la serie histórica) o temporales, aquellas que nacían en algún momento de la serie, y tenían una vida más o menos corta. Como resultado del análisis, se han estudiado las posibilidades de la facetación como mejora o complemento de otras técnicas de análisis de las intenciones de búsqueda (query intent analysis); se ha validado el modelo de estudio, de forma que sirva como corpus inicial de futuros análisis de los hábitos de lectura en España, a través del estudio de la demanda de información en motores de búsqueda; se han descubierto subtipos de intenciones de búsqueda propias del sector de la lectura, dentro de las clasificaciones clásicas de intención de búsqueda (navegacional, informacional, transaccional); se han identificado facetas adicionales, distintas a las meramente temáticas, como modificadores y características del lenguaje, que sirvan para completar las facetas halladas desde una dimensión de análisis complementaria; se ha descubierto distintos patrones de uso, nuevas abreviaturas y formas de expresión de las necesidades de búsqueda de los usuarios mediante lenguaje natural, se han relacionado distintos media y/o formatos, así como, tras una selección mediante una muestra intencionada, de distintos ejemplos paradigmáticos de estas tendencias de búsqueda y sus posibles relaciones causales, observando los efectos producidos en la evolución de la demanda de información en torno a la lectura a través de la búsqueda de la misma en Google en España, durante el período 2004-2016.Finalmente, y además de constatar su utilidad para completar otras técnicas de análisis de los hábitos lectores mediante una técnica inédita hasta la fecha en el sector del libro y bibliotecas, se ha observado cómo la demanda de información sobre lectura en España realizada a través de motores de búsqueda, ha decaído de forma paulatina en la segunda década del siglo XXI, coincidiendo con otras investigaciones y datos de estudios de hábitos lectores realizadas a través de otras técnicas. <br /

What Presentation of Search Engine Results Do Health Information Searchers Prefer?

A study of a sample of online health information searchers was conducted to see what their preferences are with respect to four different display styles for search engine results on health topics. Screen shots of search result display screens were presented to the participants via a Qualtrics (www.qualtrics.com) online survey. The other display types were Display 1: Google standard display, Display 2: Google enhanced with faceted browsable categories, Display 3: Google enhanced with a word cloud for each search result, and Display 4: Google enhanced with an overview word cloud for collection of search results. For each search task, participants were asked to rate the search engine results displays for quality indicators, using Likert-type item rating scales. At the end, in three concluding questions, the participants were asked to choose the display(s) that were best at meeting three specific criteria, based on overall impressions. The evaluations by the participants suggest that the standard Google search results display and the Google screen enhanced with faceted browsable categories were favored over the other two display types.Master of Science in Information Scienc

Improving Search Effectiveness through Query Log and Entity Mining

The Web is the largest repository of knowledge in the world. Everyday people contribute to make it bigger by generating new web data. Data never sleeps. Every minute someone writes a new blog post, uploads a video or comments on an article. Usually people rely on Web Search Engines for satisfying their information needs: they formulate their needs as text queries and they expect a list of highly relevant documents answering their requests. Being able to manage this massive volume of data, ensuring high quality and performance, is a challenging topic that we tackle in this thesis. In this dissertation we focus on the Web of Data: a recent approach, originated from the Semantic Web community, consisting in a collective effort to augment the existing Web with semistructured-data. We propose to manage the data explosion shifting from a retrieval model based on documents to a model enriched with entities, where an entity can describe a person, a product, a location, a company, through semi-structured information. In our work, we combine the Web of Data with an important source of knowledge: query logs, which record the interactions between the Web Search Engine and the users. Query log mining aims at extracting valuable knowledge that can be exploited to enhance users’ search experience. According to this vision, this dissertation aims at improving Web Search Engines toward the mutual use of query logs and entities. The contributions of this work are the following: we show how historical usage data can be exploited for improving performance during the snippet generation process. Secondly, we propose a query recommender system that, by combining entities with queries, leads to significant improvements to the quality of the suggestions. Furthermore, we develop a new technique for estimating the relatedness between two entities, i.e., their semantic similarity. Finally, we show that entities may be useful for automatically building explanatory statements that aim at helping the user to better understand if, and why, the suggested item can be of her interest

Named entity recognition and classification in search queries

Named Entity Recognition and Classification is the task of extracting from text, instances of different entity classes such as person, location, or company. This task has recently been applied to web search queries in order to better understand their semantics, where a search query consists of linguistic units that users submit to a search engine to convey their search need. Discovering and analysing the linguistic units comprising a search query enables search engines to reveal and meet users' search intents. As a result, recent research has concentrated on analysing the constituent units comprising search queries. However, since search queries are short, unstructured, and ambiguous, an approach to detect and classify named entities is presented in this thesis, in which queries are augmented with the text snippets of search results for search queries. The thesis makes the following contributions: 1. A novel method for detecting candidate named entities in search queries, which utilises both query grammatical annotation and query segmentation. 2. A novel method to classify the detected candidate entities into a set of target entity classes, by using a seed expansion approach; the method presented exploits the representation of the sets of contextual clues surrounding the entities in the snippets as vectors in a common vector space. 3. An exploratory analysis of three main categories of search refiners: nouns, verbs, and adjectives, that users often incorporate in entity-centric queries in order to further refine the entity-related search results. 4. A taxonomy of named entities derived from a search engine query log. By using a large commercial query log, experimental evidence is provided that the work presented herein is competitive with the existing research in the field of entity recognition and classification in search queries

Named entity recognition and classification in search queries

Named Entity Recognition and Classification is the task of extracting from text, instances of different entity classes such as person, location, or company. This task has recently been applied to web search queries in order to better understand their semantics, where a search query consists of linguistic units that users submit to a search engine to convey their search need. Discovering and analysing the linguistic units comprising a search query enables search engines to reveal and meet users' search intents. As a result, recent research has concentrated on analysing the constituent units comprising search queries. However, since search queries are short, unstructured, and ambiguous, an approach to detect and classify named entities is presented in this thesis, in which queries are augmented with the text snippets of search results for search queries. The thesis makes the following contributions: 1. A novel method for detecting candidate named entities in search queries, which utilises both query grammatical annotation and query segmentation. 2. A novel method to classify the detected candidate entities into a set of target entity classes, by using a seed expansion approach; the method presented exploits the representation of the sets of contextual clues surrounding the entities in the snippets as vectors in a common vector space. 3. An exploratory analysis of three main categories of search refiners: nouns, verbs, and adjectives, that users often incorporate in entity-centric queries in order to further refine the entity-related search results. 4. A taxonomy of named entities derived from a search engine query log. By using a large commercial query log, experimental evidence is provided that the work presented herein is competitive with the existing research in the field of entity recognition and classification in search queries