Interactions between users and information access systems can be analyzed and studied to gather user preferences and to learn what a user likes the most, and to use this

information to adapt the search to users and personalize the presentation of results. The LogCLEF lab - ”A benchmarking activity on Multilingual Log File Analysis: Language

identification, query classification, success of a query” deals with information contained in query logs of search engines and digital libraries from which knowledge can be mined to understand search behavior in multilingual context. LogCLEF has created the first long-term standard collection for evaluation purposes in the area of log analysis. The LogCLEF 2011 lab is the continuation of the past two editions: as a pilot task in CLEF 2009, and a workshop in CLEF 2010. The Cross-Language Evaluation Forum (CLEF) promotes research and development in multilingual information access and is an activity of the PROMISE Network of Excellence

Di Nunzio, Giorgio Maria

Leveling, Johannes

Mandl, Thomas

English

Interactions between users and information access systems can be analyzed and studied to gather user preferences and to learn what a user likes the most, and to use this
information to adapt the search to users and personalize the presentation of results. The LogCLEF lab - ”A benchmarking activity on Multilingual Log File Analysis: Language
identification, query classification, success of a query” deals with information contained in query logs of search engines and digital libraries from which knowledge can be mined to understand search behavior in multilingual context. LogCLEF has created the first long-term standard collection for evaluation purposes in the area of log analysis. The LogCLEF 2011 lab is the continuation of the past two editions: as a pilot task in CLEF 2009, and a workshop in CLEF 2010. The Cross-Language Evaluation Forum (CLEF) promotes research and development in multilingual information access and is an activity of the PROMISE Network of Excellence

Irish Universities

LogCLEF: Enabling research on multilingual log files

DCU Online Research Access Service

LogCLEF: Enabling Research on Multilingual Log FilesJohannes Leveling1, Giorgio Maria Di Nunzio2, and Thomas Mandl31 Centre for Next Generation Localisation (CNGL)School of Computing, Dublin City University, Dublin 9, Irelandjleveling@computing.dcu.ie2 Department of Information Engineering – University of PaduaVia Gradenigo, 6/a – 35131 Padua – Italydinunzio@dei.unipd.it3 Information Science,University of Hildesheim, Germanymandl@uni-hildesheim.de1 OverviewInteractions between users and information access systems can be analyzed and stud-ied to gather user preferences and to learn what a user likes the most, and to use thisinformation to adapt the search to users and personalize the presentation of results.The LogCLEF lab - ”A benchmarking activity on Multilingual Log File Analysis: Lan-guage identification, query classification, success of a query” deals with informationcontained in query logs of search engines and digital libraries from which knowledgecan be mined to understand search behavior in multilingual context. LogCLEF has cre-ated the first long-term standard collection for evaluation purposes in the area of loganalysis. The LogCLEF 2011 lab is the continuation of the past two editions: as a pilottask in CLEF 2009, and a workshop in CLEF 2010. The Cross-Language EvaluationForum (CLEF) promotes research and development in multilingual information accessand is an activity of the PROMISE Network of Excellence.2 Topic and GoalLog data constitute a relevant aspect in the evaluation process of the quality of a searchengine and the quality of a multilingual search service; log data can be used to study theusage of a search engine, and to better adapt it to the objectives the users were expectingto reach. Research on log files is an area with obstacles. Most search services, especiallylarge search engines do exploit their search logs but do not grant public access to them.A standard evaluation resource will make systems comparable and make research moretransparent. More groups who do currently not have access to real world log files canwork in the field of log file analysis.The research goal of LogCLEF is the analysis and classification of queries, the def-inition of success of a search in order to understand search behaviour in multilingualcontexts. One of the aims of LogCLEF is to foster the exchange of systems and compo-nents as well as the sharing of heterogeneous annotation of log data in order to advancethe state of the art in this research area.3 TasksLogCLEF 2011 offers three tasks, based on the exchange of ideas and proposals amongthe participants during the last LogCLEF 2010 workshop:1. Language identification: participants are required to identify the actual languageof queries. Annotated resources manually generated by participants of previouseditions form a basic set of ground-truth data. This ground truth will be used toevaluate the automatic language recognition algorithms.2. Query classification: participants are required to annotate each query with a labelwhich represents a category of interest, e.g. PERSON (i.e. Leonardo Da Vinci) orEVENT (i.e. Revolucio´n francesa).3. Success of a query: participants are required to study the trend of the success ofa search. The success can be defined in terms of time spent on a page, number ofclicked items, or actions performed on the result list.4 Log DatasetsLogCLEF 2011 offers three datasets to participants.1. The European Library (TEL) dataset, consisting of three years and a half of logdata, with more than three million records of user interactions with the TEL por-tal. The TEL portal provides search across all national libraries and other digitallibraries in Europe. Naturally, the dataset is multilingual because TEL has usersfrom all European countries and thus, queries to the TEL portal cover all majorEuropean languages.2. The Sogou dataset, which contains queries to the Chinese Sogou search engine.The data contains a user ID, the query, URL in the result ranking, and user clickinformation. The queries are mostly in Chinese, but certain parts can be written inother languages. For example, proper nouns can be transcribed into English.3. A dataset from the German Education Server (Deutscher Bildungserver, DBS). TheDBS is a clearinghouse for educational resources on the Web. It also contains con-tent as well as descriptions and reviews on Web sites on education. The DBS weblogs are server logs in standard format in which the searches and the results viewedcan be observed. The logs allow to observe two types of user queries: queries insearch engines (in the referrer when DBS files were found using a search engine)and queries within the DBS. This data allows studying search behaviour on a singleweb server and in a specific domain.LogCLEF 2011 is one of the benchmarking activities of CLEF 2011 that will takeplace in Amsterdam in September 2011.AcknowledgementsThis work has been partially supported by the PROMISE network of excellence (con-tract n. 258191) project, as part of the 7th Framework Program of the European Com-mission, and by the Science Foundation of Ireland (grant 07/CE/I1142) as part of theCentre for Next Generation Localisation (http://www.cngl.ie).

Interactions between users and information access systems can be analyzed and studied to gather user preferences and to learn what a user likes the most, and to use this\ud
information to adapt the search to users and personalize the presentation of results. The LogCLEF lab - ”A benchmarking activity on Multilingual Log File Analysis: Language\ud
identification, query classification, success of a query” deals with information contained in query logs of search engines and digital libraries from which knowledge can be mined to understand search behavior in multilingual context. LogCLEF has created the first long-term standard collection for evaluation purposes in the area of log analysis. The LogCLEF 2011 lab is the continuation of the past two editions: as a pilot task in CLEF 2009, and a workshop in CLEF 2010. The Cross-Language Evaluation Forum (CLEF) promotes research and development in multilingual information access and is an activity of the PROMISE Network of Excellence

Name not available

http://doras.dcu.ie/16449/1/LogCLEF_Enabling_Research_on_Multilingual_Log_Files.pdf

LogCLEF: Enabling research on multilingual log files

Abstract

Similar works

Full text

Available Versions

Irish Universities

DCU Online Research Access Service

Name not available