Evaluation Measures for Relevance and Credibility in Ranked Lists

Recent discussions on alternative facts, fake news, and post truth politics have motivated research on creating technologies that allow people not only to access information, but also to assess the credibility of the information presented to them by information retrieval systems. Whereas technology is in place for filtering information according to relevance and/or credibility, no single measure currently exists for evaluating the accuracy or precision (and more generally effectiveness) of both the relevance and the credibility of retrieved results. One obvious way of doing so is to measure relevance and credibility effectiveness separately, and then consolidate the two measures into one. There at least two problems with such an approach: (I) it is not certain that the same criteria are applied to the evaluation of both relevance and credibility (and applying different criteria introduces bias to the evaluation); (II) many more and richer measures exist for assessing relevance effectiveness than for assessing credibility effectiveness (hence risking further bias). Motivated by the above, we present two novel types of evaluation measures that are designed to measure the effectiveness of both relevance and credibility in ranked lists of retrieval results. Experimental evaluation on a small human-annotated dataset (that we make freely available to the research community) shows that our measures are expressive and intuitive in their interpretation

Personality and Credibility Evaluation: Online Health Information from the Viewpoint of Expert Users

Regarding the high importance of credibility evaluation of health information online in every day and career life, the current research aimed to explore how expert users including students and faculty members evaluate health information on the web especially with regard to their personality trait of conscientiousness. By considering the extensive body of literature and research methodologies, a random sample including 148 students and faculty members located at the two faculties of Medicine Sciences and Nursing and Midwife at the Tehran University of Medical Sciences was the population of the study. Two validated questionnaires related to credibility and conscientiousness was distributed among the participants. Gathered data were then analyzed by SPSS software. Data analysis showed that there existed some problems in evaluating the credibility of online health information and quick retrieval. About half of the participants evaluate web health information as credible often and very often. Most important quality assessment criteria were objectivity, currency, and authority. There was a significant correlation between conscientiousness and credibility assessment of the students and faculty members. There appears a change in trust to information found accord with the change in conscientiousness levels. There was no significant correlation between conscientiousness and trust to information among ill people. Quality and credibility were identified as most important problems in assessing health information on the web. There is a significant and meaningful correlation between conscientiousness and trust or mistrust to health information available via the Web

Personality and Credibility Evaluation: Online Health Information from the Viewpoint of Expert Users

Regarding the high importance of credibility evaluation of health information online in every day and career life, the current research aimed to explore how expert users including students and faculty members evaluate health information on the web especially with regard to their personality trait of conscientiousness. By considering the extensive body of literature and research methodologies, a random sample including 148 students and faculty members located at the two faculties of Medicine Sciences and Nursing and Midwife at the Tehran University of Medical Sciences was the population of the study. Two validated questionnaires related to credibility and conscientiousness was distributed among the participants. Gathered data were then analyzed by SPSS software. Data analysis showed that there existed some problems in evaluating the credibility of online health information and quick retrieval. About half of the participants evaluate web health information as credible often and very often. Most important quality assessment criteria were objectivity, currency, and authority. There was a significant correlation between conscientiousness and credibility assessment of the students and faculty members. There appears a change in trust to information found accord with the change in conscientiousness levels. There was no significant correlation between conscientiousness and trust to information among ill people. Quality and credibility were identified as most important problems in assessing health information on the web. There is a significant and meaningful correlation between conscientiousness and trust or mistrust to health information available via the Web

Une approche basée agent pour la recherche de document numérique

The amount of information available on the Web is increasing in exponential way. This enormous abundance of information has created new challenges for users such as information retrieval, checking credibility of the information and content management ... etc. As for information retrieval, it became more difficult, because of the continual change of Web content: the emergence of new Web sites and pages, changing or deleting theirs contents, etc. In this context, search engines are available to users for helping them to completing their information retrieval process. Unfortunately, these search engines do not take into account the specific character of the user and they treat it as a generic user. Our objective in this work is to propose a document retrieval system based-agents whom focus on the user and takes into account his preferences, behavior and knowledge. Our system is composed of two parts: client and server. Software agents whom are in the client side interact with the user to gather information about his preferences and knowledge, and help him to formulate his need of information in a butter way. Software agents whom are in the server side will use the information collected by agents of the other side to serve the information needs of the user

HealthTrust: Assessing the Trustworthiness of Healthcare Information on the Internet

As well recognized, healthcare information is growing exponentially and is made more available to public. Frequent users such as medical professionals and patients are highly dependent on the web sources to get the appropriate information promptly. However, the trustworthiness of the information on the web is always questionable due to the fast and augmentative properties of the Internet. Most search engines provide relevant pages to given keywords, but the results might contain some unreliable or biased information. Consequently, a significant challenge associated with the information explosion is to ensure effective use of information. One way to improve the search results is by accurately identifying more trustworthy data. Surprisingly, although trustworthiness of sources is essential for a great number of daily users, not much work has been done for healthcare information sources by far. In this dissertation, I am proposing a new system named HealthTrust, which automatically assesses the trustworthiness of healthcare information over the Internet. In the first phase, an unsupervised clustering using graph topology, on our collection of data is employed. The goal is to identify a relatively larger and reliable set of trusted websites as a seed set without much human efforts. After that, a new ranking algorithm for structure-based assessment is adopted. The basic hypothesis is that trustworthy pages are more likely to link to trustworthy pages. In this way, the original set of positive and negative seeds will propagate over the Web graph. With the credibility-based discriminators, the global scoring is biased towards trusted websites and away from untrusted websites. Next, in the second phase, the content consistency between general healthcare-related webpages and trusted sites is evaluated using information retrieval techniques to evaluate the content-semantics of the webpage with respect to the medical topics. In addition, graph modeling is employed to generate contents-based ranking for each page based on the sentences in the seed pages. Finally, in order to integrate the two components, an iterative approach that integrates the credibility assessments from structure-based and content-based methods to give a final verdict - a HealthTrust score for each webpage is exploited. I demonstrated the first attempt to integrate structure-based and content-based approaches to automatically evaluate the credibility of online healthcare information through HealthTrust and make fundamental contributions to both information retrieval and healthcare informatics communities

New perspectives on Web search engine research

Purpose–The purpose of this chapter is to give an overview of the context of Web search and search engine-related research, as well as to introduce the reader to the sections and chapters of the book. Methodology/approach–We review literature dealing with various aspects of search engines, with special emphasis on emerging areas of Web searching, search engine evaluation going beyond traditional methods, and new perspectives on Webs earching. Findings–The approaches to studying Web search engines are manifold. Given the importance of Web search engines for knowledge acquisition, research from different perspectives needs to be integrated into a more cohesive perspective. Researchlimitations/implications–The chapter suggests a basis for research in the field and also introduces further research directions. Originality/valueofpaper–The chapter gives a concise overview of the topics dealt with in the book and also shows directions for researchers interested in Web search engines