Increasing the credibility of scientific dissemination using crowdsourcing

Abstract. This thesis introduces Article Enhancer, a semi-automated web application that utilizes crowdsourcing services, specifically Amazon’s Mechanical Turk platform, for augmenting articles with various referencing content gathered from the crowd-workers, on demand. The main goal of Article Enhancer is to address the question of how scientific articles can be made more credible, before dissemination to the public. This application serves as a tool in helping users find suitable supporting content for their articles in a novel way, removing all the manual work of doing it themselves. Media literacy, social media, fake news and crowdsourcing are discussed as part of related work. Also, tools that offer a similar functionality are reviewed. Furthermore, system design and implementation for Article Enhancer is presented. It is important to mention that the referencing content provided through Article Enhancer comes from already existing online content. Although Article Enhancer is semi-automated system, its strongest point compared to the other systems, is that it doesn’t require extra human effort to enrich articles especially with visualization content, and providing already existing content on the web avoiding the process of creating new content, making it a fresh approach in this line of software service. To evaluate Article Enhancer, we deployed the web app in a real-life setting, a space oriented towards students known as Tellus, at the University of Oulu. This testing proceedings helped in determining that the system appears alluring and attractive to new users. Article Enhancer proved to be unique and thrilling after the first encounter for many of the users. Feedback also shows that adding and embedding content is an innovative way to make articles become more credible in the eye of the reader

Explainable Deep Classification Models for Domain Generalization

Conventionally, AI models are thought to trade off explainability for lower accuracy. We develop a training strategy that not only leads to a more explainable AI system for object classification, but as a consequence, suffers no perceptible accuracy degradation. Explanations are defined as regions of visual evidence upon which a deep classification network makes a decision. This is represented in the form of a saliency map conveying how much each pixel contributed to the network's decision. Our training strategy enforces a periodic saliency-based feedback to encourage the model to focus on the image regions that directly correspond to the ground-truth object. We quantify explainability using an automated metric, and using human judgement. We propose explainability as a means for bridging the visual-semantic gap between different domains where model explanations are used as a means of disentagling domain specific information from otherwise relevant features. We demonstrate that this leads to improved generalization to new domains without hindering performance on the original domain

Crowdsourced intuitive visual design feedback

For many people images are a medium preferable to text and yet, with the exception of star ratings, most formats for conventional computer mediated feedback focus on text. This thesis develops a new method of crowd feedback for designers based on images. Visual summaries are generated from a crowd’s feedback images chosen in response to a design. The summaries provide the designer with impressionistic and inspiring visual feedback. The thesis sets out the motivation for this new method, describes the development of perceptually organised image sets and a summarisation algorithm to implement it. Evaluation studies are reported which, through a mixed methods approach, provide evidence of the validity and potential of the new image-based feedback method. It is concluded that the visual feedback method would be more appealing than text for that section of the population who may be of a visual cognitive style. Indeed the evaluation studies are evidence that such users believe images are as good as text when communicating their emotional reaction about a design. Designer participants reported being inspired by the visual feedback where, comparably, they were not inspired by text. They also reported that the feedback can represent the perceived mood in their designs, and that they would be enthusiastic users of a service offering this new form of visual design feedback