Metamorphic testing: testing the untestable

What if we could know that a program is buggy, even if we could not tell whether or not its observed output is correct? This is one of the key strengths of metamorphic testing, a technique where failures are not revealed by checking an individual concrete output, but by checking the relations among the inputs and outputs of multiple executions of the program under test. Two decades after its introduction, metamorphic testing has become a fully-fledged testing technique with successful applications in multiple domains, including online search engines, autonomous machinery, compilers, Web APIs, and deep learning programs, among others. This article serves as a hands-on entry point for newcomers to metamorphic testing, describing examples, possible applications, and current limitations, providing readers with the basics for the application of the technique in their own projects. IEE

Development of the Web Users Self-Efficacy scale (WUSE)

The aim of this research was to develop a scale that could evaluate an individuals confidence in using the Internet. Web-based resources are becoming increasingly important within higher education and it is therefore vital that students and staff feel confident and competent in the access, provision, and utilisation of these resources. The scale developed here represents an extension of previous research (Cassidy & Eachus, 2002) that developed a measure of self-efficacy in the context of computer use. An iterative approach was used in the development of the Web User Self-Efficacy scale (WUSE) and the participants were recruited from the student body of a large University in the North West of the United Kingdom, and globally via a web site set up for this purpose. Initial findings suggest that the scale has acceptable standards of reliability and validity though work is continuing to refine the scale and improve the psychometric properties of the tool

A Machine Learning Based Analytical Framework for Semantic Annotation Requirements

The Semantic Web is an extension of the current web in which information is given well-defined meaning. The perspective of Semantic Web is to promote the quality and intelligence of the current web by changing its contents into machine understandable form. Therefore, semantic level information is one of the cornerstones of the Semantic Web. The process of adding semantic metadata to web resources is called Semantic Annotation. There are many obstacles against the Semantic Annotation, such as multilinguality, scalability, and issues which are related to diversity and inconsistency in content of different web pages. Due to the wide range of domains and the dynamic environments that the Semantic Annotation systems must be performed on, the problem of automating annotation process is one of the significant challenges in this domain. To overcome this problem, different machine learning approaches such as supervised learning, unsupervised learning and more recent ones like, semi-supervised learning and active learning have been utilized. In this paper we present an inclusive layered classification of Semantic Annotation challenges and discuss the most important issues in this field. Also, we review and analyze machine learning applications for solving semantic annotation problems. For this goal, the article tries to closely study and categorize related researches for better understanding and to reach a framework that can map machine learning techniques into the Semantic Annotation challenges and requirements

Information Extraction in Illicit Domains

Extracting useful entities and attribute values from illicit domains such as human trafficking is a challenging problem with the potential for widespread social impact. Such domains employ atypical language models, have `long tails' and suffer from the problem of concept drift. In this paper, we propose a lightweight, feature-agnostic Information Extraction (IE) paradigm specifically designed for such domains. Our approach uses raw, unlabeled text from an initial corpus, and a few (12-120) seed annotations per domain-specific attribute, to learn robust IE models for unobserved pages and websites. Empirically, we demonstrate that our approach can outperform feature-centric Conditional Random Field baselines by over 18\% F-Measure on five annotated sets of real-world human trafficking datasets in both low-supervision and high-supervision settings. We also show that our approach is demonstrably robust to concept drift, and can be efficiently bootstrapped even in a serial computing environment.Comment: 10 pages, ACM WWW 201

The unseen and unacceptable face of digital libraries

The social and organisational aspects of digital libraries (DLs) are often overlooked, but this paper reviews how they can affect users' awareness and acceptance of DLs. An analysis of research conducted within two contrasting domains (clinical and academic) is presented which highlights issues of user interactions, work practices and organisational social structures. The combined study comprises an analysis of 98 in-depth interviews and focus groups with lecturers, librarians and hospital clinicians. The importance of current and past roles of the library, and how users interacted with it, are revealed. Web-based DLs, while alleviating most library resource and interaction problems, require a change in librarians' and DL designers' roles and interaction patterns if they are to be implemented acceptably and effectively. Without this role change, users will at best be unaware of these digital resources and at worst feel threatened by them. The findings of this paper highlight the importance of DL design and implementation of the social context and supporting user communication (i.e., collaboration and consultation) in information searching and usage activities. © Springer-Verlag 2004

Invisible Pixels Are Dead, Long Live Invisible Pixels!

Privacy has deteriorated in the world wide web ever since the 1990s. The tracking of browsing habits by different third-parties has been at the center of this deterioration. Web cookies and so-called web beacons have been the classical ways to implement third-party tracking. Due to the introduction of more sophisticated technical tracking solutions and other fundamental transformations, the use of classical image-based web beacons might be expected to have lost their appeal. According to a sample of over thirty thousand images collected from popular websites, this paper shows that such an assumption is a fallacy: classical 1 x 1 images are still commonly used for third-party tracking in the contemporary world wide web. While it seems that ad-blockers are unable to fully block these classical image-based tracking beacons, the paper further demonstrates that even limited information can be used to accurately classify the third-party 1 x 1 images from other images. An average classification accuracy of 0.956 is reached in the empirical experiment. With these results the paper contributes to the ongoing attempts to better understand the lack of privacy in the world wide web, and the means by which the situation might be eventually improved.Comment: Forthcoming in the 17th Workshop on Privacy in the Electronic Society (WPES 2018), Toronto, AC

Intelligent Tutoring System Authoring Tools for Non-Programmers

An intelligent tutoring system (ITS) is a software application that tries to replicate the performance of a human tutor by supporting the theory of learning by doing . ITSs have been shown to improve the performance of a student in wide range of domains. Despite their benefits, ITSs have not seen widespread use due to the complexity involved in their development. Developing an ITS from scratch requires expertise in several fields including computer science, cognitive psychology and artificial intelligence. In order to decrease the skill threshold required to build ITSs, several authoring tools have been developed. In this thesis, I document several contributions to the field of intelligent tutoring in the form of extensions to an existing ITS authoring tool, research studies on authoring tool paradigms and the design of authoring tools for non-programmers in two complex domains - natural language processing and 3D game environments. The Extensible Problem Specific Tutor (xPST) is an authoring tool that helps rapidly develop model-tracing like tutors on existing interfaces such as webpages. xPST\u27s language was made more expressive with the introduction of new checktypes required for answer checking in problems belonging to domains such as geometry and statistics. A web-based authoring (WAT) tool was developed for the purpose of tutor management and deployment and to promote non-programmer authoring of ITSs. The WAT was used in a comparison study between two authoring tool paradigms - GUI based and text based, in two different problem domains - statistics and geometry. User-programming of natural language processing (NLP) in ITSs is not common with authoring toolkits. Existing NLP techniques do not offer sufficient power to non-programmers and the NLP is left to expert developers or machine learning algorithms. We attempted to address this challenge by developing a domain-independent authoring tool, ConceptGrid that is intended to help non-programmers develop ITSs that perform natural language processing. ConceptGrid has been integrated into xPST. When templates created using ConceptGrid were tested, they approached the accuracy of human instructors in scoring student responses. 3D game environments belong to another domain for which authoring tools are uncommon. Authoring game-based tutors is challenging due to the inherent domain complexity and dynamic nature of the environment. We attempt to address this challenge through the design of authoring tool that is intended to help non-programmers develop game-based ITSs

Learning with Augmented Features for Heterogeneous Domain Adaptation

We propose a new learning method for heterogeneous domain adaptation (HDA), in which the data from the source domain and the target domain are represented by heterogeneous features with different dimensions. Using two different projection matrices, we first transform the data from two domains into a common subspace in order to measure the similarity between the data from two domains. We then propose two new feature mapping functions to augment the transformed data with their original features and zeros. The existing learning methods (e.g., SVM and SVR) can be readily incorporated with our newly proposed augmented feature representations to effectively utilize the data from both domains for HDA. Using the hinge loss function in SVM as an example, we introduce the detailed objective function in our method called Heterogeneous Feature Augmentation (HFA) for a linear case and also describe its kernelization in order to efficiently cope with the data with very high dimensions. Moreover, we also develop an alternating optimization algorithm to effectively solve the nontrivial optimization problem in our HFA method. Comprehensive experiments on two benchmark datasets clearly demonstrate that HFA outperforms the existing HDA methods.Comment: ICML201