Designing and evaluating the usability of a machine learning API for rapid prototyping music technology

To better support creative software developers and music technologists' needs, and to empower them as machine learning users and innovators, the usability of and developer experience with machine learning tools must be considered and better understood. We review background research on the design and evaluation of application programming interfaces (APIs), with a focus on the domain of machine learning for music technology software development. We present the design rationale for the RAPID-MIX API, an easy-to-use API for rapid prototyping with interactive machine learning, and a usability evaluation study with software developers of music technology. A cognitive dimensions questionnaire was designed and delivered to a group of 12 participants who used the RAPID-MIX API in their software projects, including people who developed systems for personal use and professionals developing software products for music and creative technology companies. The results from the questionnaire indicate that participants found the RAPID-MIX API a machine learning API which is easy to learn and use, fun, and good for rapid prototyping with interactive machine learning. Based on these findings, we present an analysis and characterization of the RAPID-MIX API based on the cognitive dimensions framework, and discuss its design trade-offs and usability issues. We use these insights and our design experience to provide design recommendations for ML APIs for rapid prototyping of music technology. We conclude with a summary of the main insights, a discussion of the merits and challenges of the application of the CDs framework to the evaluation of machine learning APIs, and directions to future work which our research deems valuable

Security, Privacy and Safety Risk Assessment for Virtual Reality Learning Environment Applications

Social Virtual Reality based Learning Environments (VRLEs) such as vSocial render instructional content in a three-dimensional immersive computer experience for training youth with learning impediments. There are limited prior works that explored attack vulnerability in VR technology, and hence there is a need for systematic frameworks to quantify risks corresponding to security, privacy, and safety (SPS) threats. The SPS threats can adversely impact the educational user experience and hinder delivery of VRLE content. In this paper, we propose a novel risk assessment framework that utilizes attack trees to calculate a risk score for varied VRLE threats with rate and duration of threats as inputs. We compare the impact of a well-constructed attack tree with an adhoc attack tree to study the trade-offs between overheads in managing attack trees, and the cost of risk mitigation when vulnerabilities are identified. We use a vSocial VRLE testbed in a case study to showcase the effectiveness of our framework and demonstrate how a suitable attack tree formalism can result in a more safer, privacy-preserving and secure VRLE system.Comment: Tp appear in the CCNC 2019 Conferenc

Evaluation of social personalized adaptive E-Learning environments : end-user point of view

The use of adaptations, along with the social affordances of collaboration and networking, carries a great potential for improving e-learning experiences. However, the review of the previous work indicates current e-learning systems have only marginally explored the integration of social features and adaptation techniques. The overall aim of this research, therefore, is to address this gap by evaluating a system developed to foster social personalized adaptive e-learning experiences. We have developed our first prototype system, Topolor, based on the concepts of Adaptive Educational Hypermedia and Social E-Learning. We have also conducted an experimental case study for the evaluation of the prototype system from different perspectives. The results show a considerably high satisfaction of the end users. This paper reports the evaluation results from end user point of view, and generalizes our method to a component-based evaluation framework

Enhancing the Digital Backchannel Backstage on the Basis of a Formative User Study

Contemporary higher education with its large audiences suffers from passivity of students. Enhancing the classroom with a digital backchannel can contribute to establishing and fostering active participation of and collaboration among students in the lecture. Therefore, we conceived the digital backchannel Backstage specifically tailored for the use in large classes. At an early phase of development we tested its core functionalities in a small-scale user study. The aim of the study was to gain first impressions of its adoption, and also to form a basis for further steps in the conception of Backstage. Regarding adoption we particularly focused on how Backstage influences the participants' questioning behavior, a salient aspect in learning. We observed that during the study much more questions were uttered on Backstage than being asked without backchannel support. Regarding the further development of Backstage we capitalized on the participants' usability feedback. The key of the refinement is the integration of presentation slides in Backstage, which leads to an interesting reconsideration of the user interactions of Backstage

Designing electronic collaborative learning environments

Electronic collaborative learning environments for learning and working are in vogue. Designers design them according to their own constructivist interpretations of what collaborative learning is and what it should achieve. Educators employ them with different educational approaches and in diverse situations to achieve different ends. Students use them, sometimes very enthusiastically, but often in a perfunctory way. Finally, researchers study them and—as is usually the case when apples and oranges are compared—find no conclusive evidence as to whether or not they work, where they do or do not work, when they do or do not work and, most importantly, why, they do or do not work. This contribution presents an affordance framework for such collaborative learning environments; an interaction design procedure for designing, developing, and implementing them; and an educational affordance approach to the use of tasks in those environments. It also presents the results of three projects dealing with these three issues

ZETA - Zero-Trust Authentication: Relying on Innate Human Ability, not Technology

Reliable authentication requires the devices and channels involved in the process to be trustworthy; otherwise authentication secrets can easily be compromised. Given the unceasing efforts of attackers worldwide such trustworthiness is increasingly not a given. A variety of technical solutions, such as utilising multiple devices/channels and verification protocols, has the potential to mitigate the threat of untrusted communications to a certain extent. Yet such technical solutions make two assumptions: (1) users have access to multiple devices and (2) attackers will not resort to hacking the human, using social engineering techniques. In this paper, we propose and explore the potential of using human-based computation instead of solely technical solutions to mitigate the threat of untrusted devices and channels. ZeTA (Zero Trust Authentication on untrusted channels) has the potential to allow people to authenticate despite compromised channels or communications and easily observed usage. Our contributions are threefold: (1) We propose the ZeTA protocol with a formal definition and security analysis that utilises semantics and human-based computation to ameliorate the problem of untrusted devices and channels. (2) We outline a security analysis to assess the envisaged performance of the proposed authentication protocol. (3) We report on a usability study that explores the viability of relying on human computation in this context

QoE Modelling, Measurement and Prediction: A Review

In mobile computing systems, users can access network services anywhere and anytime using mobile devices such as tablets and smart phones. These devices connect to the Internet via network or telecommunications operators. Users usually have some expectations about the services provided to them by different operators. Users' expectations along with additional factors such as cognitive and behavioural states, cost, and network quality of service (QoS) may determine their quality of experience (QoE). If users are not satisfied with their QoE, they may switch to different providers or may stop using a particular application or service. Thus, QoE measurement and prediction techniques may benefit users in availing personalized services from service providers. On the other hand, it can help service providers to achieve lower user-operator switchover. This paper presents a review of the state-the-art research in the area of QoE modelling, measurement and prediction. In particular, we investigate and discuss the strengths and shortcomings of existing techniques. Finally, we present future research directions for developing novel QoE measurement and prediction technique

'I play, therefore I learn?' Measuring the Evolution of Perceived Learning and Game Experience in the Design Flow of a Serious Game

This article explores how the serious game Poverty Is Not a Game (PING) is experienced by high school students in its subsequent design stages. We first focus on the multifaceted construct of game experience and how it is related to serious games. To measure game experience we use the Game Experience Questionnaire and add a perceived learning scale to account for the specificity of serious games in a classroom. Next, the data obtained from testing PING in 22 classrooms are analyzed. Results suggest that the evolution in the different design stages of the game is not just an issue of game experience, but also of usability. Furthermore, little evidence is found indicating that the learning experience changed positively during the different test phases. However, findings show a strong effect of the game experience on perceived learning while the game experience also varies significantly between different classrooms