Development and validation of an integrated model for evaluating e-service quality, usability and user experience (e-SQUUX) of Web-based applications in the context of a University web portal

Text in EnglishDevelopments in Internet technology and pervasive computing over the past two and half decades have resulted in a variety of Web-based applications (WBAs) that provide products and services to online users or customers. The Internet is used not only to transfer information via the web but is increasingly used to provide electronic services including business transactions, information-delivery and social networking, as well as e-government, e-health and e-learning. For such organisations, e-service quality, usability and user experience are considered to be critical determinants of their products’ or services’ success. Many studies to model these three concepts separately have been undertaken as part of broader studies of software quality or service quality modelling. However, to the current researcher’s knowledge, none of the studies have focussed on proposing an evaluation model that integrates and combines the three of them. This research is an effort to fill that gap. The primary purpose of this mixed-methods research was to develop a conceptual integrated model for evaluating e-service quality, usability and user experience (e-SQUUX) of WBAs and then contextualise it to evaluation of a University web portal (UWP). This was undertaken using an exploratory sequential research design. During a qualitative phase, an extensive extensive systematic literature review of 264 relevant sources relating to dimensions of e-service quality, usability and user experience, was undertaken to derive an integrated conceptual e-service quality, usability and user experience (e-SQUUX) Model for evaluating WBAs. The model was then empirically refined through a sequential series of validations, thus developing various versions of the e-SQUUX Model. First, it was content validated by a set of four expert reviewers. Second, during the quantitative phase, in the context of a University web portal, a questionnaire survey was conducted that included a comprehensive pilot study with 29 partipants, prior to the main survey. The main survey data from 174 particiapants was used to determine a validated model, using Exploratory factor analysis (EFA), followed by producing a structural model, using partial least square – structural equation modelling (PLS-SEM). This version consisted of the components of the final e-SQUUX Model. Consequently, the research enriches the body of knowledge on IS and HCI by providing the e-SQUUX Model as an evaluation tool. For designers, developers and managers of UWPs, the model serves as a customisable set of evaluation criteria and also provides specific recommendations for design. In line with the Exploratory sequential design of mixed methods research, the findings of the qualitative work in this research influenced the subsequent quantitative study, since the potential Likert-scale questionnaire items were derived from the definitions and meanings of the components that emanated from the qualitative phase of the study. Consequently, this research is an exemplar for developing an integrated evaluation model for specific facets or domains, and of its application in a particular context, in this case, a University web portal. Keywords: e-service quality, usability, user experience, evaluation model, integrated model, exploratory factor analysis, partial least square – structural equation modelling (PLS-SEM), mixed methods research, Exploratory sequential design, quantitative study, qualitative study, validation, Web-based applications, University web portalInformation SystemPh D. (Information Systems

An intelligent user interface model for contact centre operations

Contact Centres (CCs) are at the forefront of interaction between an organisation and its customers. Currently, 17 percent of all inbound calls are not resolved on the first call by the first agent attending to that call. This is due to the inability of the contact centre agents (CCAs) to diagnose customer queries and find adequate solutions in an effective and efficient manner. The aim of this research is to develop an intelligent user interface (IUI) model to support and improve CC operations. A literature review of existing IUI architectures, modelbased design and existing CC software together with a field study of CCs has resulted in the design of an IUI model for CCs. The proposed IUI model is described in terms of its architecture, component-level design and interface design. An IUI prototype has been developed as a proof of concept of the proposed IUI model. The IUI prototype was evaluated in order to determine to what extent it supports problem identification and query resolution. User testing, incorporating the use of eye tracking and a post-test questionnaire, was used in order to determine the usability and usefulness of the prototype. The results of this evaluation show that the users were highly satisfied with the task support and query resolution assistance provided by the IUI prototype. This research resulted in the design of an IUI model for the domain of CCs. This model can be used to assist the development of CC applications incorporating IUIs. Use of the proposed IUI model is expected to support and enhance the effectiveness and efficiency of CC operations. Further research is needed to conduct a longitudinal study to determine the impact of IUIs in the CC domain

Automated analysis of free-text comments and dashboard representations in patient experience surveys: a multimethod co-design study

BACKGROUND: Patient experience surveys (PESs) often include informative free-text comments, but with no way of systematically, efficiently and usefully analysing and reporting these. The National Cancer Patient Experience Survey (CPES), used to model the approach reported here, generates > 70,000 free-text comments annually. MAIN AIM: To improve the use and usefulness of PES free-text comments in driving health service changes that improve the patient experience. SECONDARY AIMS: (1) To structure CPES free-text comments using rule-based information retrieval (IR) (‘text engineering’), drawing on health-care domain-specific gazetteers of terms, with in-built transferability to other surveys and conditions; (2) to display the results usefully for health-care professionals, in a digital toolkit dashboard display that drills down to the original free text; (3) to explore the usefulness of interdisciplinary mixed stakeholder co-design and consensus-forming approaches in technology development, ensuring that outputs have meaning for all; and (4) to explore the usefulness of Normalisation Process Theory (NPT) in structuring outputs for implementation and sustainability. DESIGN: A scoping review, rapid review and surveys with stakeholders in health care (patients, carers, health-care providers, commissioners, policy-makers and charities) explored clinical dashboard design/patient experience themes. The findings informed the rules for the draft rule-based IR [developed using half of the 2013 Wales CPES (WCPES) data set] and prototype toolkit dashboards summarising PES data. These were refined following mixed stakeholder, concept-mapping workshops and interviews, which were structured to enable consensus-forming ‘co-design’ work. IR validation used the second half of the WCPES, with comparison against its manual analysis; transferability was tested using further health-care data sets. A discrete choice experiment (DCE) explored which toolkit features were preferred by health-care professionals, with a simple cost–benefit analysis. Structured walk-throughs with NHS managers in Wessex, London and Leeds explored usability and general implementation into practice. KEY OUTCOMES: A taxonomy of ranked PES themes, a checklist of key features recommended for digital clinical toolkits, rule-based IR validation and transferability scores, usability, and goal-oriented, cost–benefit and marketability results. The secondary outputs were a survey, scoping and rapid review findings, and concordance and discordance between stakeholders and methods. RESULTS: (1) The surveys, rapid review and workshops showed that stakeholders differed in their understandings of the patient experience and priorities for change, but that they reached consensus on a shortlist of 19 themes; six were considered to be core; (2) the scoping review and one survey explored the clinical toolkit design, emphasising that such toolkits should be quick and easy to use, and embedded in workflows; the workshop discussions, the DCE and the walk-throughs confirmed this and foregrounded other features to form the toolkit design checklist; and (3) the rule-based IR, developed using noun and verb phrases and lookup gazetteers, was 86% accurate on the WCPES, but needs modification to improve this and to be accurate with other data sets. The DCE and the walk-through suggest that the toolkit would be well accepted, with a favourable cost–benefit ratio, if implemented into practice with appropriate infrastructure support. LIMITATIONS: Small participant numbers and sampling bias across component studies. The scoping review studies mostly used top-down approaches and focused on professional dashboards. The rapid review of themes had limited scope, with no second reviewer. The IR needs further refinement, especially for transferability. New governance restrictions further limit immediate use. CONCLUSIONS: Using a multidisciplinary, mixed stakeholder, use of co-design, proof of concept was shown for an automated display of patient experience free-text comments in a way that could drive health-care improvements in real time. The approach is easily modified for transferable application. FUTURE WORK: Further exploration is needed of implementation into practice, transferable uses and technology development co-design approaches. FUNDING: The National Institute for Health Research Health Services and Delivery Research programme

Multicentric investigation on the safety, feasibility and usability of the ABLE lower-limb robotic exoskeleton for individuals with spinal cord injury : a framework towards the standardisation of clinical evaluations

Robotic lower-limb exoskeletons have the potential to provide additional clinical benefits for persons with spinal cord injury (SCI). However, high variability between protocols does not allow the comparison of study results on safety and feasibility between different exoskeletons. We therefore incorporated key aspects from previous studies into our study protocol and accordingly conducted a multicentre study investigating the safety, feasibility and usability of the ABLE Exoskeleton in clinical settings. In this prospective pretest-posttest quasi-experimental study across two SCI centres in Germany and Spain, in- and outpatients with SCI were recruited into a 12-session training and assessment protocol, utilising the ABLE Exoskeleton. A follow-up visit after 4 weeks was included to assess after-training outcomes. Safety outcomes (device-related adverse events (AEs), number of drop-outs), feasibility and usability measures (level of assistance, donning/doffing-time) were recorded at every session together with changes in gait parameters and function. Patient-reported outcome measures including the rate of perceived exertion (RPE) and the psychosocial impact of the device were performed. Satisfaction with the device was evaluated in both participants and therapists. All 24 participants (45 ± 12 years), with mainly subacute SCI (< 1 year after injury) from C5 to L3, (ASIA Impairment Scale A to D) completed the follow-up. In 242 training sessions, 8 device-related AEs (pain and skin lesions) were reported. Total time for don and doff was 6:50 ± 2:50 min. Improvements in level of assistance and gait parameters (time, steps, distance and speed, p < 0.05) were observed in all participants. Walking function and RPE improved in participants able to complete walking tests with (n = 9) and without (n = 6) the device at study start (p < 0.05). A positive psychosocial impact of the exoskeleton was reported and the satisfaction with the device was good, with best ratings in safety (participants), weight (therapists), durability and dimensions (both). Our study results prove the feasibility of safe gait training with the ABLE Exoskeleton in hospital settings for persons with SCI, with improved clinical outcomes after training. Our study protocol allowed for consistent comparison of the results with other exoskeleton trials and can serve as a future framework towards the standardisation of early clinical evaluations. Trial Registration , DRKS00023503, retrospectively registered on November 18, 2020. The online version contains supplementary material available at 10.1186/s12984-023-01165-0

A Crowdsourced Library Help System

Help information regarding library resources and tools is a critical part of library services, but existing help channels are often fragmented and under-maintained. Library help needs to have a centralized and engaging platform for users to receive assistance and experts including librarians, faculty, and students to share knowledge and experience. To meet this critical need, we have developed CrowdAsk, an open-source help system for academic libraries based on the concept of crowdsourcing. CrowdAsk supports voting of questions and answers by users as well as scores and badges for user motivation. Our implementation and evaluation with undergraduate classes at Purdue University showed that CrowdAsk is effective in meeting users’ information needs beyond traditional library reference help. Users have asked various types of questions and reached high levels of scores and badges in one semester’s time. We have integrated CrowdAsk with existing reference service and websites in Purdue University Libraries to continue developing the crowdsourcing community after the grant

A Benchmark for Image Retrieval using Distributed Systems over the Internet: BIRDS-I

The performance of CBIR algorithms is usually measured on an isolated workstation. In a real-world environment the algorithms would only constitute a minor component among the many interacting components. The Internet dramati-cally changes many of the usual assumptions about measuring CBIR performance. Any CBIR benchmark should be designed from a networked systems standpoint. These benchmarks typically introduce communication overhead because the real systems they model are distributed applications. We present our implementation of a client/server benchmark called BIRDS-I to measure image retrieval performance over the Internet. It has been designed with the trend toward the use of small personalized wireless systems in mind. Web-based CBIR implies the use of heteroge-neous image sets, imposing certain constraints on how the images are organized and the type of performance metrics applicable. BIRDS-I only requires controlled human intervention for the compilation of the image collection and none for the generation of ground truth in the measurement of retrieval accuracy. Benchmark image collections need to be evolved incrementally toward the storage of millions of images and that scaleup can only be achieved through the use of computer-aided compilation. Finally, our scoring metric introduces a tightly optimized image-ranking window.Comment: 24 pages, To appear in the Proc. SPIE Internet Imaging Conference 200

Technical Metrics Used to Evaluate Health Care Chatbots: Scoping Review

Dialog agents (chatbots) have a long history of application in health care, where they have been used for tasks such as supporting patient self-management and providing counseling. Their use is expected to grow with increasing demands on health systems and improving artificial intelligence (AI) capability. Approaches to the evaluation of health care chatbots, however, appear to be diverse and haphazard, resulting in a potential barrier to the advancement of the field. This study aims to identify the technical (nonclinical) metrics used by previous studies to evaluate health care chatbots. Studies were identified by searching 7 bibliographic databases (eg, MEDLINE and PsycINFO) in addition to conducting backward and forward reference list checking of the included studies and relevant reviews. The studies were independently selected by two reviewers who then extracted data from the included studies. Extracted data were synthesized narratively by grouping the identified metrics into categories based on the aspect of chatbots that the metrics evaluated. Of the 1498 citations retrieved, 65 studies were included in this review. Chatbots were evaluated using 27 technical metrics, which were related to chatbots as a whole (eg, usability, classifier performance, speed), response generation (eg, comprehensibility, realism, repetitiveness), response understanding (eg, chatbot understanding as assessed by users, word error rate, concept error rate), and esthetics (eg, appearance of the virtual agent, background color, and content). The technical metrics of health chatbot studies were diverse, with survey designs and global usability metrics dominating. The lack of standardization and paucity of objective measures make it difficult to compare the performance of health chatbots and could inhibit advancement of the field. We suggest that researchers more frequently include metrics computed from conversation logs. In addition, we recommend the development of a framework of technical metrics with recommendations for specific circumstances for their inclusion in chatbot studies

Using Scrum in a side project with distributed teams

Scrum is the most popular agile software development methodology in use today (VersionOne, 2013), (West & Grant, 2010), (Begel & Nagappan, 2007). Many of the Scrum teams work in virtual distributed formations (VersionOne, 2013). The distributed work model introduces multiple burdens, like communication issues, matter of trust, time zones and cultural differences to the teams, which can severely affect performance (Deemer, s.a.). As a result, dislocated Scrum teams have a considerably lower success rate than collocated ones according to various surveys (Ambler S. W., 2008), (Ambler S. W., Agility at Scale Survey 2012, 2012), (Cohn, 2010). This paper researches the topic of “Using Scrum in a side project with distributed teams” and consist of three main parts: theoretical background, methods and results. In the theoretical background we cover the different software development methodologies, agile methodology in particular. We take a deeper look into Scrum; it's history, roles in Scrum, Scrum artifacts and ceremonies. We also briefly discuss virtual teams theory. In the methods we cover the online contract bridge card game development, Lean Startup and Business Model Generation as part of business development, background info on contract bridge, validating the idea, the design and development tools used to build the minimum viable product and the methods of data collection for Scrum. In the results we describe our implementation of Scrum, the building of the minimum viable product, and research made into other early stage startup teams to support our own findings.http://tartu.ester.ee/record=b2656166~S1*es

Access Control In and For the Real World

Access control is a core component of any information-security strategy. Researchers have spent tremendous energy over the past forty years defining abstract access-control models and proving various properties about them. However, surprisingly little attention has been paid to how well these models work in real socio-technical systems (i.e., real human organizations). This dissertation describes the results of two qualitative studies (involving 52 participants from four companies, drawn from the financial, software, and healthcare sectors) and observes that the current practice of access control is dysfunctional at best. It diagnoses the broken assumptions that are at the heart of this dysfunction, and offers a new definition of the access-control problem that is grounded in the requirements and limitations of the real world