Enhancing decision-making in user-centered web development: a methodology for card-sorting analysis

The World Wide Web has become a common platform for interactive software development. Most web applications feature custom user interfaces used by millions of people every day. Information architecture addresses the structural design of information to build quality web applications with improved usability of content, navigation, and findability. One of the most frequently utilized information architecture methods is card sorting—an affordable, user-centered approach for eliciting and evaluating categories and navigable items. Card sorting facilitates decision-making during the development process based on users’ mental models of a given application domain. However, although the qualitative analysis of card sorts has become common practice in information architecture, the quantitative analysis of card sorting is less widely applied. The reason for this gap is that quantitative analysis often requires the use of customized techniques to extract meaningful information for decision-making. To facilitate this process and support the structuring of information, we propose a methodology for the quantitative analysis of card-sorting results in this paper. The suggested approach can be systematically applied to provide clues and support for decisions. These might significantly impact the design and, thus, the final quality of the web application. Therefore, the approach includes proper goodness values that enable comparisons among the results of the methods and techniques used and ensure the suitability of the analyses performed. Two publicly available datasets were used to demonstrate the key issues related to the interpretation of card sorting results and the overall suitability and validity of the proposed methodologyThis work was partially supported by the Spanish Government [grant number RTI2018-095255-B-I00]; and the Madrid Research Council [Grant Number P2018/TCS-4314

Usability engineering for GIS: learning from a screenshot

In this paper, the focus is on the concept of Usability Engineering for GIS – a set of techniques and methods that are especially suitable for evaluating the usability of GIS applications – which can be deployed as part of the development process. To demonstrate how the framework of Usability Engineering for GIS can be used in reality, a screenshot study is described. Users were asked to provide a screenshot of their GIS during their working day. The study shows how a simple technique can help in understanding the way GIS is used in situ

Issues in Evaluating Health Department Web-Based Data Query Systems: Working Papers

Compiles papers on conceptual and methodological topics to consider in evaluating state health department systems that provide aggregate data online, such as taxonomy, logic models, indicators, and design. Includes surveys and examples of evaluations

Learner-Centred Design and Evaluation of Web-Based E-Learning Environments

Designing E-learning is a combination of pedagogical design, usability and information architecture. E-learning environments should have intuitive interfaces and clear information design, allowing learners to focus on learning. However, there is often a mismatch between what an on-line educator thinks the learner would learn, and what a learner thinks he will, and then has learned from the course. In addition, there is sometimes a mismatch between how an educator wants to teach and what is represented on the interface by the instructional designers. Such mismatches affect the learner's experience and his motivation for E-learning. In this paper, we will first discuss the source and nature of these mismatches. Next, we will discuss whether usability techniques in the HCI literature are appropriate for evaluating E-learning environments for the learner experience. We will then propose a combination of requirements elicitation and usability techniques for learner-centred design and evaluation of Web-based E-learning environments. The proposed methodology is based on our experience of conducting empirical studies for evaluating user-system interactions in E-Commerce contexts

Layered evaluation of interactive adaptive systems : framework and formative methods

Peer reviewedPostprin

What is Usability? A Characterization based on ISO 9241-11 and ISO/IEC 25010

According to Brooke* "Usability does not exist in any absolute sense; it can only be defined with reference to particular contexts." That is, one cannot speak of usability without specifying what that particular usability is characterized by. Driven by the feedback of a reviewer at an international conference, I explore in which way one can precisely specify the kind of usability they are investigating in a given setting. Finally, I come up with a formalism that defines usability as a quintuple comprising the elements level of usability metrics, product, users, goals and context of use. Providing concrete values for these elements then constitutes the investigated type of usability. The use of this formalism is demonstrated in two case studies. * J. Brooke. SUS: A "quick and dirty" usability scale. In P. W. Jordan, B. Thomas, B. A. Weerdmeester, and A. L. McClelland, editors, Usability Evaluation in Industry. Taylor and Francis, 1996.Comment: Technical Report; Department of Computer Science, Technische Universit\"at Chemnitz; also available from https://www.tu-chemnitz.de/informatik/service/ib/2015.php.e

Determination and evaluation of web accessibility

The Web is the most pervasive collaborative technology in widespread use today; however, access to the web and its many applications cannot be taken for granted. Web accessibility encompasses a variety of concerns ranging from societal, political, and economic to individual, physical, and intellectual through to the purely technical. Thus, there are many perspectives from which web accessibility can be understood and evaluated. In order to discuss these concerns and to gain a better understanding of web accessibility, an accessibility framework is proposed using as its base a layered evaluation framework from Computer Supported Co-operative Work research and the ISO standard, ISO/IEC 9126 on software quality. The former is employed in recognition of the collaborative nature of the web and its importance in facilitating communication. The latter is employed to refine and extend the technical issues and to highlight the need for considering accessibility from the viewpoint of the web developer and maintainer as well as the web user. A technically inaccessible web is unlikely to be evolved over time. A final goal of the accessibility framework is to provide web developers and maintainers with a practical basis for considering web accessibility through the development of a set of accessibility factors associated with each identified layer

Designing and evaluating mobile multimedia user experiences in public urban places: Making sense of the field

The majority of the world’s population now lives in cities (United Nations, 2008) resulting in an urban densification requiring people to live in closer proximity and share urban infrastructure such as streets, public transport, and parks within cities. However, “physical closeness does not mean social closeness” (Wellman, 2001, p. 234). Whereas it is a common practice to greet and chat with people you cross paths with in smaller villages, urban life is mainly anonymous and does not automatically come with a sense of community per se. Wellman (2001, p. 228) defines community “as networks of interpersonal ties that provide sociability, support, information, a sense of belonging and social identity.” While on the move or during leisure time, urban dwellers use their interactive information communication technology (ICT) devices to connect to their spatially distributed community while in an anonymous space. Putnam (1995) argues that available technology privatises and individualises the leisure time of urban dwellers. Furthermore, ICT is sometimes used to build a “cocoon” while in public to avoid direct contact with collocated people (Mainwaring et al., 2005; Bassoli et al., 2007; Crawford, 2008). Instead of using ICT devices to seclude oneself from the surrounding urban environment and the collocated people within, such devices could also be utilised to engage urban dwellers more with the urban environment and the urban dwellers within. Urban sociologists found that “what attracts people most, it would appear, is other people” (Whyte, 1980, p. 19) and “people and human activity are the greatest object of attention and interest” (Gehl, 1987, p. 31). On the other hand, sociologist Erving Goffman describes the concept of civil inattention, acknowledging strangers’ presence while in public but not interacting with them (Goffman, 1966). With this in mind, it appears that there is a contradiction between how people are using ICT in urban public places and for what reasons and how people use public urban places and how they behave and react to other collocated people. On the other hand there is an opportunity to employ ICT to create and influence experiences of people collocated in public urban places. The widespread use of location aware mobile devices equipped with Internet access is creating networked localities, a digital layer of geo-coded information on top of the physical world (Gordon & de Souza e Silva, 2011). Foursquare.com is an example of a location based 118 Mobile Multimedia – User and Technology Perspectives social network (LBSN) that enables urban dwellers to virtually check-in into places at which they are physically present in an urban space. Users compete over ‘mayorships’ of places with Foursquare friends as well as strangers and can share recommendations about the space. The research field of Urban Informatics is interested in these kinds of digital urban multimedia augmentations and how such augmentations, mediated through technology, can create or influence the UX of public urban places. “Urban informatics is the study, design, and practice of urban experiences across different urban contexts that are created by new opportunities of real-time, ubiquitous technology and the augmentation that mediates the physical and digital layers of people networks and urban infrastructures” (Foth et al., 2011, p. 4). One possibility to augment the urban space is to enable citizens to digitally interact with spaces and urban dwellers collocated in the past, present, and future. “Adding digital layer to the existing physical and social layers could facilitate new forms of interaction that reshape urban life” (Kjeldskov & Paay, 2006, p. 60). This methodological chapter investigates how the design of UX through such digital placebased mobile multimedia augmentations can be guided and evaluated. First, we describe three different applications that aim to create and influence the urban UX through mobile mediated interactions. Based on a review of literature, we describe how our integrated framework for designing and evaluating urban informatics experiences has been constructed. We conclude the chapter with a reflective discussion on the proposed framework

Records management capacity and compliance toolkits : a critical assessment.

This article seeks to present the results of a project that critically evaluated a series of toolkits for assessing records management capacity and/or compliance. These toolkits have been developed in different countries and sectors within the context of the e-environment and provide evidence of good corporate and information governance. Design/methodology/approach - A desk-based investigation of the tools was followed by an electronic Delphi with toolkit developers and performance measurement experts to develop a set of evaluation criteria. Different stakeholders then evaluated the toolkits against the criteria using cognitive walkthroughs and expert heuristic reviews. The results and the research process were reviewed via electronic discussion. Findings - Developed by recognised and highly respected organisations, three of the toolkits are software tools, whilst the fourth is a methodology. They are all underpinned by relevant national/international records management legislation, standards and good practice including, either implicitly or explicitly, ISO 15489. They all have strengths, complementing rather than competing with one another. They enable the involvement of other staff, thereby providing an opportunity for raising awareness of the importance of effective records management. Practical implications - These toolkits are potentially very powerful, flexible and of real value to organisations in managing their records. They can be used for a "quick and dirty" assessment of records management capacity or compliance as well as in-depth analysis. The most important criterion for selecting the appropriate one is to match the toolkit with the scenario. Originality/value - This paper aims to raise awareness of the range and nature of records management toolkits and their potential for varied use in practice to support more effective management of records