Using the ISO/IEC 9126 product quality model to classify defects : a Controlled Experiment

Background: Existing software defect classification schemes support multiple tasks, such as root cause analysis and process improvement guidance. However, existing schemes do not assist in assigning defects to a broad range of high level software goals, such as software quality characteristics like functionality, maintainability, and usability. Aim: We investigate whether a classification based on the ISO/IEC 9126 software product quality model is reliable and useful to link defects to quality aspects impacted. Method: Six different subjects, divided in two groups with respect to their expertise, classified 78 defects from an industrial web application using the ISO/IEC 9126 quality main characteristics and sub-characteristics, and a set of proposed extended guidelines. Results: The ISO/IEC 9126 model is reasonably reliable when used to classify defects, even using incomplete defect reports. Reliability and variability is better for the six high level main characteristics of the model than for the 22 sub- characteristics. Conclusions: The ISO/IEC 9126 software quality model provides a solid foundation for defect classification. We also recommend, based on the follow up qualitative analysis performed, to use more complete defect reports and tailor the quality model to the context of us

Razvoj sveobuhvatne metode evaluacije upotrebljivosti web zasnovanih geografskih informacionih sistema za specijalne namene

Today, many various kinds of GIS applications are in everyday use. GIS is defined as a set of tools used to collect, store, retrieve, transform and display spatial data from the real world. Today, Web browser is used as a general multi-purpose client application and it provides “well-known” user environment because people are used to use Internet so they know how to work with a web browser. The users of Web-based geographic information systems (Web GIS) are mostly casual end users who have only a very low computer literacy and limited knowledge of GIS technologies. This is why Web-based GIS applications should be designed very carefully and with a strong focus on their usability. Usability is one relevant factor of the quality of Web applications. Recently, it has been receiving great attention, being recognized as a fundamental property for the success of Web applications. Determining the degree of usability is a process in which systems are evaluated in order to determine success product using methods available to the evaluator. One of the current goals of the Web engineering research is defining methods for ensuring usability. Usability as a quality characteristics is defined by many authors and several ISO standards. Many various methods of usability evaluation have been developed. These studies are certainly important as that would further deepen our understanding on factors that contributes towards the usability of Web GIS applications. One important success factor is, therefore, the need to warranty the levels of quality of the Web GIS as software products. High-quality software products are essential to provide value, and avoid potential negative consequences, for the stakeholders. Assessing the quality in use will allow Web GIS application owners to estimate how usable a Web GIS application might be and the user’s satisfaction. User characteristics are also important determinants of usability. The term 'context' includes the characteristics of the users and the work goals they are seeking to achieve, as well as the technical, physical and organizational environments in which they work. Therefore, this PhD thesis describes a practical method for identifying contextual aspects of usability in software systems, and for helping ensure that usability evaluations reflect the context of use and give data with acceptable validity. Attempts to objectively evaluate usability of information systems are old. In order to evaluate the quality of developed systems, a set of quality characteristics and criteria are required as a basis to describe the system quality. During the decades, different literature described various models and attributes of usability. Many usability models have been proposed to allow software usability evaluation. The main purpose of the software product usability model is to specify and assess the level of usability of a product through internal measures of inherent properties of the software, and through external measures of the behavior of the system of which the software is part. Aim of the PhD thesis is to propose a suitable usability model for evaluating the usability of Web-based GIS applications. Researchers have not developed yet any model that precisely describes usability definition and all its attributes that takes into account the varying aspects of usability. It developes a new more comprehensive definition of quality in use, which has usability in use, flexibility in use, satisfaction in use and safety in use as characteristics that can be quantified from the perspectives of different stakeholders, including users, managers and maintainers. Since all of the usability factors do not have the same importance in the overall usability assessment of the WebGIS applications, the proposed factors have been weighted by adopting AHP (Analytic Hierarchy Process) approach. The main goal of this PhD thesis is to propose a comprehensive methodology for usability evaluation and identify the most serious usability problems of special purpose Web-based geographic information system. Usability evaluation can increase software quality and its capability to satisfy users and their needs. The proposed methodology should lead to decreasing time necessary for evaluation of applications and results processing

Understanding data reuse and barriers to reuse of archaeological data: a quality-in-use methodological approach

Over the last decade, innovation has centred on making archaeological data more interoperable, increasing the discoverability of data through integrated cross-search and facilitating knowledge creation by combining data in new ways. An emerging research challenge for the next decade is optimising archaeological data for reuse and defining what constitutes good practice around reuse. Critical to this research is understanding the current state-of-the-art regarding both existing best practices and barriers to using and reusing archaeological data. This research aimed to understand how to optimise archives and interfaces to maximise the discovery, use and reuse of archaeological data and explore how archaeological archives can better respond to user needs. The study was bound by (i) the reuse of digital archaeological archives; (ii) orientation to content usability and reusability; (iii) maintaining a user-orientated approach; (iv) collecting data from professionals in archaeology and heritage. The research group members adopted the quality-in-use conceptual approach for this study. Quality in use is 'the degree to which a product or system can be used by specific users to meet their needs to achieve specific goals with effectiveness, efficiency, satisfaction, and freedom from risk in specific contexts of use'. The research methodology is based on the SQuaRE (System and Software Quality Requirements and Evaluation) model, represented in the ISO/IEC 25000 standards series. In addition, the quality-in-use metric for investigation of reuse and barriers to reuse of archaeological data were adopted from the standardised measurement functions and methods of ISO/IEC 25022:2016. The result was a methodological model composed of 5 characteristics (Effectiveness, Efficiency, Satisfaction, Context coverage and Usability) with 14 measures (Task completeness, Objectives achievement, Task time, Cost-effectiveness, Overall satisfaction, Satisfaction with features, User trust in the system, data and paradata, User pleasure, Physical comfort, Context completeness, Flexible context of use and User guidance completeness). The methodology was tested with specific Contexts of use (use cases), orientated to a distinct user with the specific professional goal of data reuse. Three use cases relating to 3D Pottery, radiocarbon, and GIS data were created. The pilot study has proven that the methodology works and could be applied in future research. This article discusses the application of the quality-in-use approach for evaluating the quality of digital archaeological archives, as well as presenting the methodology and the results of the pilot study

Quality of experience aware adaptive hypermedia system

The research reported in this thesis proposes, designs and tests a novel Quality of Experience Layer (QoE-layer) for the classic Adaptive Hypermedia Systems (AHS) architecture. Its goal is to improve the end-user perceived Quality of Service in different operational environments suitable for residential users. While the AHS’ main role of delivering personalised content is not altered, its functionality and performance is improved and thus the user satisfaction with the service provided. The QoE Layer takes into account multiple factors that affect Quality of Experience (QoE), such as Web components and network connection. It uses a novel Perceived Performance Model that takes into consideration a variety of performance metrics, in order to learn about the Web user operational environment characteristics, about changes in network connection and the consequences of these changes on the user’s quality of experience. This model also considers the user’s subjective opinion about his/her QoE, increasing its effectiveness and suggests strategies for tailoring Web content in order to improve QoE. The user related information is modelled using a stereotype-based technique that makes use of probability and distribution theory. The QoE-Layer has been assessed through both simulations and qualitative evaluation in the educational area (mainly distance learning), when users interact with the system in a low bit rate operational environment. The simulations have assessed “learning” and “adaptability” behaviour of the proposed layer in different and variable home connections when a learning task is performed. The correctness of Perceived Performance Model (PPM) suggestions, access time of the learning process and quantity of transmitted data were analysed. The results show that the QoE layer significantly improves the performance in terms of the access time of the learning process with a reduction in the quantity of data sent by using image compression and/or elimination. A visual quality assessment confirmed that this image quality reduction does not significantly affect the viewers’ perceived quality that was close to “good” perceptual level. For qualitative evaluation the QoE layer has been deployed on the open-source AHA! system. The goal of this evaluation was to compare the learning outcome, system usability and user satisfaction when AHA! and QoE-ware AHA systems were used. The assessment was performed in terms of learner achievement, learning performance and usability assessment. The results indicate that QoE-aware AHA system did not affect the learning outcome (the students have similar-learning achievements) but the learning performance was improved in terms of study time. Most significantly, QoE-aware AHA provides an important improvement in system usability as indicated by users’ opinion about their satisfaction related to QoE

What makes a satisfying user experience in the context of human-computer interaction?

The purpose of this research is to study the relations of user interface aesthetics, usability, and user experience in the context of human-computer interaction (HCI). This study aims to discover the key factors contributing to a satisfying user experience from the user's perspective. We present a novel user experience model that connects perceived usability, and its subcharacteristics, into perceived user experience, and to the emotional responses it evokes. The ISO/IEC 205010:2011 standard of the International Standardization Organization will act as a framework for the constructed user experience model, combining characteristics from both Product Quality and Quality-in-Use. An empirical study including 149 participants, consisting mostly of young, Finnish university students, will be conducted, in which the respondents evaluate three European festival websites by 28 different user experience aspects per each website. A partial least squares structural equation modeling (PLS-SEM) will be used as a statistical methodology in order to discover the structural relations between usability subcharacteristics (Appropriateness Recognizability, User Interface Aesthetics, Learnability and Operability) and Satisfying User Experience in HCI

Evaluasi Website Akademik Menggunakan ISO/IEC 9126

Abstrak Penggunaan website untuk mendukung aktivitas akademik dalam dunia pendidikan terus berkembang sehingga website yang berkualitas pun diperlukan. Namun, standar evaluasi kualitas website masih terbatas. ISO 9126 adalah standar evaluasi yang telah diakui secara internasional dalam mengevaluasi sebuah perangkat lunak dari sisi software engineering. Model ini dipandang sebagai standar yang pengukurannya cukup valid, reliable dan efisien dalam mengukur kualitas dari sebuah perangkat lunak. Website akademik dalam penerapannya menggunakan perangkat lunak berupa website yang dapat diakses secara online baik oleh dosen maupun mahasiswa. Pada tugas ini dilakukan evaluasi kualitas website akademik menggunakan ISO/IEC 9126 pada website akademik Universitas Telkom. Evaluasi dilakukan untuk memastikan apakah website tersebut memenuhi harapan dan tujuan pengguna. Pengguna yang dimaksud dalam hal ini adalah mahasiswa. Karakteristik ISO/IEC 9126 mampu memberikan tolak ukur yang baik dalam mengevaluasi website akademik Universitas Telkom. Evaluasi dilakukan pada karakteristik functionality, reliability, usability dan efficiency. Berdasarkan hasil evaluasi diperoleh tiga karakteristik yang perlu untuk ditingkatkan yaitu reliability, usability dan functionality. Karakteristik efficiency ditemukan tidak valid sehingga tidak diikutkan dalam analisis dan rekomendasi dalam perbaikan selanjutnya. Kata kunci --- evaluasi, website akademik, ISO/IEC 9126, quality model, kano model, q-score Abstract The use of the website to support academic activities in education continues to grow so that a quality website was needed. However, evaluation of website quality standards are still limited. ISO 9126 is one of standards of evaluation that has been recognized internationally in evaluating a software from the software engineering perspective. This model is seen as a measurement standard that quite valid, reliable and efficient in measuring the quality of a software. Academic websites in its application using the software in the form of a website that can be accessed online by both lecturers and students. In this research, we evaluate the quality of academic websites using ISO / IEC 9126 on the academic website of Telkom University. The evaluation is conducted to ascertain whether the website meets user expectations and goals. Users are referred to in this case was university students from different faculty and study. Characteristics of ISO / IEC 9126 is able to provide a good benchmark to evaluate the University\u27s academic website Telkom. Evaluation focused on the characteristics they are functionality, reliability, usability and efficiency. Based on the evaluation results we obtained three characteristics that need to be improved, they are reliability, usability and functionality. Efficiency characteristics found invalid on validity test so it is not included in the analysis and recommendations for further improvement. Keywords --- evaluation, academic websites, ISO / IEC 9126, quality models, Kano models, q-score</p

Evaluation of

The use of software in the educational system has significantly improve the process of learning of the students. Today most schools adopt their curriculum model using software; oriented application that educational improvements and bonding of the technological world with education. In this article the JAWS tool is evaluated at the Basic Education School Specialized Blind and Deaf of Machala city, in which we will present the educational contribution that has provided the JAWS software, Job Access With Speech to students with disabilities visual. The research was conducted within the framework of a quantitative study, obtaining data through surveys and the application of standard ISO/IEC 9126 quality based on the characteristics of software USAbility. The result of the research showed that students have improved their skills by using this system

Development of Rubric of Higher Order Thinking Skills Assessment on Mathematics Learning

This research aims to develop a rubric of Higher-Order Thinking Skills Assessment on mathematics learning that is valid, reliable, and practical. The Plomp's developmental method, the Generic model for educational design, was deployed to develop the HOTS assessment rubric. The method consisted of 4 phases, namely; (1) preliminary investigation, (2) design, (3) development, and (4) implementation and evaluation. The rubric was designed and developed to measure students' higher-order thinking skills in mathematics learning. The rubric was developed by integrating HOTS characteristics in mathematics: critical thinking, problem-solving, mathematical understanding, mathematical modeling, proof and reasoning, mathematical representation, and mathematical communication. The HOTS assessment rubric's quality was examined using Plomp's product/prototype quality criteria: (1) validity, (2) reliability, and (3) practicality/usability. The rubric of the HOTS assessment was validated by two validators that are experts on HOTS in mathematics. The practicality/usability of the rubrics was examined by five mathematics teachers from different Junior High schools. The validity and reliability were measured using Gregory's Expert Agreement Index (EAI), while the rubric's practicality was analyzed using practicality product criteria. The research shows that the validity of the rubric of HOTS assessment on mathematics learning is 0.82 (valid), and the reliability is 0.79 (reliable). Meanwhile, the practicality of the rubric is 81.02 (Good). In conclusion, the rubric of the HOTS assessment on mathematics learning is valid, reliable, and practical to use to measure students' higher-order thinking skills in mathematics learning