A comparison of user testing and heuristic evaluation methods for identifying website usability problems

This study compared the effectiveness and efficiency of two usability testing methods, user testing and heuristic evaluation. Thirty two participants took part in the study, sixteen for each of the two methods. Four measures were used to compare their performance: number of problems identified, severity of problems, type of problems and time taken to find problems. It was found that heuristic evaluation found nearly 5 times more individual problems than user testing, so could be seen as more effective. However, user testing found on average slightly more severe problems and took less time to complete than heuristic evaluation. Heuristic evaluation had a faster problem identification rate (number of seconds per problem found), so could also be seen as more efficient. While each method had advantages in the test both methods are seen as complementary to each other in practice

Usability Methodologies and Data Selection: Assessing the Usability techniques on Educational Websites

The purpose of an educational website is to efficiently provide up-to-date services and resources to students, faculty, and staff. However, the needs of university students are not always prioritized throughout the design phase of websites; instead, technological considerations, organizational concerns, and commercial goals may all drive the process. When it comes to meeting the demands of the target audience, website designers often fall short. At the same time, visitors may encounter a wide variety of usability issues while visiting the sites. Thus, the purpose of this study is to examine the efficacy of different approaches to evaluating the usability of university websites from the viewpoint of their target audiences via user-based testing and questionnaire methods involving actual users. The outcomes, which are based on data from tests conducted in real time, outline the difficulties and advantages associated with each strategy. Both a user-based test and a questionnaire, which are both trustworthy evaluation methodologies, were used in the process of analyzing and evaluating the websites

Analisis Usability Menggunakan Metode Heuristic Evaluation dan Web Usability Evaluation Tool pada Website ACC Career

Abstract. ACC Career website is a website from PT. Astra Credit Companies which aims for work applications. The website has been around for a long time but has never been updated. Recently, ACC has been revamping the website. Before the new website is launched, a usability evaluation is carried out using the heuristic evaluation and WEBUSE methods which aims to determine the website usability assessment for both the old and the new user interface. Usability is said to be good if users find it easy, happy and satisfied using the system. The heuristic evaluation method brings out problems in the system and recommendations for improvement from the evaluator. The Web Usability Evaluation Tool brings out usability levels based on the assessment of the questionnaire respondents. From the old website, the evaluator found 48 problems and the usability level obtained from the respondents' answers were Good for all categories. From the new website, the evaluators found 42 issues and the usability level was evenly divided between Good and Excellent.Keywords: Revamp, Website, Usability, Heuristic Evaluation, WEBUSE. Abstrak. Website ACC Career merupakan website dari PT. Astra Credit Companies (ACC) yang bertujuan untuk pengaplikasian pekerjaan. Website ini sudah ada sejak lama tetapi belum pernah ada pembaharuan. Baru-baru ini ACC sedang melakukan pengembangan ulang (revamp) website tersebut. Sebelum produk baru diluncurkan, diadakan evaluasi usability menggunakan metode Heuristic Evaluation dan WEBUSE yang bertujuan untuk mengetahui penilaian usability website baik untuk tampilan lama maupun tampilan yang sedang dikembangkan. Usability dikatakan baik jika pengguna mudah, senang, dan puas menggunakan sistem. Metode Heuristic Evaluation menghasilkan permasalahan yang ada pada sistem dan rekomendasi perbaikan dari evaluator. Metode Webuse menghasilkan tingkat usability berdasarkan penilaian responden kuisioner. Pada tampilan lama, evaluator menemukan 48 permasalahan dan level usability yang didapat dari jawaban responden adalah Good untuk seluruh kategori. Pada tampilan baru, evaluator menemukan 42 permasalahan dan level usability terbagi rata antara Good dan Excellent.Kata Kunci: Revamp, Website, Usability, Heuristic Evaluation, WEBUSE

A quantitative approach to evaluate usability of academic websites based on human perception

AbstractIn this competitive world, websites are considered to be a key aspect of any organization’s competitiveness. In addition to visual esthetics, usability of a website is a strong determinant for user’s satisfaction and pleasure. However, lack of appropriate techniques and attributes for measuring usability may constrain the usefulness of a website. To address this issue, we conduct a statistical study to evaluate the usability and accessibility levels of three popular academic websites based on human (user) perception. Two types of usability evaluation techniques were employed in this study. First one is Questionnaire-based evaluation and second one is performance-based evaluation. Usability assessment was performed by analyzing the results from the observed task success rates, task completion times, post-task satisfaction ratings and feedback. We also investigate the possibility of there being any impact of task completion times on participant’s satisfaction levels. The results of the questionnaire based evaluation were observed to be consistent with the results of performance-based evaluation. Accessibility evaluation was carried out by testing the degree of compliance of the web pages as per WCAG 2.0 guidelines

USABILITY EVALUATON OF USERS’ EXPERIENCE ON SOME EXISTING E-COMMERCE PLATFORMS

Internet has become increasingly popular nowadays. Several million of websites have been built and used for electronic buying and selling. Many designers have begun to focus their attention on whether these platforms can really be used to the satisfaction of users. Hence, the need to evaluate users’ experience on E-Commerce platforms. This research compares five platforms (Jumia, Ali-Express, Konga, Amazon and Jiji) based on users’ review through the use of online questionnaires for evaluating the platforms. From the data retrieved, Jumia, Konga and Ali Express recorded a total number of 105, 67 and 45 respondents representing 47.29%, 31.08% and 20.27% of the used sample population respectively. Amazon and Jiji recorded 2 and 3 respondents respectively accounting for 0.9% and 1.35% of the total population size. Attention should be given to attractive and easy-to-Navigate E-Commerce platform designs for users to have good user experience

ANALISIS PERBANDINGAN TINGKAT USABILITAS PADA WEBSITE BELANJA ONLINE DI INDONESIA (STUDI KASUS: TOKOPEDIA, BUKA LAPAK, SHOPEE)

Seiring meningkatkanya jumlah pengguna internet di Indonesia, fenomena belanja online saatini juga semakin meningkat. Website belanja yang diakses oleh para pembeli online dituntutuntuk memiliki tingkat usabilitas yang tinggi agar memudahkan penggunanya dalam berbelanjaonline. Penelitian ini dilakukan untuk mengevaluasi tingkat usabilitas dari 3 website belanjaonline di Indonesia yaitu Tokopedia, Bukalapak, dan Shopee yang ditinjau dari 5 dimensiusabilitas menurut Nielsen yakni learnability, efficiency, memorability, errors, dan satisfaction.Metode yang digunakan yakni thinking aloud yang melibatkan pengguna internet sebagairesponden dengan rentang usia 20-40 tahun sebanyak 42 orang, dimana responden tersebutterbagi dalam dua kategori user, yakni novice user dan casual user. Hasil akhir dari penelitianini berupa masukan dan rekomendasi perbaikan yang dapat dilakukan untuk meningkatkankepuasan konsumen bagi para pengembang website belanja online di Indonesia.  Kata kunci: thinking aloud, usabilitas, website belanja onlin

HEURISTIC EVALUATION OF E-COMMERCE MARKETPLACE

This paper adapts one of the most popular heuristic evaluation techniques to effectively assess the design of an e-commerce marketplace. Heuristic evaluation for the e-commerce marketplace with a detailed explanation is presented as a guideline for the e-commerce marketplace. This research involved usability evaluation that analyses the e-commerce marketplace using a set of design characteristics (usability heuristics). The process of discovering usability problems is one of the three objectives of this project. The selected set of heuristics could significantly influence the results (usability findings) of performed heuristic evaluations. Based on the evaluators' evaluation, a set of heuristics was produced. This will be used as a guideline to be applied to the design of the e-commerce marketplace called University Textbook Marketplace (UniText). The result of this research presents the heuristic evaluation for e-commerce marketplace guidelines to perform a cost-effective usability evaluation of e-commerce marketplace applications. Therefore, improving website design that is both pleasurable and meets the user's and customer's needs

A usability evaluation of the prototype Afrikaanse idiome-woordeboek

The Afrikaanse idiome-woordeboek is a prototype e-dictionary of Afrikaans fixed expres-sions developed with the intention to test the functionalities of the e-dictionary. This dictionary is based on the function theory of lexicography. The e-dictionary makes use of various technologies. When digital tools are developed it is important to consider the usability of the tool. Usability evaluation was done on the Afrikaanse idiome-woordeboek to determine with what success it can be used. Discount usability methods, viz. heuristic evaluation and usability testing were used. This article reports on the findings from the usability tests which are discussed under the categories of content, information architecture, navigation, access (searching and browsing), help, customisation and the use of innovative technologies to manage data in e-dictionaries for search and display. The usability evaluation showed that the users did not always use the e-dictionary as the designers intended. Various recommendations are made to the designers of the Afrikaanse idiome-woordeboek, as well as for the design of e-dictionaries in general. Recommendations appropriate to e-dictionaries in general are made regarding usability evaluation, information architecture, searching in e-dic-tionaries, the data that can be included in e-dictionaries and training of users of e-dictionaries. Keywords: E-Dictionaries, Online Dictionaries, Electronic Dictionaries, Usability Evaluation, Usability Tests, Discount Usability, Dictionary Literacy

Tutkimus vaateverkkokauppojen informaatioarkkitehtuurista

Tämän tutkimuksen tavoitteena oli tutkia informaatioarkkitehtuuria neljän eri tavoitteen avulla. Ensimmäisenä tavoitteena oli onnistua määrittelemään informaatioarkkitehtuuri ja sen osa-alueet kattavasti ja ymmärrettävästi. Toisena tavoitteena oli hyödyntää eri heuristiikkoja sellaisen tarkistuslistan luomiseen, joka sopisi erityisesti verkkokauppojen arviointiin. Kolmantena tavoitteena oli varsinaisen heuristisen evaluoinnin toteutus verkkokaupoille, ja neljäntenä tavoitteena oli yhdistää kauppatieteiden opinnot tähän graduun tutkimalla myös verkkokauppojen kaupallisia ominaisuuksia. Informaatioarkkitehtuurin käsite elää yhä, eikä yhteen kaikkia miellyttävään määritelmään ole vielä päädytty. Informaatioarkkitehtuurin juuret ulottuvat 1970-luvulle (Ding & Lin, 2010), ja osittain tästä syystä käsitteellä on suhteita useisiin eri aloihin. Määritelmien eroista huolimatta niissä on myös paljon yhteistä; kuten pyrkimys tiedon organisoinnin, navigoinnin, nimeämisen, hakujärjestelmien ja sivuston ulkoasun saumattomaan yhteistyöhön. Siinä missä tiedon määrä kasvaa vuosi vuodelta suuremmaksi, tulee myös sen käsittelystä ja asettelusta – eli informaatioarkkitehtuurista entistä tärkeämpää. Suomalaisten verkkokauppojen arviointi toteutettiin soveltamalla Nielsenin ja Molichin (1990) esittelemää heuristista evaluointia. Kyseinen menetelmä on tehty käyttöliittymien analysointiin ilman käyttäjätestausta, ja sen aikaisemman menestyksen ja nykyisen suosion vuoksi koin sen olevan sopiva arviointimenetelmä tähän tutkimukseen. Arvioinnin avuksi käytetyn kysymyslistan loin aikaisempien arviointilistojen sekä eri heuristiikkajoukkojen avulla. Arvioiduiksi verkkokaupoiksi valitsin suomalaiset Marimekon, Nanson ja R-Collectionin verkkokaupat. Arvioinnissa tunnistettiin yhteensä 83 ongelmaa, joista 43 löytyi Marimekon verkkokaupasta, 26 Nanson ja 23 R-Collectionin. Lähes neljäsosa löydetyistä ongelmista liittyi hakujärjestelmiin, ja vähiten ongelmia havaittiin markkinointiin ja ulkoasuun liittyen, sillä näiden kahden ongelmat kattoivat kaikista ongelmista vain 24 %. Kysymyslista ja heuristiikat soveltuivat tulokset huomioon ottaen hyvin tämän tutkimuksen tekoon, ja niiden avulla pystyi arvioimaan myös kaupallisuutta tyydyttävällä tasolla. Suuresta ongelmien lukumäärästä huolimatta yli 83 % löydetyistä ongelmista oli joko pieniä tai kosmeettisia. Vaikka tässä tutkimuksessa onnistuttiin kysymyslistalla ja käytetyillä heuristiikoilla hyvin, vaatii niiden yleinen sopivuus vielä paljon tutkimusta. Erityisesti heuristiikkojen soveltaminen muihinkin kuin käytettävyyden osa-alueisiin on mielenkiintoinen aihe, josta voi tulevaisuudessa selvitä vielä paljon uutta