Usability is an attribute of any good product, just as its functionality. It refers mainly to the utility of a product for its indented users, as well as to its ease of use. And whilst a correct functionality is critical for the commercial success of any product, its value comes through the human needs that it fulfills, which is determined through various marketing research techniques. In parallel, the IT&C community has developed in the last two decades its own type of research, called usability testing, used mainly to evaluate interface ease of use and all usability problems associated with.software products. This article aims at finding the right place for usability testing and usability professionals in the marketing community, as well as drawing a wider picture, from a marketing research perspective, on one of the most popular topics in IT&C community for the benefit of marketing scholars and professionals

Orzan Gheorghe

Orzan Mihai

Stanculescu Anca-Maria

Directory of Open Access Journals

1038 QUALITATIVE MARKETING RESEARCH THROUGH USABILITY TESTING Orzan Mihai Academia de Studii Economice, Facultatea de Marketing, Str. Mihai Eminescu nr. 13-15, sect. 1, cod 010511, camera 1406, Bucure￿ti, e-mail: mihai.orzan@ase.ro, telefon: 0722318323 Orzan Gheorghe Academia de Studii Economice, Facultatea de Marketing, Adresa: Str. Mihai Eminescu nr. 13-15, sect. 1, cod 010511, camera 1401, Bucure￿ti, e-mail: orzang@ase.ro, telefon: 0722218140 Stanculescu Anca-Maria Academia de Studii Economice, Facultatea de Rela￿ii Economice Interna￿ionale, Adresa: Bd. Stirbei-Voda, nr. 4, bl. 2, sc. 4, ap. 139, sect. 1, Bucure￿ti, e-mail: sam33ro@yahoo.com, telefon: 0730 511997  Abstract: Usability is an attribute of any good product, just as its functionality. It refers mainly to the utility of a product for its indented users, as well as to its ease of use. And whilst a correct functionality is critical for the commercial success of any product, its value comes through the human needs that it fulfills, which is determined through various marketing research techniques. In parallel, the IT&C community has developed in the last two decades its own type of research, called usability testing, used mainly to evaluate interface ease of use and all usability problems associated with.software products. This article aims at finding the right place for usability testing and usability professionals in the marketing community, as well as drawing a wider picture, from a marketing research perspective, on one of the most popular topics in IT&C community for the benefit of marketing scholars and professionals. Keywords: usability, marketing, marketing research, computer human interaction, qualitative research.  One of the general usability principle observes that „if people don’t understand the functions of a given product, they will not use it; if it complicates their existence, as well they will not use it; and if they are current or potential customers they will only buy those products that are easy to use and that bring an obvious improvement to their lives” [14]. Usability is inherent in any  product or service, just as  well as functionality.  And,  whilst functionality defines what a specific product or service will (or is expected to) accomplish and a functionality test will try to determine weather its target performs according to its specifications (and expectations), accordingly a usability test is a tool used to identify how buyers will use the product or service to satisfy their needs and expectations. A correct functionality is critical, but not enough to ensure market success. Its value for customers comes from the needs it satisfies, as well as the degree of satisfaction they derive from it – and only  if  the  satisfaction  they  get  is  at  least  equal  to  what  they  expect  the  product  will  be  considered successful and they will continue to buy it and recommend it other people. And the satisfaction that people get after using a IT&C product is determined with usability testing. The goal of usability testing is usually the improvement of the product under test. A secondary objective is the improvement of the design and manufacturing processes for that particular product, or other in the same line, and this is one of the reasons that usability testing professionals feel that this tool is different from marketing  research,  as  these  „focus  on  the  investigation  of  marketing  phenomena  and  not  product improvement”[13].  Other  objectives  of  usability  testing  include  ease  of  use,  acceptance  of  new  or improved tools, portability or other interface problems between users and products or between the product under test and other items, with which it could or should come into contact. Usability testing is usually performed with a small sample of respondents, which have to be part of the target customers of the product or service subjected to the test. These will be asked to perform typical product functions and to use it as they will in normal, daily activities, in a controlled environment, under the  supervision  of  the  test  team,  which  will  record  everything  they  do  during  the  testing,  as  well  as everything they say. Not only will their test performance be analyzed, but their verbal assertions and body language will be subject to scrutiny after the test is completed. Data generated in this process are analyzed from both qualitative and quantitative perspectives and there are presented in a Usability Test Report, 1039 together with all the data (test performance, verbal comments and body language) to decision makers, in order to get valuable information about product  strengths and  weaknesses.  As a point of observation, qualitative data is very rich and complex and their analysis is usually a lengthy process, while quantitative data is usually more general in nature and is used only as an overview of the phenomena under scrutiny, as the samples used in usability testing are not representative for the consumer population of the product. In general, in order to have a usable product, this has to be easy to use, its functionality easy to understand and easy to remember. In addition, it has to satisfy efficiently the need that it addresses, to allow for a quick update or correction anytime such an action is deemed necessary, to prevent proactively human errors and recover quickly from detrimental events, and to have an attractive design [12]. And, what is most important to keep in mind, is that users are those who determine all these product qualities that make up a usable product. Researchers alone, even experts in usability testing or a certain product’s industry cannon establish, by themselves, whether a specific good is going to be perceived as usable or not.  In order to bring to market a usable product, usability testing have to be performed during its entire life cycle, starting with the design and implementation phases up to the last revision [8]. All the usability related attributes have to be considered at all times in a product’s life cycle, and iterative designs and implementation  phases  are  almost  unavoidable,  as  they  are  needed  to  correct  late  usability  problems identified in successive testing phases. Usability testing is done using several different approaches, but all have a number of similar characteristics [15]: •  their scope is to increase the perceived usability for the indented users target of a certain product.  This  implies  a  significant  improvement  of  product  efficiency,  starting  with  the design and development phases of product life cycle.  •  the  participants  (sample)  are  selected  from  a  targeted  population  segment.  A  typical  test includes  between  8  and  16  participants  per  sessions,  but  it  has  been  argued  that  most significant data appears even with samples of five persons [19].  •  the participants do real, common tasks and activities associated with the tested product. The activities tested must be the same activities expected by the real users of the tested product. This involves the fact that the testing must start with an exploratory research of the target segment in order to identify the uses of the tested product, the estimated frequency of those activities, their environment and trigger influences [18]. From this data test cases are derived, realist and relevant for the tested product target population. These cases are translated into testing scenarios, which will guide participants (respondents) during tests. These must solve as many as possible utility problems, based on potential worrying issues of the producers and/or problems identified by the specialists in that domain. •  to  observe,  record,  and  analyze  subjects’  tasks  performances.  The  whole  activity  of  the subjects  in  the  testing  laboratory  is  recorded  for  subsequently  comparisons  and  analyses.  Analyze include both commentaries and the performances during the testing process. In some variants of usability testing, the participants are asked to verbalize their thoughts, including reasoning and observations during task performance. •  data  resulted  from  the  testing  phase  is  analyzed  using  quantitative,  psychological  and behavioral techniques, in order to identify usability problems and, possibly, their potential solutions. •  the information resulted from usability tests is used to improve the qualities, the usability and the production process of the tested good. Thus, a utility test is considered closed only when the information resulted after its application is used in practice to improve the respective product or service. Usability testing that by their very nature ensures high usability for all products they employ it, so both producers and customers will benefit greatly from their results [20]. Users gain satisfaction from products with a steep learning curve, which allows maximum ease of use. Producers and sellers gain by increased sales, brought on by improved customer satisfaction, on both products that have been subjected to usability testing and other company products, which will benefit from a better corporate image, increased customer awareness, scale economies in later life cycle stages, lower customer care costs, as well as a more focused and less extensive update process [17]. 1040 As a result, the customer are the first to notice the improvement of product usability and, in many cases, usability is a more significant factor in buying behavior than price, or even functionality [2]. Positive experiences  with  any  kind  of  product  or  service  are  very  important  in  customer  loyalty,  leading  to significant increases in customer lifetime value, not only in regard to a certain product that benefited from usability testing input, but for all of those companies’ products. The same stands true in reverse: negative usability experiences lead to loss of sales and a diminished company image, not only directly from those dissatisfied users, but from their friends and acquaintances as well, through world-of-mouth [3]. Longtime organizational  success  relies  on  customer  loyalty  and  customer  satisfaction  and  product  usability  is paramount for both.  Marketing research on the other hand, as the first function of marketing, is long established as a mandatory tool of competitive market success [1]. It aims at identifying threats and opportunities from outside the company,  strength  and  weaknesses  from  within.  The  European  Society  for  Opinion  and  Marketing Research (ESOMAR) defines marketing research a “a key element within the total field of marketing information, which links the consumer, customer and public to the marketer through information which is used  to  identify  and  define  marketing  opportunities  and  problems;  to  generate,  refine  and  evaluate marketing actions. Marketing research specifies the information required to address these issues, design methods for collecting information,  manages and implements the data collection process, analyses the results  and  communicates  the  findings  and  their  implications”  [7].  As  it  is  easily  observed  from  the usability testing description made in the previous pages, all these attributes can be easily attributed to usability testing, making it a textbook tool of marketing research. In general, marketing research is employed to identify opportunities and problems and to generate and refine  marketing  actions,  a  distinction  used  to  classify  marketing  research  into  problem  identification research and problem-solving research.  Marketing research is usually separated in qualitative and quantitative. We argue that usability testing can be assimilated to qualitative marketing research, an assumption explicitly refuted by Jakob Nielsen, one of the leading figures of usability testing today [14]. Distinction between qualitative and quantitative analysis is usually based on the possibility of numerical analysis on data generated by marketing research [4]. The term qualitative research is loosely used to refer to research whose findings are not subject to quantification or quantitative analysis. More significant might be the fact that qualitative research is characterized by small samples, while quantitative research uses large, statistically significant samples of data. In essence, many managers are reluctant to base important strategy decisions on small-sample (qualitative) research, because it relies so greatly on the subjectivity and interpretation of the researcher [11]. Qualitative research encompasses a variety of methods that can be a applied in a flexible manner, to enable participants to reflect upon and express their views or to observe their behaviour. It seeks to encapsulate the behaviour, experiences and feelings of participants (just like usability tests) in their own terms and context [10]. Qualitative research is based on a set of ideas and associated methods from the broad area of psychological sciences [5], used to gain access to individuals’ subconscious and unconscious levels. So, while individuals may present a superficial explanation of events to themselves or others, these methods sought to dig deeper and penetrate the superficial. At the same time, qualitative research analysis aims to gain a holistic understanding of the world-view of individuals tested. Qualitative research techniques are usually classified as direct, which assume that the purposes of the research process are disclosed to the participant or are obvious, given the nature of the interview, which consist mainly of focus groups and in-depth interviews, and indirect, where the purposes of the research are disguised from the participants, in order to obtain more “genuine” responses and reactions, which include observation, mystery shopper and ethnographic techniques. The following table offers some of the main characteristics of usability testing, in the context of qualitative and quantitative marketing research attributes, as presented by Carmen B￿lan in C￿toiu et al (p.84) [4].    1041 No.  Comparison criteria Qualitative Marketing Research Quantitative Marketing Research Usability Testing 1  Research objective Understanding of marketing phenomena, identification of motivation Data quantification and extrapolation of data for the entire population surveyed. Identify usability problems and how to eliminate them. 2  Sample size  Small  Large  Small 3  Amount of Information from each Respondent Substantial  Varies  Substantial 4  Requirements for administration Interviewer with special skills Interviewer without special skills No interviewer 5  Type of analysis  Subjective, interpretative Statistical, numerical  Subjective, behavioral 6  Hardware  Audio & video recorders, pictures, discussion guides Questionnaires, computers, printouts Audio & video recorders, motion trackers, test cases 7  Degree of replicability Low  High  Low 8  Researcher training Psychology, sociology, social psychology, consumer behavior, marketing, marketing research Statistics, decision models, decision support systems, computer programming, marketing, marketing research Psychology, sociology, computer programming, decision support systems 9  Type of research  Exploratory  Descriptive or causal  Exploratory  and causal 10  Data  acquisition model Unstructured  Structured  Unstructured Table 1: Usability Testing characteristics in the context of Qualitative and Quantitative Marketing Research As it can be observed from Table 1, third and last columns are almost identical, and we can conclude that a usability test matches most of the qualitative marketing research criteria. We must keep in mind that even some marketing research tools imply the lack of an interviewer, for instance surveys done using online or mailed  questionnaires.  In  addition,  some  ethnical  research  techniques  imply  the  use  of  test  cases  and scenarios, just like usability testing and the skills required from a researcher imply a mix of qualitative and quantitative analysis. And, what it is most important, the scope of marketing research is not limited to general information about phenomena [9], but goes into as much detail to uncover causes, predict patterns and discover solutions for all problems investigated. At this point, qualitative researchers are count amongst their tools instruments such as in-depth interviews, focus groups, brainstorming sessions, mystery shopper, and customer observation. I believe it is time to pay serious attention to the newest arrival in this family – usability testing – a technique targeted to IT&C products, but which can be successfully employed in a large number of market research areas. 1042 Bibliography 1.  Balaure, Virgil (coord.), Marketing, edi￿ia a II-a, Ed. Uranus, Bucure￿ti, 2002. 2.  Campbell, N., Chisman, J., Diller, K. ￿i Walbridge, S., „Designing for the User: How to Test for Usability”,  published  in  Proceeding  of  the  ACLR  X:  Crossing  the  Divide  Conference,  Martie 2001. 3.  C￿toiu, I. & Teodorescu, N., Comportamentul consumatorului, Editura Uranus, Bucure￿ti, 2003. 4.  C￿toiu, Iacob (coord.), Cercet￿ri de Marketing, Ed.a II-a, Editura Uranus, Bucure￿ti, 2002. 5.  DeGroot,  G.,  “Qualitative  Research:  Deeply  Dangerous  or  Just  Plain  Dotty?”,  published  in European Research 14 (3), p.136-141, 1986. 6.  Dumas, J. & Redish, J., A Practical Guide to Usability Testing. Ed. Norwood, NJ: Ablex, 1999. 7.  ESOMAR,  ICC/ESOMAR  International  Code  of  Marketing  and  Social  Research  Practice, available online at http://www.esomar.org/uploads/pdf/ps_cg_icccode.pdf, 2005. 8.  Gould, J.D. & Lewis, C., „Designing for Usability – key principles and what designers think of them”, published in Communications of the ACM, 28(3), p.300-311, 1985. 9.  Jarrett, C., “Market Research and Usability”, published in Usability & User Experience 7 (1), July, available online at: http://www.stcsig.org/usability/newsletter/0007-marketing.html, 2000. 10.  Kenway, J., “Keep on moving”, published in Research, November 2005, p.36. 11.  MacDaniel, C. & Gates, R., Marketing Research: The Impact of the Internet, John Wiley & Sons Publishing, Hoboken, NJ, 2002. 12.  Morkes,  J.  &  Nielsen,  J.,  „Concise,  scannable,  and  objective:  How  to  write  for  the  Web”, published in Alertbox, available online at:  http://www.useit.com/papers/webwriting/writing.html, 1997. 13.  Nielsen, J. & Mack, R., Usability Inspection Methods, John Wiley and Sons, New York, 1994. 14.  Nielsen, J., Designing Web Usability, New Riders Publishing, Indianapolis, USA, 2000. 15.  Orzan, G. & Orzan, M., Cybermarketing, Ed. Uranus, Bucure￿ti, 2007. 16.  Orzan, M., „Usability tests”, published in Revista Român￿ de Marketing 2(1), p.135-148, 2007. 17.  Preece, J., Rogers, Y., & Sharp, H. Interaction Design: Beyond Human-Computer Interaction. Ed. John Wiley & Sons, New York: NY, 2002. 18.  Sargeant, A. & West, D.C., Direct and Interactive Marketing, Oxford Press, New York, NY, 2001. 19.  Spool, J., Web Site Usability: A Designer's Guide, Morgan Kaufmann Publishers, San Francisco, 1999. 20.  Tedeschi, B., „Good web site design can lead to healthy sales”, published by The New York Times on  the  Web  E-Commerce  Report,  available  online  at:   http://www.nytimes.com/library/tech/99/08/cyber/commerce/30commerce.html, 1999.  

Qualitative Marketing Research Through Usability Testing

Abstract

Similar works

Full text

Available Versions

Directory of Open Access Journals