2.3. User trial and usability testing
User trial is a typical experimental method (McClelland 1995). Pheasant (1996) has defined user trial as an experimental investigation in which a group of users test versions of products under controlled conditions. Setting up a user trial (Fig. 7) is primarily about creating an environment that enables the interaction between a product and a user to be systematically examined and measured (McClelland 1995). User trial is regarded as synonymous to usability testing insofar as this term refers to the evaluation of an artefact under controlled conditions involving users (McClelland 1995).
Usability is defined by the ISO 9241-11 (1998) standard as ”the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use”, where effectiveness is ”the accuracy and completeness with which users achieve specified goals”, efficiency refers to ”the resources expended in relation to the accuracy and completeness with which users achieve goals”, and satisfaction is ”the comfort and acceptability of use”. Actually, the same attributes are common to traditional goals of ergonomics and human factors engineering (cf. Pheasant 1996, Sanders & McCormick 1993). Keinonen (1998) defines usability as a set of measures, design aims and practices for designers to ensure that a product will fulfil a specific set of user needs. It is easy to believe that consumers are becoming increasingly demanding with regard to usability. People are disappointed if a product is cumbersome to use. Usability testing itself is a multidisciplinary activity that draws on computer science, cognitive psychology, ergonomics, safety, graphic design and industrial design.
Usability testing refers to the systematic experimental evaluation of the ”interaction between people and the products, equipment, environments, and services they use” (McClelland 1995). It evaluates how easy a product is to use and whether it is functional and acceptable (Bogner 1998). Usability test results may not be valid unless the conditions of the test closely match those of actual product use (Cushman & Rosenberg 1991). Therefore, the operating characteristics of the prototype, the tasks, the duration of the test, and the environmental conditions must be realistic. Both extreme and typical conditions should be included in the test.
There are a variety of methods to choose from, when planning how to identify user needs and how to integrate them into product development, e.g. usability studies. With limited resources and limited access to different user groups, such methods as discount usability engineering, heuristic evaluation, cognitive walkthrough, etc., have been used, which involve small numbers of expert users and/or advocates for users in the design cycle (Hyppönen 1999). One widely used method to involve users into a process of design is focus groups (O’Donnell et al. 1991, Calder 1977).
McClelland (1995) has gathered some comparisons of usability study methods. Usability testing seems to be quite a good approach to product evaluation of HCI. Heuristic methods require several experienced usability specialists to provide results comparable to usability testing, and the usability issues raised are different from those found in usability tests. In another study, usability testing revealed more severe problems, more recurring problems and more global problems than heuristic evaluation methods. In comparison with walkthrough, user testing revealed the largest number of usability problems and identified a number of severe problems that were missed by using walkthrough methods. User testing also required less time to identify each problem. However, McClelland (1995) points out the need to verify whether the conclusions are equally valid for other types of user interface.