A Process for Appraising Commercial Usability Evaluation Methods

Ronan Fitzpatrick http://www.comp.dit.ie/rfitzpatrick/

Alan Dix http://www.hcibook.com/alan/

Download copy of full paper (PDF, 76K).

Full reference:

Ronan Fitzpatrick and Alan Dix (1999).
A Process for Appraising Commercial Usability Evaluation Methods.
Human-Computer Interaction: Ergonomics and User Interfaces, Proceedings of HCI International ’99, Munich, Lawrence Erlbaum Associates, New Jersey, USA
http://www.hcibook.com/alan/papers/hcii99/

Keywords International standards, European Community legislation, software quality, usability evaluation, commercial evaluation methods, strategic application, life cycle processes, context of use, usability measures, usability attributes.

abstract

Recent international quality standards and European Community legislation have identified new software quality factors. These new factors include suitability, installability and adaptability. Other quality factors need to be reviewed in the light of these developments. This has impacted on established commercial usability evaluation methods to the extent that it is appropriate to ask if these evaluation methods comply with the new standards and legislation. In order to answer this question the commercial evaluation methods need to be appraised (meta-evaluation) using a suitable method appraisal process. This paper describes such an appraisal process which specifically addresses the many considerations raised by the standards and the law. The appraisal method consists of two parts which provide an overview of the commercial method and a methodical analysis of how it complies. By combining this analysis with a weighting and rating technique the appraised method can achieve a score which can be compared with other commercial methods. The process is an essential tool for strategic managers who are responsible for usability evaluation during systems acquisition. It is also of benefit to supplier organisations who, in their efforts to develop the highest quality systems, need to demonstrate compliance with international standards and development process maturity models.