EN ISO 13407:1999

SLOVENSKI STANDARD
01-marec-2001
3URFHVLQDþUWRYDQMDLQWHUDNWLYQLKVLVWHPRYRVUHGRWRþHQLKQDþORYHND,62

Human-centred design processes for interactive systems (ISO 13407:1999)
Benutzer-orientierte Gestaltung interaktiver Systeme (ISO 13407:1999)
Processus de conception centrée sur l'opérateur humain pour les systemes interactifs
(ISO 13407:1999)
Ta slovenski standard je istoveten z: EN ISO 13407:1999
ICS:
13.180 Ergonomija Ergonomics
35.020 Informacijska tehnika in Information technology (IT) in
tehnologija na splošno general
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.

INTERNATIONAL ISO
STANDARD 13407
First edition
1999-06-01
Human-centred design processes for
interactive systems
Processus de conception centrée sur l'opérateur humain pour les systèmes
interactifs
A
Reference number
ISO 13407:1999(E)
ISO 13407:1999(E)
Contents Page
1 Scope .1
2 Terms and definitions .1
3 Structure of this International Standard.2
4 Rationale for adopting a human-centred design process .2
5 Principles of human-centred design.3
5.1 General.3
5.2 The active involvement of users and a clear understanding of user and task requirements.3
5.3 An appropriate allocation of function between users and technology.3
5.4 Iteration of design solutions.3
5.5 Multi-disciplinary design.4
6 Planning the human-centred design process.4
7 Human-centred design activities .5
7.1 General.5
7.2 Understand and specify the context of use .6
7.3 Specify the user and organizational requirements .7
7.4 Produce design solutions.8
7.5 Evaluate designs against requirements .9
8 Conformance.12
Annex A (informative) Guidance on other relevant standards .13
Annex B (informative) Example of a structure of a usability evaluation report .17
Annex C (informative) Sample procedure for demonstrating conformance to this International Standard.20
Bibliography.25
©  ISO 1999
All rights reserved. Unless otherwise specified, no part of this publication may be reproduced or utilized in any form or by any means, electronic
or mechanical, including photocopying and microfilm, without permission in writing from the publisher.
International Organization for Standardization
Case postale 56 • CH-1211 Genève 20 • Switzerland
Internet iso@iso.ch
Printed in Switzerland
ii
© ISO
ISO 13407:1999(E)
Foreword
ISO (the International Organization for Standardization) is a worldwide federation of national standards bodies (ISO
member bodies). The work of preparing International Standards is normally carried out through ISO technical
committees. Each member body interested in a subject for which a technical committee has been established has
the right to be represented on that committee. International organizations, governmental and non-governmental, in
liaison with ISO, also take part in the work. ISO collaborates closely with the International Electrotechnical
Commission (IEC) on all matters of electrotechnical standardization.
International Standards are drafted in accordance with the rules given in the ISO/IEC Directives, Part 3.
Draft International Standards adopted by the technical committees are circulated to the member bodies for voting.
Publication as an International Standard requires approval by at least 75 % of the member bodies casting a vote.
International Standard ISO 13407 was prepared by Technical Committee ISO/TC 159, Ergonomics, Subcommittee
SC 4, Ergonomics of human-system interaction.
Annexes A, B and C of this International Standard are for information only.
iii
© ISO
ISO 13407:1999(E)
Introduction
Human-centred design is an approach to interactive system development that focuses specifically on making
systems usable. It is a multi-disciplinary activity which incorporates human factors and ergonomics knowledge and
techniques. The application of human factors and ergonomics to interactive systems design enhances effectiveness
and efficiency, improves human working conditions, and counteracts possible adverse effects of use on human
health, safety and performance. Applying ergonomics to the design of systems involves taking account of human
capabilities, skills, limitations and needs.
Human-centred systems support users and motivate them to learn. The benefits can include increased productivity,
enhanced quality of work, reductions in support and training costs, and improved user satisfaction. Although there is
a substantial body of human factors and ergonomics knowledge about how such design processes can be
organized and used effectively, much of this information is only well-known by specialists in these fields. This
International Standard aims to help those responsible for managing hardware and software design processes to
identify and plan effective and timely human-centred design activities. It complements existing design approaches
and methods.
.
iv
INTERNATIONAL STANDARD  © ISO ISO 13407:1999(E)
Human-centred design processes for interactive systems
1 Scope
This International Standard provides guidance on human-centred design activities throughout the life cycle of
computer-based interactive systems. It is aimed at those managing design processes and provides guidance on
sources of information and standards relevant to the human-centred approach.
This International Standard is concerned with both hardware and software components of interactive systems.
NOTE Computer-based interactive systems vary in scale and complexity. Examples include off-the-shelf (shrink wrap)
software products, custom office systems, plant monitoring systems, automated banking systems and consumer products.
This International Standard addresses the planning and management of human-centred design. It does not address
all aspects of project management.
This International Standard provides an overview of human-centred design activities. It does not provide detailed
coverage of the methods and techniques required for human-centred design, nor does it address health and safety
aspects in detail.
The main users of this International Standard will be project managers. This International Standard therefore
addresses technical human factors and ergonomics issues only to the extent necessary to allow managers to
understand their relevance and importance in the design process as a whole. Such issues are dealt with more fully
in ISO 9241 (see bibliography) which is complementary to this International Standard and is aimed at system
developers, specifiers and purchasers of systems. Nonetheless, all parties involved in human-centred system
development, including the end-users of systems, should find the guidance in this International Standard relevant.
2 Terms and definitions
For the purposes of this International Standard, the following terms and definitions apply.
2.1
interactive system
combination of hardware and software components that receive input from, and communicate output to, a human
user in order to support his or her performance of a task
NOTE The term “system” is often used rather than “interactive system”.
2.2
prototype
representation of all or part of a product or system that, although limited in some way, can be used for evaluation
2.3
usability
extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency
and satisfaction in a specified context of use
[ISO 9241-11:1998, definition 3.1]
© ISO
ISO 13407:1999(E)
2.4
effectiveness
accuracy and completeness with which users achieve specified goals
[ISO 9241-11:1998, definition 3.2]
2.5
efficiency
resources expended in relation to the accuracy and completeness with which users achieve goals
[ISO 9241-11:1998, definition 3.3]
2.6
satisfaction
freedom from discomfort, and positive attitudes to the use of the product
[ISO 9241-11:1998, definition 3.4]
2.7
context of use
users, tasks, equipment (hardware, software and materials), and the physical and social environments in which a
product is used
[ISO 9241-11:1998, definition 3.5]
2.8
user
individual interacting with the system
[ISO 9241-10:1996, definition 2.2]
3 Structure of this International Standard
Clause 4 outlines the reasons for adopting a human-centred design process. These can be used to provide a
rationale for the use of human-centred methods, or to determine priorities for resource allocation during a project.
Clause 5 gives guidance on the principles of human-centred design. Clause 6 lists the issues to be considered
when planning human-centred design activities and discusses how these should relate to system design goals.
Clause 7 is the core of this International Standard. It describes each of the four essential human-centred activities
which should take place during the design process. Clause 8 gives further guidance on reporting human-centred
activities.
4 Rationale for adopting a human-centred design process
All work systems should follow the ergonomic principles described in ISO 6385:1981. Making interactive systems
more human-centred has substantial economic and social benefits. In most countries, employers and system
providers have legal obligations to protect users from risks to their health and safety. Making systems more usable
means systems can contribute to these aims, meeting user and organizational needs better. They
a) are easier to understand and use, thus reducing training and support costs,
b) improve user satisfaction and reduce discomfort and stress,
c) improve the productivity of users and the operational efficiency of organizations, and
d) improve product quality, appeal to the users and can provide a competitive advantage.
The complete benefits of human-centred design can be determined by taking into account the total life-cycle costs
of the system including conception, design, implementation, support, use and maintenance.
© ISO
ISO 13407:1999(E)
5 Principles of human-centred design
5.1 General
There are many industry and proprietary standard methods for the design of computer-based interactive systems.
This International Standard does not assume any one standard design process, nor does it cover all the different
activities necessary to ensure effective system design. It is complementary to existing design methods and provides
a human-centred perspective that can be integrated into different forms of design process in a way that is
appropriate to the particular context. All the human-centred design activities identified in clause 7 are applicable, to
a greater or lesser extent, at any stage in the development of a system.
Whatever the design process and allocation of responsibilities and roles adopted, the incorporation of a human-
centred approach is characterized by the following:
a) the active involvement of users and a clear understanding of user and task requirements;
b) an appropriate allocation of function between users and technology;
c) the iteration of design solutions;
d) multi-disciplinary design.
5.2 The active involvement of users and a clear understanding of user and task requirements
The involvement of users in the development process provides a valuable source of knowledge about the context of
use, the tasks, and how users are likely to work with the future product or system. The effectiveness of user
involvement increases as the interaction between the developers and the users increases. The nature of user
involvement varies depending on the design activities which are being undertaken.
When custom-made products are being developed, the proposed users and the tasks performed can be directly
linked to the development process. The organization procuring the system has the opportunity to have a direct
influence on the design as it emerges, and solutions can be evaluated by those who are actually going to be
working with them. Such involvement and participation also increase user acceptance and commitment.
When generic or consumer products are being developed, the user population is dispersed and is perhaps not
easily accessible. It is still essential that users or appropriate representatives are involved in development, in order
that the relevant user and task requirements can be identified for inclusion in the system specification, and in order
to provide feedback through testing of the proposed design solutions.
5.3 An appropriate allocation of function between users and technology
One of the most important human-centred design principles concerns the appropriate allocation of function – the
specification of which functions should be carried out by the users and which by the technology. These design
decisions determine the extent to which a given job, task, function or responsibility is to be automated or assigned
to human performance.
The decisions should be based on many factors, such as relative capabilities and limitations of humans versus
technology in terms of reliability, speed, accuracy, strength, flexibility of response, financial cost, the importance of
successful or timely accomplishment of tasks and user well-being. They should not simply be based on determining
which functions the technology is capable of performing and then simply allocating the remaining functions to users,
relying on their flexibility to make the system work. The resulting human functions should form a meaningful set of
tasks. Representative users should generally be involved in these decisions. For further guidance, see ISO 9241-2
and ISO 10075.
5.4 Iteration of design solutions
In iterative design approaches, feedback from users becomes a critical source of information. Iteration, when
combined with active user involvement, provides an effective means of minimizing the risk that a system does not
meet user and organizational requirements (including those requirements that are hidden or difficult to specify
© ISO
ISO 13407:1999(E)
explicitly). Iteration allows preliminary design solutions to be tested against “real world” scenarios, with the results
being fed back into progressively refined solutions.
Iteration can be incorporated in other design approaches. Even in the “waterfall” model, where there is a systematic
top-down hierarchy of design decisions and the relationship between the stages generally precludes iteration
between them, there can be extensive iteration within a stage.
5.5 Multi-disciplinary design
Human-centred design needs a variety of skills. A range of personnel is necessary to address the human aspects of
the design. This means that multi-disciplinary teams should be involved in a user-centred design process. These
can be small, dynamic and need only last the life of the project. The composition of the teams should reflect the
relationship between the organization responsible for technical development and the customer. The roles can
include the following
a) end-user;
b) purchaser, manager of user;
c) application domain specialist, business analyst;
d) systems analyst, systems engineer, programmer;
e) marketer, salesperson;
f) user interface designer, visual designer;
g) human factors and ergonomics expert, human-computer interaction specialist;
h) technical author, trainer and support personnel.
Individual team members can cover a number of different skill areas and viewpoints. Multi-disciplinary teams do not
have to be large but the team should be sufficiently diverse to make appropriate design trade-off decisions.
6 Planning the human-centred design process
A plan should be developed to specify how the human-centred activities fit into the overall system development
process.
The plan should identify:
a) the human-centred design process activities described in clause 7, i.e. understanding and identifying context of
use, specifying user and organizational requirements, producing prototypes and evaluating designs according
to user criteria;
b) procedures for integrating these activities with other system development activities, e.g. analysis, design,
testing;
c) the individuals and the organization(s) responsible for the human-centred design activities and the range of
skills and viewpoints they provide;
d) effective procedures for establishing feedback and communication on human-centred design activities as they
affect other design activities, and methods for documenting these activities;
e) appropriate milestones for human-centred activities integrated into the overall design and development
process;
f) suitable timescales to allow feedback, and possible design changes, to be incorporated into the project
schedule.
© ISO
ISO 13407:1999(E)
This human-centred design process plan should form part of the overall system development project plan and
should also be subject to the same project disciplines (e.g. responsibilities, change control) as other key activities to
ensure that it is followed through and implemented effectively. The plan should be revised as requirements change
and updated to reflect the status of activities.
Project planning should allow for iteration and for incorporating user feedback. Some time is also required for
effective communication among design team participants and for reconciling potential conflicts and trade-offs.
Projects benefit from additional creativity and ideas from the interaction of team members who, collectively have an
extensive skill base. Extra communication and discussion to identify and resolve problems early on in the project
can result in significant savings at later stages when changes are generally more costly.
Design organizations should incorporate human-centred design into their existing internal procedures and
development standards. This can include organization procedures for prototyping, for testing, for establishing
appropriate user involvement, for ensuring the right mix of skills and competence in the development team.
If the developing organization has a quality system and associated quality plans for system development, then a
specific plan should be included for the human-centred design process covering both the type of development
process adopted and the quality control measures.
7 Human-centred design activities
7.1 General
There are four human-centred design activities that should take place during a system development project.
These activities are
a) to understand and specify the context of use,
b) to specify the user and organizational requirements,
c) to produce design solutions,
d) to evaluate designs against requirements.
The human-centred design process should start at the earliest stage of the project (e.g. when the initial concept for
the product or system is being formulated), and should be repeated iteratively until the system meets the
requirements, as illustrated in Figure 1.
The need for a human-centred design approach will be identified from the operational objectives of the system, for
example, to satisfy customer requirements for usability.
When planning a system development project, the description of each activity and its sub-tasks should be studied
and used as guidance in designing or selecting the human-centred design methods and techniques for carrying out
the activity and reporting progress and findings. Whereas all of the human design activities described in this clause
are generally relevant, the relative focus and overall investment in them will depend on the size and type of the
product; for example, a large project, new product or new system could have a full multi-disciplinary team with a
member for each relevant role and implement all the human-centred design activities recommended in this clause.
In contrast, small projects, existing legacy products or systems or products targeted at niche or small markets could
have a smaller design team with individual members representing multiple roles and use a more limited range of
methods and techniques to support the activities.
© ISO
ISO 13407:1999(E)
Figure 1 — The interdependence of human-centred design activities
7.2 Understand and specify the context of use
7.2.1  The characteristics of the users, tasks and the organizational and physical environment define the context in
which the system is used. It is important to understand and identify the details of this context in order to guide early
design decisions, and to provide a basis for evaluation.
Information should be gathered about the context of use of new products and systems. If an existing system is to be
upgraded or enhanced, this information may already be available but should be checked. If there are extensive
results from user feedback, help desk reports and other data, these provide a basis for prioritizing user
requirements for system modifications and changes.
The context in which the system is to be used should be identified in terms of the following.
a) The characteristics of the intended users: relevant characteristics of the users can include knowledge, skill,
experience, education, training, physical attributes, habits, preferences and capabilities. If necessary, define
the characteristics of different types of users, for example, with different levels of experience or performing
different roles (maintainers, installers, etc.).
b) The tasks the users are to perform: the description should include the overall goals of use of the system. The
characteristics of tasks that can influence usability should be described, e.g. the frequency and the duration of
performance. If there are implications for health and safety, e.g. controlling the behaviour of a computer-
controlled production machine, these should also be described. The description should include the allocation of
activities and operational steps between the human and technological resources. Tasks should not be
described solely in terms of the functions or features provided by a product or system.
c) The environment in which the users are to use the system: the environment includes the hardware, software
and materials to be used. Their description can be in terms of a set of products, one or more of which can be
the focus of human-centred specification or evaluation, or it can be in terms of a set of attributes or
performance characteristics of the hardware, software and other materials.
© ISO
ISO 13407:1999(E)
Relevant characteristics of the physical and social environment should also be described. These can include
relevant standards, attributes of the wider technical environment (e.g., a local area network), the physical
environment (e.g., workplace, furniture), the ambient environment (e.g., temperature, humidity), the legislative
environment (e.g., laws, ordinances and directives) and the social and cultural environment (e.g., work practices,
organizational structure and attitudes).
7.2.2  The output from this activity should be a description of the relevant characteristics of the users, tasks and
environment which identifies what aspects have an important impact on the system design. (See ISO 9241-11 for
more information about the context of use and a sample report.)
NOTE This description is unlikely to be a single output that is issued once. It is more often a “working document” that is
first produced in outline terms and is then reviewed, maintained, extended and updated during the design and development
process.
The context of use description should
a) specify the range of intended users, tasks and environments in sufficient detail to support design activity;
b) be derived from suitable sources;
c) be confirmed by the users or if they are not available, by those representing their interests in the process;
d) be adequately documented;
e) be made available to the design team at appropriate times and in appropriate forms to support design activities.
7.3 Specify the user and organizational requirements
7.3.1  In most design processes, there is a major activity specifying the functional and other requirements for the
product or system. For human-centred design, this activity should be extended to create an explicit statement of
user and organizational requirements in relation to the context of use description. The following aspects should be
considered in order to identify relevant requirements:
a) required performance of the new system against operational and financial objectives;
b) relevant statutory or legislative requirements, including safety and health;
c) cooperation and communication between users and other relevant parties;
d) the users’ jobs (including the allocation of tasks, users’ well-being, and motivation);
e) task performance;
f) work design and organization;
g) management of change, including training and personnel to be involved;
h) feasibility of operation and maintenance;
i) the human-computer interface and workstation design.
7.3.2  User and organizational requirements should be derived and objectives set with appropriate trade-offs
identified between the different requirements. This specification should define the “allocation of function” — the
division of system tasks into those performed by humans and those performed by technology. These requirements
should be stated in terms that permit subsequent testing and should be confirmed or updated during the life of the
project.
NOTE Specific guidance on specifying software in a form that can be tested is contained in ISO/IEC 14598-1.
The specification of user and organizational requirements should
© ISO
ISO 13407:1999(E)
a) identify the range of relevant users and other personnel in the design,
b) provide a clear statement of the human-centred design goals,
c) set appropriate priorities for the different requirements,
d) provide measurable criteria against which the emerging design can be tested,
e) be confirmed by the users or those representing their interests in the process,
f) include any statutory or legislative requirements, and
g) be adequately documented.
7.4 Produce design solutions
7.4.1 General
Potential design solutions are produced by drawing on the established state of the art, the experience and
knowledge of the participants and the results of the context of use analysis. The process therefore involves the
following activities:
a) use existing knowledge to develop design proposals with multi-disciplinary input;
b) make the design solutions more concrete using simulations, models, mock-ups, etc.;
c) present the design solutions to users and allow them to perform tasks (or simulated tasks);
d) alter the design in response to the user feedback and iterate this process until the human-centred design goals
are met;
e) manage the iteration of design solutions.
7.4.2 Use existing knowledge to develop design proposals with a multi-disciplinary input
There is a substantial body of scientific knowledge and theory from ergonomics, psychology, cognitive science,
product design and other relevant disciplines that can indicate potential design solutions. Many organizations have
internal user interface style guides, product knowledge and marketing information which can be useful in supporting
the initial design, particularly when designing similar products. Generic human factors and ergonomics design
guidance and standards are also available from national and international standards bodies. See annex A for
relevant standards and bibliography for further sources of information.
7.4.3 Make the design solution more concrete using simulations, models, mock-ups, etc.
Using simulations, models and mock-ups or other forms of prototype allows designers to communicate more
effectively with users and reduces the need and cost of reworking that can occur when products need to be revised
later in the life cycle — in some cases after initial release to real customers.
The benefits are the following:
a) to make design decisions more explicit (this enables members of the design team to communicate with each
other early in the process);
b) to allow designers to explore several design concepts before they settle on one;
c) to make it possible to incorporate user feedback into the design early in the development process;
d) to make it possible to evaluate several iterations of a design and alternative designs;
e) to improve the quality and completeness of the functional design specification.
© ISO
ISO 13407:1999(E)
Prototyping can be carried out at most stages of design, from earliest design ideas based on the context of use
information (for example, using scenarios) to pre-production prototypes that are virtually complete in all details. A
prototype can be as simple as a pencil and paper sketch or as complex as a computer-based simulation, barely
distinguishable from the real thing.
7.4.4 Present the design solution to users and allow them to perform tasks (or simulated tasks)
Users can be involved very early in the design through the use of static, paper-based mock-ups. This could involve
presenting users with sketches of screen images of what a product/system is to look like and asking them to try
them out in a realistic context. Some aspects of the design (e.g., how easy it is to work with the menu hierarchies)
can then be quickly and inexpensively assessed. For hardware products, three-dimensional models constructed of
simple materials can yield similar benefits.
Simple prototypes are valuable at an early stage to explore alternative design solutions. Although there is benefit in
making the design solutions as realistic as possible, it is important not to invest so much time, money or
commitment on realistic prototypes, that there is reluctance to change the design.
In a human-centred approach, prototypes are not simply demonstrations to show users a preview of the design but
are used to collect user feedback that is then used to drive the design process.
If it is impractical to show prototypes to users early in the design process (for example, for reasons of
confidentiality), evaluation can be conducted by experts. Expert evaluation can be valuable and cost-effective and
can complement user testing. However, for a design process to be human-centred, the final testing (at least) should
take place with real users.
See 7.5 for details of design evaluation.
7.4.5 Alter the design in response to the user feedback and iterate this process until design objectives are
met
The level of prototype and the degree of iteration vary depending on several factors, including the importance
attached to optimizing the design. In software developments, prototyping can start with paper visualizations of
screen designs and progress through several stages of iteration to interactive software with just enough functionality
to support a subset of user tasks. Later in design, prototypes can be evaluated in a more realistic context. To obtain
the maximum benefits, it is best to carry out several iterations with users. In order to determine whether the overall
objectives have been met, more formal evaluation should be conducted in a realistic context, for example, without
help or interruptions from the evaluator.
User comments, and difficulties observed when using a prototype, offer guidance on functional design changes that
can improve system usability. In some cases such feedback can also help to refine the scope and purpose of an
interactive system (see 7.5.1)
7.4.6 Manage the iteration of design solutions
In order to manage the progress of iterative design, the results of activities 7.4.2 to 7.4.5 should be recorded. These
records can be wholly documentary or can include the design artefact itself, for example, some prototype hardware
or software. They include
a) the sources of existing knowledge and standards used, with an indication of how they have been incorporated
(or why they have not been followed, if appropriate),
b) the steps taken to ensure that the prototype covered key requirements and followed good practice, and
c) the nature of the problems identified and the subsequent changes to the design.
7.5 Evaluate designs against requirements
7.5.1 General
Evaluation is an essential step in human-centred design and should take place at all stages in the system life cycle.
Evaluation can be used:
© ISO
ISO 13407:1999(E)
a) to provide feedback which can be used to improve design,
b) to assess whether user and organizational objectives have been achieved, and
c) to monitor long-term use of the product or system.
Early in design the emphasis is on obtaining feedback that can be used to guide design, while later when a more
complete prototype is available it is possible to measure whether user and organizational objectives (see 7.3) have
been achieved.
In the early stages of the development and design process, changes are relatively inexpensive. The longer the
process has progressed and the more fully the system is defined, the more expensive the introduction of changes
is. It is therefore important to start evaluation as early as possible.
7.5.2 Evaluation plan
An evaluation plan should be developed which identifies the relevant aspects of the following;
a) the human-centred design goals;
b) who is responsible for the evaluation;
c) what parts of the system are to be evaluated and how they are to be evaluated, for example, the use of test
scenarios, mock-ups or prototypes;
d) how evaluation is to be performed and the procedures for carrying out the tests;
e) resources required for evaluation and analysis of results and access to users (as necessary);
f) scheduling of evaluation activities and their relation to the project timetable;
g) feedback and use of results in other design activities.
Evaluation techniques vary in their degree of formality, rigour and user involvement, depending on the environment
in which the evaluation is conducted. The choice is determined by financial and time constraints, the stage in the
development life cycle and the nature of the system under development.
7.5.3 Provide design feedback
Evaluations should take place at all stages in the system life cycle in order to influence the system to be delivered.
Particular evaluation goals should reflect one or more of the objectives below:
a) to assess how well the system meets its organizational goals;
b) to diagnose potential problems and identify needs for improvements in the interface, the supporting material,
the workstation environment or the training proposals;
c) to select the design option that best fits the functional and user requirements;
d) to elicit feedback and further requirements from the users.
Expert evaluation can be fast and economical and is good for identifying major problems but is not sufficient to
guarantee a successful interactive system. The standards and guidelines referenced in annex A and the
bibliography provide processes and criteria which can be used as a basis for this type of evaluation.
User-based evaluation can be used to provide feedback at any stage of design. In the early stages, users can be
involved in the evaluation of scenarios, simple paper mock-ups or partial prototypes (see 7.4.5 for details of
prototyping and iteration).
As design solutions become more developed, evaluations involving users are based on progressively more
complete and concrete versions of the system. When trying to improve a prototype to meet human-centred design
© ISO
ISO 13407:1999(E)
objectives, cooperative evaluation can be valuable, where the evaluator discusses problems with the user as they
occur. See the bibliography for sources of further information.
7.5.4 Assess whether objectives have been achieved
Evaluation can be used
a) to demonstrate that a particular design meets the human-centred requirements and
b) to assess conformity to international, national, local, corporate or statutory standards.
Further information on evaluation criteria can be found in the standards listed in annex A. To obtain valid results, the
evaluation should use appropriate methods, with a representative sample of users performing realistic tasks.
The choice of evaluation criteria for human-centred goals depends on the requirements for the product and the
needs of the organization setting the criteria. Objectives can relate to a primary goal (e.g. produce a letter) or a sub-
goal (e.g. successfully search and replace an item) or secondary goals (e.g. maintainability). Focusing objectives on
the most important user goals can mean ignoring other functions, but is normally the most practical approach.
Setting objectives for specific sub-goals can permit evaluation earlier in the development process. It may be
necessary to specify criteria both for the minimum acceptable levels and for the target levels to be achieved. For
further information, see ISO 9241-11.
7.5.5 Field validation
The aim of field validation is to test the functioning of the final system to ensure that it meets the requirements of the
users, the tasks and the environment. The main techniques which can be used include help desk data, field reports,
real user feedback, performance data, reports of health impacts, design improvements, and requests for changes.
7.5.6 Long-term monitoring
There should be a plan and a process for long-term monitoring of the use of the product or system. Systematic
collection of user input is needed as part of the design and evaluation activities in a human-centred design process.
Long-term monitoring means collecting user input in different ways, over a period of time. There is an important
difference between short-term evaluation and long-term evaluation. Some effects of working with an interactive
system are not recognizable until the system has been used for a period of time or there may be effects which result
from external factors, for example, unforeseen changes in working practices.
Performance criteria and company health reports can provide assessment parameters for the long-term evaluation
process. Attention to human-centred design principles during the design phase can identify those parameters most
important to assess. Performance criteria can be quite straightforward: does the system achieve its productivity
goals? Information can be gained from standard economic and marketing studies, analyses of support costs,
modification requests and other data.
Criteria and measurements should be sensitive enough to identify system failure, or system problems, at an early
stage.
NOTE Identifying unsafe behaviour is clearly preferable to registering accidents, and identifying mental or physiological
overload is preferable to registering medical disorders.
7.5.7 Reporting the results
7.5.7.1  In order to manage the progress of iterative design, the results of the evaluations should be recorded in a
systematic way. Annex B provides an example of the structure of a report used to provide feedback to design based
on user testing. If claims are to be made about a design process meeting the recommendations in this International
Standard, then those assessing the claims, whether customers, third-party assessors or the suppliers themselves
will require suitable evidence of adequate evaluation. See annex A and the bibliography for examples of standards
giving information on suitable evidence of adequate evaluation. In particular, there should be suitable evidence that
a) an adequate number of users took part in the testing and these users were representative of those identified in
the context of use,
b) there was testing of the major human-centred objectives,
© ISO
ISO 13407:1999(E)
c) there were valid testing and data collection methods,
d) there was an appropriate treatment of test results, and
e) the conditions of testing were appropriate.
There are three types of evaluation reporting that can be useful during the design process, depending on whether
the purpose of the evaluation is to feedback to design, to test against specific standards or to provide evidence of
achieving human-centred goals, for example, in terms of usability or user health and safety.
7.5.7.2  Reporting of feedback to design should
 take place at an appropriate time in the development process,
 be based on appropriate sources of evaluation e.g. users, design reviews,
 provide design feedback in a form which supports design decisions, and
 result in demonstrable changes in the system, where applicable.
7.5.7.3  Reporting on tests of the design against specific standards should
 identify relevant standards and provide a rationale for their use,
 provide evidence that the assessment was conducted by a competent person using appropriate procedures,
 provide evidence that sufficient parts of the system were tested to give meaningful results for the system as a
whole,
 report how non-conformities were dealt with in the design, and
 justify any deviations from applicable standards.
7.5.7.4  Reporting on user testing should
 define the context of use which was used for evaluation,
 provide information on the user and organizational requirements,
 describe the product tested and its status, e.g. production prototype,
 describe the measurements undertaken, users and methods used,
 contain results with relevant statistical analysis, and
 indicate a pass/fail decision in relation to the requirements.
8 Conformance
If
...