ISO/IEC TS 42119-2

ISO/IEC TS DTS 42119-2:2025(E)
ISO/IEC JTC1 JTC 1/SC 42/JWG 2
Secretariat: ANSI
Date: 2025-04-1405-26
Artificial intelligence — Testing of AI —
Part 2:
Overview of testing AI systems

DTSFDIS stage
© ISO/IEC 2025 – All rights reserved

ISO/IEC TSDTS 42119-2:2025(E:(en)
© ISO/IEC 2025
All rights reserved. Unless otherwise specified, or required in the context of its implementation, no part of this publication
may be reproduced or utilized otherwise in any form or by any means, electronic or mechanical, including photocopying,
or posting on the internet or an intranet, without prior written permission. Permission can be requested from either ISO
at the address below or ISO’s member body in the country of the requester.
ISO copyright office
CP 401 • Ch. de Blandonnet 8
CH-1214 Vernier, Geneva
Phone: + 41 22 749 01 11
EmailE-mail: copyright@iso.org
Website: www.iso.orgwww.iso.org
Published in Switzerland
© ISO/IEC 2025 – All rights reserved
ii
ISO/IEC TSDTS 42119-2:2025(E:(en)
Contents
Foreword . iv
Introduction . v
1 Scope . 1
2 Normative references . 1
3 Terms and definitions . 1
4 Abbreviated terms . 9
5 Introduction to AI systems and software testing . 9
5.1 General . 9
5.2 AI system life cycle . 10
5.3 AI system functional view . 10
5.4 Risk-based testing . 14
5.5 Test processes . 15
5.6 Test documentation . 17
5.7 Testing stakeholders . 17
6 Identifying risks in AI systems . 17
7 Test approaches for testing AI systems . 18
7.1 Introduction to test approaches for AI systems . 18
7.2 Test levels . 18
7.3 Test types . 19
7.4 Test design techniques and measures . 26
Annex A (informative) Introduction to software testing . 31
Annex B (informative) Characteristics of AI systems . 35
Annex C (informative) Example risk assessment . 36
Bibliography . 40

© ISO/IEC 2025 – All rights reserved
iii
ISO/IEC TSDTS 42119-2:2025(E:(en)
Foreword
ISO (the International Organization for Standardization) is a and IEC (the International Electrotechnical
Commission) form the specialized system for worldwide federation of national standardsstandardization.
National bodies (that are members of ISO member bodies). The workor IEC participate in the development of
preparing International Standards is normally carried out through ISO technical committees. Each member
body interested in a subject for which a technical committee has been established has the right to be
represented on that committee. Internationalby the respective organization to deal with particular fields of
technical activity. ISO and IEC technical committees collaborate in fields of mutual interest. Other international
organizations, governmental and non-governmental, in liaison with ISO and IEC, also take part in the work.
ISO collaborates closely with the International Electrotechnical Commission (IEC) on all matters of
electrotechnical standardization.
The procedures used to develop this document and those intended for its further maintenance are described
in the ISO/IEC Directives, Part 1. In particular, the different approval criteria needed for the different types of
ISO documentsdocument should be noted. This document was drafted in accordance with the editorial rules
of the ISO/IEC Directives, Part 2 (see www.iso.org/directives or www.iec.ch/members_experts/refdocs).
This document was prepared jointly by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittees SC 7 Software and systems engineering and SC 42, Artificial intelligence.
ISO and IEC draw attention to the possibility that the implementation of this document may involve the use of
(a) patent(s). ISO and IEC take no position concerning the evidence, validity or applicability of any claimed
patent rights in respect thereof. As of the date of publication of this document, ISO and IEC had not received
notice of (a) patent(s) which may be required to implement this document. However, implementers are
cautioned that this may not represent the latest information, which may be obtained from the patent database
available at www.iso.org/patents and https://patents.iec.ch. ISO and IEC shall not be held responsible for
identifying any or all such patent rights.
Any trade name used in this document is information given for the convenience of users and does not
constitute an endorsement.
For an explanation of the voluntary nature of standards, the meaning of ISO specific terms and expressions
related to conformity assessment, as well as information about ISO's adherence to the World Trade
Organization (WTO) principles in the Technical Barriers to Trade (TBT) see www.iso.org/iso/foreword.html.
In the IEC, see www.iec.ch/understanding-standards.
This document was prepared jointly by Joint Technical Committee ISO/IEC JTC 1, Information technology,
Subcommittees SC 7 Software and systems engineering and SC 42, Artificial intelligence.
A list of all parts in the ISO/IEC 42119 series can be found on the ISO and IEC websites.
Any feedback or questions on this document should be directed to the user’s national standards body. A
complete listing of these bodies can be found at www.iso.org/members.html.
© ISO/IEC 2025 – All rights reserved
iv
ISO/IEC TSDTS 42119-2:2025(E:(en)
Introduction
This document facilitates understanding of how ISO/IEC/IEEE 29119 parts -1, ISO/IEC/IEEE 29119-2,
ISO/IEC/IEEE 29119-3 and 4 and ISO/IEC 20246 apply to the testing of AI systems.
The purpose of ISO/IEC/IEEE 29119 (all parts) is to define an internationally agreed set of standards for
software testing that can be used by any organization when performing any form of software testing.
ISO/IEC/IEEE 29119-1 introduces software testing concepts, which can be applied to any AI system.
ISO/IEC/IEEE 29119-2 comprises test process descriptions that define the software test processes at the
organizational level, test management level and dynamic test levels. It supports dynamic testing, functional
and non-functional testing, manual and automated testing and scripted and unscripted testing, and can be
utilized for the testing of any software-based system, including AI systems.
ISO/IEC/IEEE 29119-3 defines software test documentation. The requirements specified for templates and
examples of test documentation defined in ISO/IEC/IEEE 29119-3 can be met in the test documentation for
any AI system.
ISO/IEC/IEEE 29119-4 defines test design techniques, which can be utilized for the testing of AI systems and
components.
ISO/IEC 20246 defines processes and templates for work product reviews, including inspections,
walkthroughs and technical reviews.
This document explains how ISO/IEC/IEEE 29119-2 can be adopted for the testing of AI systems or
components and how the test documentation templates defined in ISO/IEC/IEEE 29119-3 can be
implemented when testing AI systems or components. This document also explains how ISO/IEC 20246 can
be adopted for the review of AI systems and related documentation. This document is structured as follows:
— Clauses 1Clauses 1-4-4 define the Scope, Normativescope, normative references, Termsterms and
definitions and Abbreviatedabbreviated terms;
— Clause 5Clause 5 defines concepts of AI system architectures, the AI system life cycle and testing processes
and documentation;
— Clause 6Clause 6 explains how risk is identified for AI systems;
— Clause 7Clause 7 defines test approaches suitable for testing AI systems and components;
— Annexes A-CAnnexes A-C provide supporting details and examples.
The aim of the ISO/IEC 42119 series is to provide requirements and guidance on the testing of AI components
and systems.
Other parts of the ISO/IEC 42119942119 series include:
— — ISO/IEC TS 42119-3 describes approaches and provides guidance on processes for the verification and
validation analysis of AI systems;
— — ISO/IEC TS 42119-7 provides technology-agnostic guidance for conducting red teaming assessments
on AI systems;
— — ISO/IEC TS 42119-8 provides definitions, concepts, requirements and guidance related to assessing
prompt-based text-to-text AI systems that utilize generative AI.
© ISO/IEC 2025 – All rights reserved
v
ISO/IEC TSDTS 42119-2:2025(E:(en)
Artificial Intelligence intelligence — Testing of AI — —
Part 2:
Overview of testing AI systems
1 Scope
This document provides requirements and guidance on the application of the ISO/IEC/IEEE 29119 series to
the testing of AI systems. This document follows a risk-based approach and uses risks associated with AI
systems, and their development and maintenance, to identify suitable test practices, approaches and
techniques applicable to AI systems and their components. When the test practices, approaches and
techniques are already specified in the ISO/IEC/IEEE 29119 series, this document provides additional detail
and describes their application in the context of AI systems.
2 Normative references
The following documents are referred to in the text in such a way that some or all of their content constitutes
requirements of this document. For dated references, only the edition cited applies. For undated references,
the latest edition of the referenced document (including any amendments) applies.
ISO/IEC/IEEE 29119--2, Software and systems engineering — Software testing — Part 2: Test processes
ISO/IEC/IEEE 29119--3, Software and systems engineering — Software testing — Part 3: Test documentation
3 Terms and definitions
For the purposes of this document, the following terms and definitions apply.
ISO and IEC maintain terminology databases for use in standardization at the following addresses:
— ISO Online browsing platform: available at https://www.iso.org/obp
Field Code Changed
— IEC Electropedia: available at https://www.electropedia.org/
3.1 3.1
artificial intelligence
AI
research and development of mechanisms and applications of AI systems (3.3(3.3))
Note 1 to entry: Research and development can take place across any number of fields such as computer science, data
science, humanities, mathematics and natural sciences.
[SOURCE: ISO/IEC 22989:2022, 3.1.3]
3.2 3.2
AI model
machine-readable representation of knowledge (3.15(3.15))
Note 1 to entry: An ML model (3.19(3.19)) and the knowledge captured from experts as rules in an expert system are
both forms of AI model.
© ISO/IEC 2025 – All rights reserved
ISO/IEC TSDTS 42119-2:2025(E:(en)
3.3
AI system
3.3
artificial intelligence system
AI system
engineered system that generates outputs such as content, forecasts, recommendations or decisions for a
given set of human-defined objectives
Note 1 to entry: The engineered system can use various techniques and approaches related to artificial intelligence
(3.1(3.1)) to develop a model to represent data, knowledge (3.15(3.15),), processes, etc. which can be used to conduct
tasks.
Note 2 to entry: AI systems are designed to operate with varying levels of automation.
[SOURCE: ISO/IEC 22989:2022, 3.1.4]
3.4 3.4
bias
systematic difference in treatment of certain objects, people or groups in comparison
to others
Note 1 to entry: Treatment is any kind of action, including perception, observation, representation, prediction
(3.24(3.24)) or decision.
[SOURCE: ISO/IEC TR 24027:2021, 3.3.2, modified – remove Oxford comma removed in definition and note to
entry].]
3.5 3.5
classification model
machine learning model (3.19(3.19)) whose expected output for a given input is one or
more classes
[SOURCE: ISO/IEC 23053:2022, 3.1.1]
3.6 3.6
concept drift
phenomenon where the statistical properties of input data (3.
...