ETSI TS 101 329-5 V1.1.2 (2002-01)

SLOVENSKI STANDARD
01-april-2004
Harmonizacija telekomunikacij in internetnega protokola prek omrežij (TIPHON), 3.
izdaja - Kakovost storitve od konca do konca v sistemih TIPHON - 5.del:
Metodologije meritve kakovosti storitve (QoS)
Telecommunications and Internet Protocol Harmonization Over Networks (TIPHON)
Release 3; End-to-end Quality of Service in TIPHON systems; Part 5: Quality of Service
(QoS) measurement methodologies
Ta slovenski standard je istoveten z: TS 101 329-5 Version 1.1.2
ICS:
33.020 Telekomunikacije na splošno Telecommunications in
general
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.

Technical Specification
Telecommunications and Internet Protocol
Harmonization Over Networks (TIPHON) Release 3;
End-to-end Quality of Service in TIPHON systems;
Part 5: Quality of Service (QoS) measurement methodologies

2 ETSI TS 101 329-5 V1.1.2 (2002-01)

Reference
RTS/TIPHON-05008a
Keywords
internet, IP, methodology, quality, service,
system, telephony
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE

Tel.: +33 4 92 94 42 00  Fax: +33 4 93 65 47 16

Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88

Important notice
Individual copies of the present document can be downloaded from:
http://www.etsi.org
The present document may be made available in more than one electronic version or in print. In any case of existing or
perceived difference in contents between such versions, the reference version is the Portable Document Format (PDF).
In case of dispute, the reference shall be the printing on ETSI printers of the PDF version kept on a specific network drive
within ETSI Secretariat.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
http://portal.etsi.org/tb/status/status.asp
If you find errors in the present document, send your comment to:
editor@etsi.fr
Copyright Notification
No part may be reproduced except as authorized by written permission.
The copyright and the foregoing restriction extend to reproduction in all media.

© European Telecommunications Standards Institute 2002.
All rights reserved.
ETSI
3 ETSI TS 101 329-5 V1.1.2 (2002-01)
Contents
Intellectual Property Rights.5
Foreword.5
Introduction .6
1 Scope.7
2 References.7
3 Definitions, symbols and abbreviations .8
3.1 Definitions.8
3.2 Symbols.8
3.3 Abbreviations.9
4 Test Set-up for Terminals and Systems Including Terminals .10
5 Call Establishment Measurements .10
5.1 Start Dial Signal Delay.10
5.2 Post Dial Delay.10
5.3 Call Duration.11
5.4 Release on Request.11
6 Speech Quality Measurements .11
6.1 Subjective Speech Quality.11
6.2 Objective Speech Quality.12
6.3 Advanced Objective Speech Quality Parameters .12
6.4 Mean One Way Delay .13
6.5 Echo Path Loss .13
6.6 Loudness Ratings.13
6.7 Overall Transmission Quality Rating [R].14
7 Transport layer measurements.14
7.1 One way transmission time .14
7.2 Roundtrip transmission time.14
7.3 2 Point packet delay variation .15
7.4 1 Point packet delay variation .15
7.5 Network packet loss .15
7.6 Effective packet loss.15
7.7 Packet errors.16
7.8 Mis-sequenced packets.16
7.9 Voice client induced PDV.16
7.10 Packet loss correlation.16
8 QoS mechanism tests.17
8.1 Simulated media for QoS calibration .17
8.2 Passive media path monitoring for QoS .17
Annex A (normative): Call establishment measurements .18
Annex B (normative): Speech quality measurements.19
B.1 Delay measurement.19
B.2 Loudness rating.20
Annex C (normative): QoS mechanism tests .21
C.1 Simulated media for QoS calibration .21
Annex D (normative): Testing procedures for TIPHON terminals and systems .24
D.1 Introduction.24
ETSI
4 ETSI TS 101 329-5 V1.1.2 (2002-01)
D.2 Parameters, defining speech transmission quality including terminals.24
D.3 Test set-up for terminals and systems including terminals .25
D.3.1 Test Signals.25
D.4 Measurement of standard parameters.25
D.5 Advanced Measurements, Taking Into Account the Conversational Situation.26
D.5.1 Measurement set-up for objective tests .28
D.5.2 Practical realization of test signals .28
D.5.3 Test procedures.28
Annex E (informative): Method for determining an equipment impairment factor using
passive monitoring .29
E.1 Introduction.29
E.2 Passive QoS Monitor Framework .29
E.3 Determining equipment impairment factor for packet loss.30
E.4 Determining Equipment Impairment Factor for Packet Delay Variation.32
E.5 Measuring Delay.33
E.6 Determining Equipment Impairment Factor for CODEC .33
E.7 Determining Overall Equipment Impairment Factor .33
E.7.1 Determining Average Equipment Impairment Factor .33
E.7.2 Recency Effect.34
E.8 Use of the E Model.34
Annex F (informative): Bibliography.35
History .36

ETSI
5 ETSI TS 101 329-5 V1.1.2 (2002-01)
Intellectual Property Rights
IPRs essential or potentially essential to the present document may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (http://webapp.etsi.org/IPR/home.asp).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Foreword
This Technical Specification (TS) has been produced by ETSI Project Telecommunications and Internet Protocol
Harmonization Over Networks (TIPHON).
The present document is part 5 of a multi-part deliverable covering End-to-end Quality of Service in TIPHON systems,
as identified below:
TR 101 329-1: "General aspects of Quality of Service (QoS)";
TS 101 329-2: "Definition of speech Quality of Service (QoS) classes";
TS 101 329-3: "Signalling and control of end-to-end Quality of Service (QoS)";
TS 101 329-5: "Quality of Service (QoS) measurement methodologies";
TR 101 329-6: "Actual measurements of network and terminal characteristics and performance parameters in
TIPHON networks and their influence on voice quality";
TR 101 329-7: "Design guide for elements of a TIPHON connection from an end-to-end speech transmission
performance point of view".
Quality of Service aspects of TIPHON Release 4 and 5 systems will be covered in TS 102 024 and TS 102 025
respectively (see Bibliography), and more comprehensive versions of the Release 3 documents listed above will be
published as part of Release 4 and 5 as work progresses.
ETSI
6 ETSI TS 101 329-5 V1.1.2 (2002-01)
Introduction
The present document forms one of a series of technical specifications and technical reports produced by TIPHON
Working Group 5 addressing Quality of Service (QoS) in TIPHON Systems. The structure of this work is illustrated in
figure 1.
Introduction Definition of 5
Speech Classes
TR 101 329-1 TS 101 329-2
General Speech
QoS
Aspects
of QoS Classes
SPEC
REPORT
Generic QoS
Specific Aspects of QoS
TS 101 329-5 TR 101 329-6 TR 101 329-7
TS 101 329-3
QoS Measure- Actual Design
Control ment Test Guidelines
Methods Results
SPEC REPORT REPORT
SPEC
QoS signalling Measurement Repository of Useful info for
requirements methodologies real test results designers

Figure 1: Structure of TIPHON QoS Documentation for Release 3

ETSI
7 ETSI TS 101 329-5 V1.1.2 (2002-01)
1 Scope
The present document applies to IP networks that provide voice telephony in accordance with any of the TIPHON
scenarios.
It contains:
- test methodologies for end to end QoS parameters;
- test methodologies for network performance parameters.
It should be noted that the work has tried to reference already developed measurement techniques rather than defining
new techniques unnecessarily.
Background information and discussions are contained in the General Aspects of QoS document TR 101 329-1 [1].
2 References
The following documents contain provisions which, through reference in this text, constitute provisions of the present
document.
• References are either specific (identified by date of publication and/or edition number or version number) or
non-specific.
• For a specific reference, subsequent revisions do not apply.
• For a non-specific reference, the latest version applies.
[1] ETSI TR 101 329-1: "Telecommunications and Internet Protocol Harmonization Over Networks
(TIPHON) Release 3; End to End Quality of Service in TIPHON Systems; Part 1: General aspects
of Quality of Service (QoS)".
[2] ITU-T Recommendation P.800: "Methods for subjective determination of transmission quality".
[3] ITU-T Recommendation G.131: "Control of talker echo".
[4] Draft ITU-T Recommendation P.861: "Perceptual Evaluation of Speech Quality (PESQ), an
objective method for end-to-end speech quality assessment of narrowband telephone networks and
speech codecs".
[5] ITU-T Recommendation G.169: "Automatic level control devices".
[6] ITU-T Recommendation P.340: "Transmission characteristics of hands-free telephones".
[7] ITU-T Recommendation P.76: "Determination of loudness ratings; fundamental principles".
[8] ITU-T Recommendation P.79: "Calculation of Loudness Ratings for telephone sets".
[9] ITU-T Recommendation G.107: "The E-Model, a computational model for use in transmission
planning".
[10] ITU-T Recommendation G.108: "Application of the E-model: A planning guide".
[11] ITU-T Recommendation G.177: "Transmission planning for voice-band services over hybrid
Internet/PSTN connections".
[12] ITU-T Recommendation P.59: "Artificial conversational speech".
[13] ITU-T Recommendation P.501: "Test signals for use in telephonometry".
[14] ITU-T Recommendation P.502: "Objective test methods for speech communication systems, using
complex test signals".
ETSI
8 ETSI TS 101 329-5 V1.1.2 (2002-01)
[15] ITU-T Recommendation P.581: "Use of head and torso simulator (HATS) for hands-free terminal
testing".
[16] ITU-T Recommendation P.831: "Subjective performance evaluation of network echo cancellers".
[17] ITU-T Recommendation P.832: "Subjective performance evaluation of Hands-free Terminals".
[18] ITU-T Recommendation P.51: "Artificial mouths".
[19] ITU-T Recommendation P.57: "Artificial ears".
[20] ITU-T Recommendation P.58: "Head and torso simulator for telephonometry".
[21] ITU-T Recommendation P.64: "Determination of sensitivity/frequency characteristics of local
telephone systems".
[22] ITU-T Recommendation P.50: "Artificial Voices".
[23] ETSI TBR 8: "Integrated Services Digital Network (ISDN); Telephony 3,1 kHz teleservice;
Attachment requirements for handset terminals".
[24] ITU-T Contribution COM12-42 Federal Republic of Germany: "Listening only test results for
hands-free telephones and their dependence upon room surroundings (1997)".
[25] ITU-T Recommendation G.711: "Pulse code modulation (PCM) of voice frequencies".
[26] ITU-T Recommendation G.726: "40, 32, 24, 16 kbit/s Adaptive Differential Pulse Code
Modulation (ADPCM)".
[27] ITU-T Recommendation G.113 Appendix 1: "Transmission Impairments - Appendix I: Provisional
planning values for the equipment impairment factor I ".
e
[28] ITU-T Recommendation G.114: "One-way transmission time".
[29] ITU-T Contribution COM12-D139 France Télécom R&D (Q14/12): "Study of the relationship
between instantaneous and overall subjective speech quality for time-varying quality speech
sequences: influence of a recency effect (Delayed Contributions 9-18 May 2000)".
[30] ANSI T1A1.7/98-031: "Testing the quality of connections having time varying impairments".
[31] ITU-T Recommendations P.831: "Subjective performance evaluation of network echo cancellers".
[32] ITU-T Recommendations P.501: "Test signals for use in telephonometry".
[33] ITU-T Recommendations G.108: "Application of the E-model: A planning guide".
3 Definitions, symbols and abbreviations
3.1 Definitions
For the purposes of the present document, the following terms and definitions apply:
codec: combined speech encoder and decoder
3.2 Symbols
For the purposes of the present document, the following symbols apply:
ms milliseconds
s seconds
ETSI
9 ETSI TS 101 329-5 V1.1.2 (2002-01)
3.3 Abbreviations
For the purposes of the present document, the following abbreviations apply:
CRC Cyclic Redundancy Check
DTX Discontinuous Transmission
FFT Fast Fourier Transform
GSM FR Global System for Mobile, Full Rate codec
IP Internet Protocol
LR Loudness Ratings
LSTR Listener Sidetone Rating
MNRU Modulated Noise Reference Unit
MOS Mean Opinion Score
Nc Circuit Noise referred to the 0 dBr-point
OLR Overall Loudness Rating
PDD Post Dial Delay
PDV Packet Delay Variation
qdu Number of Quantizing Distortion Units
QoS Quality of Service
RLR Receive Loudness Rating
SCN Switched Circuit Network
SDSD Start Dial Signal Delay
SLR Send Loudness Rating
STMR Sidetone Masking Rating
TCLw Terminal Coupling Loss (weighted)
TELR Talker Echo Loudness Rating
UDP User Datagram Protocol
VoIP Voice over Internet Protocol
WEPL Weighted Echo Path Loss
ETSI
10 ETSI TS 101 329-5 V1.1.2 (2002-01)
4 Test Set-up for Terminals and Systems Including
Terminals
The general access to terminals is described in figure 2. The preferred way of testing is the connection of the terminal to
a network simulator or a complete network. When testing without acoustical access, the test sequences can be fed to the
electrical interface as indicated in figure 2. The test sequences are fed in either electrically, using a reference codec or
using the direct signal processing approach or acoustically using ITU-T specified devices such as artificial ear and
mouth according to the Recommendations P.51 [18], P.57 [19] and P.58 [20]. The positioning and set-up for handset
type telephones is described in ITU-T Recommendation P.64 [21], for hands-free type telephones the set-up is
described in ITU-T Recommendation P.581 [15]. The test set-up can be used on both sides of a connection if complete
configurations are tested.
Reference Codec
ITU-T ITU-T
G.711, G.726
test sequences test sequences
or GSM FR
(P.50, P.501) (P.50, P.501)
acoustical access
0 dBr TIPHON Terminal
Reference point
Electrical
Electrical Acoustic
Network Simulator
Part
Part Part
electrical access
NOTE: Packet loss distribution is for further study.

Figure 2: Methodology for testing TIPHON Terminal/Systems Speech Quality
5 Call Establishment Measurements
5.1 Start Dial Signal Delay
Definition
Time in milliseconds for the dial tone to be audible after the phone is placed off-hook from the idle state.
Test Metrics
- start dial signal delay/ms;
- percentage of calls with no dial tone.
Comments
None.
5.2 Post Dial Delay
Definition
Time in milliseconds between dialling the last digit and an audible tone being heard at the originating end. The audible
tone is typically ring-back or the engaged tone.
ETSI
11 ETSI TS 101 329-5 V1.1.2 (2002-01)
Test Metrics
- post dial delay/ms.
Comments
Some systems have shown to present the user with a ring-back tone before a connection has been established, this gives
the impression that the PDD is low. If the connection fails this is later switched to an engaged tone. This is unacceptable
operation and should be tested.
5.3 Call Duration
Definition
The time in seconds between bi-directional media path establishment and media path closure at both ends of the
connection.
Test Metrics
- Call duration/s (accurate to 1 ms);
- Percentage of premature releases.
Comments
The call duration information can be used to check billing system accuracy.
5.4 Release on Request
Definition
Check to identify that connection is released when placing phone on-hook.
Test Metrics
- Percentage of correctly terminated calls.
Comments
None.
6 Speech Quality Measurements
6.1 Subjective Speech Quality
Definition
A subjective quality measure, or Mean Opinion Score (MOS), is determined from performing a subjective test in
accordance with P.800 [2]. A MOS is an average opinion of quality for a system based on asking people their opinion of
quality under control conditions. Further evaluation procedures specifically for echo canceller and hands-free terminal
testing can be found in ITU-T Recommendations P.831 [16] and P.832 [17].
Test Metrics
- Listening quality absolute category rating (P.800 [2], annex B);
- Listening distortion category rating (P.800 [2], annex D).
ETSI
12 ETSI TS 101 329-5 V1.1.2 (2002-01)
Comments
A subjective test, for TIPHON QoS class classification, should include the following reference conditions:
- Clean speech;
- G.711 [25] with no additional distortions;
- G.726 [26] at 32 kbit/s with no additional distortions;
- GSM FR with no additional distortions;
- MNRU conditions (Q = 6, 12, 24 and 30).
6.2 Objective Speech Quality
Definition
A measure of speech quality by a computer based software program. The ETSI TIPHON project endorses the ITU-T
Recommendation P.861 [4].
Test Metrics
- Speech Quality MOS prediction (ITU-T Recommendation P.861 [4]).
Comments
It is paramount to use an appropriate speech or speech-like test signal. The signal should be at least 8 to 10 seconds in
duration to ensure both system stability and the opportunity for errors to be assessed.
6.3 Advanced Objective Speech Quality Parameters
Definition
Various measures of speech quality parameters based on test signals and procedures are described in ITU-T
Recommendations P.501 [13], P.502 [14] and P.340 [6].
Test Metrics
- Convergence parameters of echo cancellers (section 4 of ITU-T Recommendation P.502 [14]);
- Speech quality parameters during double talk (section 5 of ITU-T Recommendation P.502 [14]);
- Companding and AGC characteristics (section 6 of ITU-T Recommendation P.502 [14]);
- Quality of background noise transmission (section 7 of ITU-T Recommendation P.502 [14]);
- Switching parameters (section 8 of ITU-T Recommendation P.502 [14]).
Comments
The tests require various (speech-like) test signals as described in ITU-T Recommendation P.501 [13] and ITU-T
Recommendation P.50 [22]. The test duration should be at least 8 seconds to 10 seconds in duration to ensure both
system stability and the opportunity for errors to be assessed. In addition to the test method described in clause 6.2
individual parameters influencing the speech quality can be assessed, in single and double talk situations. The test
methodology allows the assessment of terminals as well as of network components and configurations.
Further details can be found in annex D.
ETSI
13 ETSI TS 101 329-5 V1.1.2 (2002-01)
6.4 Mean One Way Delay
Definition
Mean one way delay is the time taken in milliseconds for a test signal to go from the near-end voice test point, traverse
the network, get looped back at the far voice test point and arrive back at the near voice test point divided by two.
Test Metric
- Mean one-way delay/ms;
- Average of 10 delay measures or 90 % of largest delay (whichever is greatest).
Comments
VoIP systems exhibit bulk delay variations and therefore a number of delay measures should be made to have a
statistical average. Delays should be measured over 30 seconds.
One methodology to measure delay is described in annex E.
6.5 Echo Path Loss
Definition
The ratio of r.m.s values of the incident to reflected speech signals with the echo path delay removed.
Test Metrics
- Steady state residual and returned echo level (ITU-T Recommendation G.169 [5], test 1).
Comments
The electrical performance of echo cancellers in a TIPHON system should conform to ITU-T Recommendation
G.169 [5], guidelines for measuring acoustical echo are given in section 10 of ITU-T Recommendation P.340 [6].
6.6 Loudness Ratings
Definition
A Loudness Rating (LR) is a single-figure weighted average of the frequency-dependent loss between two reference
points.
Test Metrics
- SLR (ITU-T Recommendation P.76 [7] and P.79 [8]);
- RLR (ITU-T Recommendation P.76 [7] and P.79 [8]);
- OLR (ITU-T Recommendation P.76 [7] and P.79 [8]).
Comments
LR calculations are traditionally performed using sine waves placed at 1/3 octave centre frequencies. However, when
assessing complex non-linear systems there is a need to use a speech-like test signals to pass through low-bit rate
codecs. The LR calculations are performed on the speech-like signal by calculating Fast Fourier Transform based
1/3-octave parameters.
ETSI
14 ETSI TS 101 329-5 V1.1.2 (2002-01)
6.7 Overall Transmission Quality Rating [R]
Definition
The "Overall Transmission Quality Rating [R]" is the output of the e-model, a planning tool, which relates aspects of
telephony transmission performance to a single figure R. R is representative of a users perceived conversational
performance of a system. The E-model is described in ITU-T Recommendation G.107 [9] and guidance can be found in
ITU-T Recommendations G.108 [10] and ITU-T Recommendation G.177 [11].
Test Metrics
- R-value.
Comments
A default telephone-handset profile is used for TIPHON classification. This profile is based on a "traditional" telephone
handset by using the default values for e-model calculations. Acoustic characteristics of TIPHON terminals are not
considered in order to focus on the parameters specific to TIPHON network related issues (i.e. where TIPHON
networks differ from existing SCN networks).
One methodology to passively monitor the overall transmission quality is described in annex E.
7 Transport layer measurements
7.1 One way transmission time
Definition
Time in milliseconds between the emission of a signal and the time it is received, includes delays due to equipment
processing as well as propagation delay.
Test Metrics
- Mean packet transmission time/ms;
- Minimum and maximum packet transmission times/ms.
Comments
Measurement requires two synchronized test boxes.
7.2 Roundtrip transmission time
Definition
Time in milliseconds for a packet to be transmitted from host A and received at host B and to be re-transmitted from
host B and received back at host A.
Test Metrics
- Mean roundtrip packet transmission time/ms;
- Minimum and maximum packet transmission times/ms.
Comments
The reflection of a packet for roundtrip measurement should be at the protocol layer that the measurement is addressing.
ETSI
15 ETSI TS 101 329-5 V1.1.2 (2002-01)
7.3 2 Point packet delay variation
Definition
PDV is the difference between upper and lower percentiles on the packet delay distribution. 2pt PDV uses 2 monitoring
points. The measurement uses the difference between the inter-packet sending and inter-packet arrival times.
Test Metrics
- 2pt packet delay variation/ms.
Comments
Measurement requires two synchronized test boxes.
7.4 1 Point packet delay variation
Definition
PDV is the difference between upper and lower percentiles on the packet delay distribution. 1pt PDV uses only 1
monitoring point. The measurement is based on the inter-packet arrival times.
Test Metrics
- 1pt packet delay variation/ms.
Comments
Measurement requires a single test box and therefore no synchronization. This measure gives a clear illustration of the
end-systems view of PDV but cannot be used so easily to quantify where any PDV has occurred.
7.5 Network packet loss
Definition
Percentage of packets lost at an IP test point; this metric does not include any losses due to the end-terminal equipment.
Test Metric
- Percentage network packet loss;
- Total number of lost packets.
Comments
None.
7.6 Effective packet loss
Definition
Percentage of packets lost as measured at the input of the speech codec, affecting the speech coder performance.
Test Metric
- Percentage network packet loss;
- Total number of lost packets;
- Packet loss distribution.
ETSI
16 ETSI TS 101 329-5 V1.1.2 (2002-01)
Comments
None.
One methodology to measure effective packet loss is described in annex E.
7.7 Packet errors
Definition
Packets that fail the CRC when received at an IP test point.
Test Metric
- Percentage of errored packets;
- Total number of errored packets.
Comments
Errors in a data packet will normally result in a packet being dropped by the layer 2 protocol which have checksums for
the whole packet. However CRC can sometimes fail and this can be monitored using the test tools available.
7.8 Mis-sequenced packets
Definition
Out of sequence packets at the receiving IP test point.
Test Metrics
- Number of mis-sequenced packets.
Comments
A large number of mis-sequenced packets may indicate a congested network or that load balancing is in use.
7.9 Voice client induced PDV
Definition
A measure of the inter-packet delay variations as packets are transmitted onto the network by a voice client.
Test Metrics
- Client transmit PDV/ms.
Comments
Client Induced PDV is a significant contributory factor in the total delay variation experienced on a connection.
7.10 Packet loss correlation
Definition
A description of the "burstiness" of packet losses at a test point.
Test Metrics
- Average number of successive lost packets;
- Distribution of burst loss lengths;
ETSI
17 ETSI TS 101 329-5 V1.1.2 (2002-01)
- Markov loss model (as described in annex E).
Comments
None.
8 QoS mechanism tests
8.1 Simulated media for QoS calibration
Definition
A simulated media stream is used to determine a network's ability to deliver a required QoS level.
Test Metrics
- Delay variation;
- Packet loss;
- Packet loss correlation;
- Packet delay.
Comments
The use of a simulated media stream ensures that the QoS mechanism is fully tested for it is in-service use.
8.2 Passive media path monitoring for QoS
Definition
A non-intrusive monitoring of media paths to determine customers QoS.
Test Metrics
- Packet loss;
- Packet loss correlation;
- Delay variation.
Comments
None.
One methodology to estimate effective packet loss and the overall transmission rating is described in annex E.
ETSI
18 ETSI TS 101 329-5 V1.1.2 (2002-01)
Annex A (normative):
Call establishment measurements
This annex provides a description of how to calculate the call establishment measurements in described in clause 5.
From a TIPHON QoS perspective call set-up measures are generally time related, although there are other equally
important measures such as the number of correctly connected calls. With a TIPHON system, there are two different
scenarios to consider:
- The traditional telephone user;
- The TIPHON terminal user.
Figure A.1 shows the call set-up sequence used by general telephony services. However, with TIPHON terminal
equipment, such as PC clients, there is likely to be no off-hook, dial tone sequence. For this situation the act of a user
pressing the "connect button" is regarded as step C - last digit dialled.
NOTE: Connect button describes the process by which a user instigates a call.
time
AB C D D EF
1 2
Figure A.1: Call set-up sequence
From the users perspective the significant time sequences are:
Start dial signal delay (SDSD):  B - A
Post dial delay (PDD):   D - C
NOTE 1: Traditionally, call set-up progression is signified by audible tones, this is now being supplemented or
replaced by text based messages.
NOTE 2: It is worth noting that there should not be a significant delay between D and D , delays can result in the
1 2
receiving party answering the phone before the calling party is aware a connection has been made.
Similarly it is inappropriate to allow D to occur before the far-end connection has been identified as
being accessible; either in terms that it exists or in that it is not engaged.
Measurement of these set-up times shall be based on the progression mechanisms presented to the user, and not lower
level signalling information.
ETSI
off-hook
dial tone
last digit dialled
far-end rings
near-end rings
call answered
nn ct o
co e i n
19 ETSI TS 101 329-5 V1.1.2 (2002-01)
Annex B (normative):
Speech quality measurements
B.1 Delay measurement
A single assessment of a TIPHON system's delay is inadequate. For VoIP systems, it is important to determine a
statistical average of the delay.
It is proposed that TIPHON use the mean delay from at least 10 measurements or 90 % of the largest delay measure,
whichever is greatest.
The delay measurement test signal is illustrated in figure B.1.
Talk 1 Silence Talk2 Silence
15s 5s 30s 10s
pauses
talk
spurts
Figure B.1: Delay measurement test signal composition
The test signal contains periods of speech activity (Talk1 and Talk2) and periods of silence. Talk 1 is an initialization
sequence, allowing the dynamic jitter buffer to converge.
Both Talk1 and Talk2 contain periods of talk spurts and silence intervals. This is important because jitter buffers are
generally designed to adjust their length, so altering the delay of a system, during silence intervals. In ITU-T
Recommendation P.59 [12], the average measured talk spurt is 1,0 seconds and the average pause is 1,6 seconds. It is
further recommended that the silence intervals are at least 300 ms long.
Delay assessments, using cross-correlation or another appropriate technique, will be made for each talk spurt. At least
10 measurements are required to determine the TIPHON delay measure during "pseudo-stable" operation, which
implies 10 opportunities for the jitter buffers to adjust. Therefore, the Talk2 period will contain at least 11 talk spurts.
The measured stable delay for the TIPHON system is the mean delay from all measurements (at least 10) during Talk2
or 90 % of the largest delay measure, whichever is greatest.
Delay measures should be performed for each speech burst during the Talk1 period. Although these measurements are
not used for TIPHON classification, they are important, as a slow convergence time maybe unacceptable to a user.
(Study is required into jitter buffer convergence time effects on perceived performance).
NOTE 1: For the delay measurement the talk spurts consist of either a speech-like test signal or natural speech.
NOTE 2: Delay measurements should be accurate to within ±5 ms.
ETSI
20 ETSI TS 101 329-5 V1.1.2 (2002-01)
B.2 Loudness rating
LR calculations are traditionally performed using sine waves placed at 1/3 octave centre frequencies. However, when
assessing complex non-linear systems there is a need to use a speech-like test stimulus to pass through devices such as
low-bit rate codecs. Since Loudness Ratings are a measure of frequency-dependent loss, it is possible to use wide-band
signals to obtain equivalent results. When using a speech-like test stimulus it is important to ensure a reasonable degree
of spectrum coverage in the reference signal.
Offsets between reference and recorded signals should be removed and a Fast Fourier Transform (FFT) performed on
each. The FFT signals should then be divided into 1/3 octave bands and the loss in each band calculated. These losses
can then be used in the LR formulas.
NOTE: The use of the artificial test stimulus to determine LR will be more susceptible to error from circuit noise
contributions at higher frequencies.
ETSI
21 ETSI TS 101 329-5 V1.1.2 (2002-01)
Annex C (normative):
QoS mechanism tests
C.1 Simulated media for QoS calibration
The purpose of this measurement is to allow intrusive testing of network performance for determining network class.
A measurement is made by transmitting a simulated VoIP media stream between two measurement hosts. It is important
that the simulated stream represents the in-service VoIP traffic for which the network will transport. The measurement
is round-trip (e.g. MH1 - MH2 - MH1) to emulate a full-duplex VoIP call, but one-way measurements (MH1-MH2 and
MH2-MH1) can be performed as a part of the round-trip measurement. The measurement hosts can be located within
the same domain, such as the core network as shown in figure C.1, or different domains. The measurement
methodology is independent of where in the end-to-end path the measurement hosts are located. Locating the
measurement hosts in different parts of the network allows for various parts of a network to be qualified.
End-to-end
Terminal 1 Access network 1 Core network Access network 2 Terminal 2
AN 1 core network AN 2
VoIP
terminal
multimedia
workstation
MH 1 MH2
Figure C.1: A test scenario for determining core-network QoS level
The measurement is performed in an application-level fashion using the same protocol stack as a VoIP application
(figure C.2). The streams are transmitted through the UDP interfaces in both MH1 and MH2.
UDP UDP
IP IP IP
LL / PHY LL / PHY LL / PHY
MH1 router MH2
Figure C.2: Protocol stack view of the application-level measurement principal
ETSI
22 ETSI TS 101 329-5 V1.1.2 (2002-01)
Simulated Stream Definition and Measurement Process
The simulated media stream needs to be representative of packet sizes, transmission intervals, codec type and more. For
a constant bit-rate traffic stream the following parameters need defining:
- Packet size.
- Packet transmission interval.
- Number of packets transmitted.
If discontinuous transmission (DTX) is to be simulated then talkspurt/silence alternation characteristics must be
included in the stream description.
An example test stream is a 20 ms framed, continuously transmitted, ITU-T Recommendation G.711 [25] stream. This
can be simulated by transmitting 200-byte IP packets (the size includes payload and protocol headers) every 20 ms.
- Payload (20 ms of speech)  = 160 bytes
- RTP     = 12 bytes
- UDP + IP headers    = 28 bytes
Timing and sequence information is then inserted into each packet as the packets are transmitted between the two hosts
as shown in figure C.3.
Measurement
IP UDP dummy length
data
TS1 TS2 TS3 TS4 SEQ 1 SEQ 2
Figure C.3: Structure of a measurement packet
The appropriate protocol layers add IP and UDP headers and padding is used to make the measurement packet
correspond to requirements of the simulated stream specifications. The TS and SEQ fields are filled as the packet is
transmitted between MH1 and MH2. Here is the process by which these are filled.
Field Description Set
TS1 Transmission timestamp at host 1 Set immediately before sending packet to host 2
through UDP interface
TS2 Reception timestamp at host 2 Set immediately after receiving packet at host 2
through UDP interface
TS3 Transmission timestamp at host 2 Set immediately before sending packet to host 1
through UDP interface
TS4 Reception timestamp at host 1 Set immed
...