ETSI TR 101 329-6 V2.1.1 (2002-02)

SLOVENSKI STANDARD
01-april-2004
Harmonizacija telekomunikacij in internetnega protokola prek omrežij (TIPHON), 3.
izdaja - Kakovost storitve od konca do konca v sistemih TIPHON - 6. del: Dejanske
meritve parametrov karakteristik in zmogljivosti omrežja in terminalov v omrežjih
TIPHON ter njihov vpliv na kakovost govora
Telecommunications and Internet Protocol Harmonization Over Networks (TIPHON)
Release 3; End-to-end Quality of Service in TIPHON systems; Part 6: Actual
measurements of network and terminal characteristics and performance parameters in
TIPHON networks and their influence on voice quality
Ta slovenski standard je istoveten z: TR 101 329-6 Version 2.1.1
ICS:
33.020 Telekomunikacije na splošno Telecommunications in
general
2003-01.Slovenski inštitut za standardizacijo. Razmnoževanje celote ali delov tega standarda ni dovoljeno.

Technical Report
Telecommunications and Internet Protocol
Harmonization Over Networks (TIPHON) Release 3;
End-to-end Quality of Service in TIPHON systems;
Part 6: Actual measurements of network and terminal
characteristics and performance parameters in
TIPHON networks and their influence on voice quality

2 ETSI TR 101 329-6 V2.1.1 (2002-02)

Reference
RTR/TIPHON-05013
Keywords
IP, network, performance, QoS, quality, speech,
terminal, voice
ETSI
650 Route des Lucioles
F-06921 Sophia Antipolis Cedex - FRANCE

Tel.: +33 4 92 94 42 00  Fax: +33 4 93 65 47 16

Siret N° 348 623 562 00017 - NAF 742 C
Association à but non lucratif enregistrée à la
Sous-Préfecture de Grasse (06) N° 7803/88

Important notice
Individual copies of the present document can be downloaded from:
http://www.etsi.org
The present document may be made available in more than one electronic version or in print. In any case of existing or
perceived difference in contents between such versions, the reference version is the Portable Document Format (PDF).
In case of dispute, the reference shall be the printing on ETSI printers of the PDF version kept on a specific network drive
within ETSI Secretariat.
Users of the present document should be aware that the document may be subject to revision or change of status.
Information on the current status of this and other ETSI documents is available at
http://portal.etsi.org/tb/status/status.asp
If you find errors in the present document, send your comment to:
editor@etsi.fr
Copyright Notification
No part may be reproduced except as authorized by written permission.
The copyright and the foregoing restriction extend to reproduction in all media.

© European Telecommunications Standards Institute 2002.
All rights reserved.
ETSI
3 ETSI TR 101 329-6 V2.1.1 (2002-02)
Contents
Intellectual Property Rights.5
Foreword.5
Introduction .6
1 Scope.7
2 References.7
3 Abbreviations.9
4 List of Measurement Results.10
5 General Measurement Results.10
5.1 Subjective Testing.10
5.1.1 Simulation Results of VoIP Scenarios.10
5.1.1.1 Introduction.10
5.1.1.2 Measurement Set Up.11
5.1.1.2.1 Basics.11
5.1.1.2.2 Test Cases.11
5.1.1.3 Results.13
5.1.1.3.1 G.711 + Codec + G.711.13
5.1.1.3.2 Tandem Conditions with GSM x FR .13
5.1.2 Subjective Results on impairment effects of IP packet loss.14
5.1.2.1 Introduction.14
5.1.2.2 Measurement Set Up.14
5.1.2.2.1 Subjects.14
5.1.2.2.2 Speech Processing.14
5.1.2.2.3 Procedure.15
5.1.2.2.4 MOS Data and MNRU Conditions.15
5.1.2.3 Results.15
5.1.2.3.1 Reference Codec Conditions .15
5.1.2.3.2 Packet Loss Conditions.16
5.1.2.3.3 Tables of Results .17
5.2 Objective Testing.19
5.2.1 Objective Results of SuperOp event 1999 in Hawaii.19
5.2.1.1 Introduction.19
5.2.1.2 Measurement Set Up.19
5.2.1.3 Quality Assessment.20
5.2.1.4 Results.20
5.2.2 Objective Results of ETSI Speech Quality Test Event 2000 .20
5.2.2.1 Introduction.20
5.2.2.2 Measurement Set Up.21
5.2.2.3 Quality Assessment.22
5.2.2.4 TOSQA Results.24
5.2.2.4.1 G.711 codec.24
5.2.2.4.2 G.723 codec.24
5.2.2.4.3 G.729 codec.25
5.2.2.4.4 Summary and conclusion.25
5.3 Delay.25
5.3.1 Delay between two analogue PBX subscribers.25
5.3.1.1 Introduction.25
5.3.1.2 Measurement Set Up.26
5.3.1.3 Results.26
5.3.1.4 Conclusion.27
5.3.2 Delay between two PABX systems with analogue subscribers .27
5.3.2.1 Measurement Set Up.27
5.3.2.2 Result.28
5.3.3 Delay between two PC SW clients .28
ETSI
4 ETSI TR 101 329-6 V2.1.1 (2002-02)
5.3.3.1 Measurement Set Up.28
5.3.3.2 Results.28
5.3.4 Delay between an analogue PBX subscriber and a PC client .28
5.3.4.1 Measurement Set Up.28
5.3.4.2 Results.29
5.3.4.3 Conclusion.29
5.4 Echo.29
5.4.1 Echo between two analogue PBX subscribers.29
5.4.1.1 Measurement Set Up.30
5.4.1.2 Results.30
5.4.1.3 Conclusion.30
5.5 Impairment Factors.31
5.5.1 Transmission Impairments according to ITU-T G.113.31
5.6 R-Values.33
5.6.1 Analogue PBX- and SW Client scenarios.33
5.6.1.1 Introduction.33
5.6.1.2 Results.33
5.6.1.2.1 Two Gateways type A with analogue telephone sets.33
5.6.1.2.2 Gateway type A with analogue subscriber behind PBX.34
5.6.1.2.3 Gateway type B with analogue telephone sets on both sides.35
5.6.1.3 Conclusion.36
5.7 Advanced Measurement Techniques.36
5.7.1 QoS Measurements of IP-Configurations .36
5.7.1.1 Configuration and Measurement Set Up .36
5.7.1.2 Results.37
5.7.1.2.1 Parameters in Single Talk Conditions .37
5.7.1.2.2 Level dependant input-output characteristics .38
5.7.1.2.3 Echo loss and convergence.40
5.7.1.2.4 Performance in sending direction in the presence of background noise.41
5.7.1.2.5 Background noise performance in receiving direction .42
5.7.1.2.6 Evaluation of Double Talk Conditions .42
5.7.1.2.7 Relationship to subjective tests.44
5.7.1.3 Conclusion.44
5.7.2 QoS Measurements of ETSI Speech Quality Test Event 2000.44
5.7.2.1 Introduction.44
5.7.2.2 Measurement Set Up.44
5.7.2.3 Results.47
5.7.2.3.1 Packet loss.47
5.7.2.3.2 Transmission characteristics for background noise .48
5.7.2.3.3 Transmission performance under double talk performance.48
5.7.2.3.4 Detailed analysis of echo during double talk.50
5.8 Jitter Buffer.52
5.8.1 Simulation.52
5.8.1.1 Static Jitter Buffer .52
5.8.1.2 Dynamic Jitter Buffer.53
5.8.1.3 Comparison between Static and Dynamic Jitter Buffer Simulations .54
5.8.2 Practical Experiments.54
5.8.2.1 Introduction.54
5.8.2.2 Results.55
5.8.3 Conclusions.55
Annex A: Bibliography.56
History .57

ETSI
5 ETSI TR 101 329-6 V2.1.1 (2002-02)
Intellectual Property Rights
IPRs essential or potentially essential to the present document may have been declared to ETSI. The information
pertaining to these essential IPRs, if any, is publicly available for ETSI members and non-members, and can be found
in ETSI SR 000 314: "Intellectual Property Rights (IPRs); Essential, or potentially Essential, IPRs notified to ETSI in
respect of ETSI standards", which is available from the ETSI Secretariat. Latest updates are available on the ETSI Web
server (http://webapp.etsi.org/IPR/home.asp).
Pursuant to the ETSI IPR Policy, no investigation, including IPR searches, has been carried out by ETSI. No guarantee
can be given as to the existence of other IPRs not referenced in ETSI SR 000 314 (or the updates on the ETSI Web
server) which are, or may be, or may become, essential to the present document.
Foreword
This Technical Report (TR) has been produced by ETSI Project Telecommunications and Internet Protocol
Harmonization Over Networks (TIPHON).
The present document is part 6 of a multi-part deliverable covering End-to-end Quality of Service in TIPHON systems,
as identified below:
TR 101 329-1: "General aspects of Quality of Service (QoS)";
TS 101 329-2: "Definition of speech Quality of Service (QoS) Classes";
TS 101 329-3: "Signalling and control of end-to-end Quality of Service (QoS)";
TS 101 329-5: "Quality of Service (QoS) measurement methodologies";
TR 101 329-6: "Actual measurements of network and terminal characteristics and performance parameters
in TIPHON networks and their influence on voice quality";
TR 101 329-7: "Design guide for elements of a TIPHON connection from an end-to-end speech transmission
performance point of view".
Quality of Service aspects of TIPHON Release 4 and 5 Systems will be covered in TS 102 024 and TS 102 025
respectively, and more comprehensive versions of the Release 3 documents listed above will be published as part of
Release 4 and 5 as work progresses.
ETSI
6 ETSI TR 101 329-6 V2.1.1 (2002-02)
Introduction
The present document forms one of a series of technical specifications and technical reports produced by TIPHON
Working Group 5 addressing Quality of Service (QoS) in TIPHON Systems. The structure of this work is illustrated in
Figure 1.
Introduction Definition of 5
Speech Classes
TR 101 329-1 TS 101 329-2
General Speech
QoS
Aspects
of QoS Classes
SPEC
REPORT
Generic QoS
Specific Aspects of QoS
TS 101 329-5 TR 101 329-6 TR 101 329-7
TS 101 329-3
QoS Measure- Actual Design
Control ment Test Guidelines
Methods Results
SPEC REPORT REPORT
SPEC
QoS signalling Measurement Repository of Useful info for
requirements methodologies real test results designers

Figure 1: Structure of TIPHON QoS Documentation for Release 3
ETSI
7 ETSI TR 101 329-6 V2.1.1 (2002-02)
1 Scope
The present document applies to IP networks that provide voice telephony in accordance with any of the TIPHON
Scenarios.
The objective with the present document is to collect all results of various VoIP speech transmission quality tests and
related information. This collection should be used for information and to review and discuss the values of the TIPHON
QoS classes which are described in WG5 documents TR 101 329-1 [3] and TR 101 329-7 [6].
The separate measurements should give a very good opportunity to understand the goal of the measurement itself and
the exact measurement Set Up conditions to understand under which framework the measurements were done.
The present document covers measurement results provided to TIPHON during the years 1999 to 2001, which have
contributed to the measurement methodologies in 101 329-5 [5] as well as providing design parameters in
101 329-7 [6].
2 References
For the purposes of this Technical Report (TR) the following references apply:
[1] ETSI ETR 275 (1996): "Transmission and Multiplexing (TM); Considerations on transmission
delay and transmission delay values for components on connections supporting speech
communication over evolving digital networks".
[2] ETSI TR 101 329 (V2.1.1): "Telecommunications and Internet Protocol Harmonization Over
Networks (TIPHON); General aspects of Quality of Service (QoS)".
[3] ETSI TR 101 329-1: "Telecommunications and Internet Protocol Harmonization Over Networks
(TIPHON) Release 3; End-to-end Quality of Service in TIPHON systems; Part 1: General aspects
of Quality of Service (QoS)".
[4] ETSI TR 101 329-2: "Telecommunications and Internet Protocol Harmonization Over Networks
(TIPHON) Release 3; End-to-end Quality of Service in TIPHON systems; Part 2: Definition of
speech Quality of Service (QoS) classes".
[5] ETSI TR 101 329-5: "Telecommunications and Internet Protocol Harmonization Over Networks
(TIPHON) Release 3; End-to-end Quality of Service in TIPHON systems; Part 5: Quality of
Service (QoS) measurement methodologies".
[6] ETSI TR 101 329-7: "Telecommunications and Internet Protocol Harmonization Over Networks
(TIPHON) Release 3; End-to-end Quality of Service in TIPHON systems; Part 7: Design guide for
elements of a TIPHON connection from an end-to-end speech transmission performance point of
view".
[7] ETSI ES 201 168 (V1.1.1): "Corporate Telecommunication Networks (CN); Transmission
characteristics of digital Private Branch eXchanges (PBXs)".
[8] ETSI GTS 06.10 (V3.2.0): "European digital cellular telecommunications system (Phase 1); GSM
Full Rate Speech Transcoding (GSM 06.10)".
[9] ITU–T Recommendation G.107: "The E-Model, a computational model for use in transmission
planning".
[10] ITU–T Recommendation G.108: "Application of the E-model: A planning guide".
[11] ITU–T Recommendation G.109 (1999): "Definition of categories of speech transmission quality".
[12] ITU–T Recommendation G.113 (2001): "Transmission impairments due to speech processing".
[13] ITU–T Recommendation G.131 (1996): "Control of talker echo".
[14] ITU–T Recommendation G.165 (1993): "Echo cancellers".
ETSI
8 ETSI TR 101 329-6 V2.1.1 (2002-02)
[15] ITU–T Recommendation G.168 (2000): "Digital network echo cancellers".
[16] ITU–T Recommendation G.711 (1988): "Pulse code modulation (PCM) of voice frequencies".
[17] ITU–T Recommendation G.721 (1988): "32 kbit/s adaptive differential pulse code modulation
(ADPCM)".
[18] ITU–T Recommendation G.723.1 (1996): "Dual rate speech coder for multimedia communications
transmitting at 5.3 and 6.3 kbit/s".
[19] ITU–T Recommendation G.726 (1990): "40, 32, 24, 16 kbit/s adaptive differential pulse code
modulation (ADPCM)".
[20] ITU–T Recommendation G.727 (1990): "5-, 4-, 3- and 2-bit/ sample embedded adaptive
differential pulse code modulation (ADPCM)".
[21] ITU–T Recommendation G.728 (1992): "Coding of speech at 16 kbit/s using low-delay code
excited linear prediction".
[22] ITU–T Recommendation G.729 (1996): "Coding of speech at 8 kbit/s using conjugate-structure
algebraic-code-excited linear-prediction (CS-ACELP)".
[23] ITU–T Recommendation G.729A (Annex A - 1996): "Reduced complexity 8 kbit/s CS-ACELP
speech codec".
[24] ITU–T Recommendation G.729B (Annex B - 1996): "A silence compression scheme for G.729
optimized for terminals conforming to Recommendation V.70".
[25] ITU–T Recommendation H.323 (2000): "Packet-based multimedia communications systems".
[26] ITU-T Recommendation P.57: "Artificial ears".
[27] ITU-T Recommendation P.58: "Head and torso simulator for telephonometry".
[28] ITU–T Recommendation P.64 (1999): "Determination of sensitivity/frequency characteristics of
local telephone systems".
[29] ITU-T Recommendation P.501: "Test signals for use in telephonometry".
[30] ITU-T Recommendation P.502: "Objective test methods for speech communication systems, using
complex test signals".
[31] ITU-T Recommendation P.581: "Use of head and torso simulator (HATS) for hands-free terminal
testing".
[32] ITU–T Recommendation P.800 (1996): "Methods for subjective determination of transmission
quality".
[33] ITU–T Recommendation P.861 (1998): "Objective quality measurement of telephone-band
(300-3 400 Hz) speech codecs".
[34] ITU–T Recommendation Q.13/12: "Rapporteur of Question 13 inside ITU-T Study Group 12;
ETSI TIPHON 17TD135".
[35] SG 16, Santiago, Chile, 17-28 May 1999; D.249 (WP 3/16): "A High Quality Low-Complexity
Algorithm For Frame Erasure Concealment (FEC) With G.711" (Source: AT&T).
[36] T1A1.7/99-012r3; Jul-28-1999: "Draft Proposed American National Standard - A Packet Loss
Concealment Technique for Use with ITU-T Recommendation G.711" (Source: AT&T).
[37] ETSI ETS 300 581-2: "Digital cellular telecommunications system (Phase 2) (GSM); Half rate
speech; Part 2: Half rate speech transcoding (GSM 06.20 version 4.3.1)".
[38] ETSI EN 300 726: "Digital cellular telecommunications system (Phase 2+) (GSM); Enhanced Full
Rate (EFR) speech transcoding (GSM 06.60 version 8.0.1 Release 1999)".
ETSI
9 ETSI TR 101 329-6 V2.1.1 (2002-02)
[39] ITU-T Recommendation P.862 (2001): "Perceptual evaluation of speech quality (PESQ), an
objective method for end-to-end speech quality assessment of narrowband telephone networks and
speech codecs".
[40] ITU-T Recommendation G.723: "Extensions of Recommendation G.721 adaptive differential
pulse code modulation to 24 and 40 kbit/s for digital circuit multiplication equipment application".
[41] ITU-T Recommendation G.723.1-A: "Speech coders : Silence compression scheme".
[42] ITU-T Recommendation P.50: "Artificial voices".
[43] ITU-T Recommendation G.114: "One-way transmission time".
3 Abbreviations
For the purposes of the present document, the following abbreviations apply:
ACR Absolute Category Rating
ASL Active Speech Level
CAS Communication Analysis System (HEAD acoustics test system)
CSS Composite Source Signal
EC Echo Canceller
EP Error Pattern
GSM Global System for Mobile communications
GSM EFR GSM Enhanced Full Rate Speech Coder
GSM FR GSM Full Rate Speech Coder
HATS Head And Torso Simulator
IP Internet Protocol
IRS Intermediate Reference System
ISDN Integrated Services Digital Network
JLR Junction Loudness Rating
LAN Local Area Network
MNRU Modulated Noise Reference Unit
MOS Mean Opinion Score
NLP Non-Linear Processor
OLR Overall Loudness Rating
OVL Over-Load Point
PESQ Perceptual Evaluation of Speech Quality (see ITU-T Recommendation P.862)
PLC Packet Loss Concealment
PSTN Public Switched Telephone Network
PVS PC Voice Switch
QoS Quality of Service
RLR Receive Loudness Rating
SCN Switched Communications Network
SLR Send Loudness Rating
TMOS TOSQA Mean Opinion Score (output of TOSQA)
TOSQA Telecommunication Objective Speech Quality Assessment
VAD Voice Activity Detection
ETSI
10 ETSI TR 101 329-6 V2.1.1 (2002-02)
4 List of Measurement Results
Table 1: List of measurement results
Nr. Document Source Document Date
Introduction
1 Simulation Results of VoIP scenarios Deutsche Telekom Berkom ETSI TIPHON 11TD064 11/01/1999
t.scheerbarth@berkom.de;
i.kliche@berkom.de
2 APPENDIX I (to ITU-T Recommendation Mark E. Perkins ETSI TIPHON 11TD084 11/01/1999
G.113 [12] mperkins@att.com
3 Speech Quality Test results of IP Robert Bosch GmbH ETSI TIPHON 14TD081 16/07/1999
equipment in a LAN environment Joachim.Pomy@Tenovis.com
4 QoS Measurements of IP-Configurations HEAD acoustics, Robert Bosch ETSI TIPHON 05/10/1999
GmbH, T-Nova (Deutsche 15TD089
Telekom)
h.w.gierlich@head-acoustics.de
5 Subjective Results on impairment effects Nortel Networks ETSI TIPHON 14/03/2000
of IP packet loss paulcov@nortelnetworks.com 17TD167
6 Subjective and Objective Speech Quality Rapporteur of ITU-T ETSI TIPHON 15/03/2000
Evaluation on Speech Data recorded at Recommendation Q.13/12 [34] 17TD135
the SuperOp 99 event in Hawaii
7 Anonymous Test report of ETSI Speech Deutsche Telekom, T-Nova; ESTI TIPHON 22TD38 26/03/2001
Quality Test Event 2000 HEAD acoustics
8 Problems with the behaviour of jitter Pieter Veenstra ESTI TIPHON 22TD47 26/03/2001
buffers and their influence on the p.k.veenstra@kpn.com
end-to-end speech quality
5 General Measurement Results
5.1 Subjective Testing
5.1.1 Simulation Results of VoIP Scenarios
Source: Deutsche Telekom Berkom; Simulation Results of VoIP Scenarios; ETSI TIPHON 11TD064.
5.1.1.1 Introduction
ETSI TIPHON WG 5 has defined a methodology for testing VoIP End-to-End speech quality. This methodology was
used as a basis model for the T Berkom simulation processing. Figure 2 shows the methodology used for simulation.
ITU-T
Reference Codec
speech
G.711, G.726, GSM-FR,
signals
G.729
Reference
recorded speech Subjective
Acoustic
database Assessment
Device
Test Point
Electrical Acoustic
Acoustic Electrical
TIPHON Network
Part Part
Part Part
Terminal
Terminal
Simulation
Figure 2: Simulation methodology for testing TIPHON speech quality
ETSI
11 ETSI TR 101 329-6 V2.1.1 (2002-02)
A set of speech signals designed according to ITU-T Recommendation P.800 [32] was used as input of the simulation
path. The simulation path includes the terminal side (electrical part) and the network itself. The influence of the
terminal side was focussed to the speech conversion and IP packet size issue. The influence of the network side was
simulated by different packet loss rates.
After the simulation the speech samples were recorded and stored in a database.
The subjective assessment was carried out according to the ITU-T Recommendation P.800 [32] method.
5.1.1.2 Measurement Set Up
5.1.1.2.1 Basics
All source speech samples consisted of German sentences spoken by four talkers (2 male, 2 female). The input level
taken for all scenarios was ASL = -26 dB Ovl. In the pre- and post-processing phase the speech samples were filtered
with the modified IRS transmit and receive filter.
For every test condition the speech file was encoded and then assembled in IP packets. These IP packets were
assembled with different lengths, according to the concerned speech frame number per packet.
For simulation of network influences in the case of packet loss, a common channel model was designed, realized by
channel files which describe the network condition with the same time resolution as the source speech sample rate. So
the network has a certain condition (good or bad) for every speech sample (every 125 µs), two adjacent network states
were considered as statistically independent because the network speed was assumed to be much higher than the sample
rate (8 000 samples per second). So for each packet loss rate one channel file was created using a random generator.
The length of this channel file was exactly the same as the length of the speech file.
In a further step the speech file, assembled in IP packets, was matched to the channel file. According to the length of the
IP packet (10 ms, 20 ms,.) the channel file was checked every time when a packet was ready to send. That means if the
packet size was 10 ms the channel file was checked also every 10 ms if the condition is good or bad. In a bad case the
IP packet was lost, otherwise it was further processed.
This information (IP packet lost or not) was stored in a description file which was the input of the re-assembler and
speech decoder.
5.1.1.2.2 Test Cases
The test cases consisted a group of single codec scenarios (references), phone to phone scenarios in fixed network
environments and a group of tandeming conditions. The tandeming conditions based on real scenarios where a mobile
customer is connected to an ordinary telephone via IP. For this cases the GSM Full Rate Codec (GSM-FR) and the
GSM Enhanced Full Rate Codec (EFR) were used. In such scenarios mainly the influence of the IP network was taken
into account. Only one condition was chosen to simulate a voice transmission from a mobile phone to an ordinary
telephone via an impaired radio channel and via an IP network with packet loss.
Table 2: G.711 [16] + Codec + G.7xx (Phone to Phone Scenario in fixed network environments)
Codec Packet Loss Speech Frame Nr. of Substitution
Size Frames/Packet
G.711 [16] 5 %, 10 %, 15 %, 20 % 0,125 ms 80, 320, 480, 800 Silence
G.729B [24] 5 % 10 ms 1, 4, 6, 10 G.729 [22] internal
G.723.1 [18] (5.3) 0 %, 5 %, 10 %, 15 %, 20 % 30 ms 1, 2, 3 G.723.1 [18] internal
G.728 [21] 0 %, 5 %, 10 %, 15 %, 20 % 0,625 ms 16, 64, 96, 160 proprietary

ETSI
12 ETSI TR 101 329-6 V2.1.1 (2002-02)
logical scenario:
ISDN / PSTN ISDN / PSTN
IP
G.711- G.72x
Decoder Decoder
G.711 G.711
packet loss
Encoder Decoder
G.72x G.711-
Encoder
Encoder
TIPHON network class
simulation scenario:
Pre- G.711 Encoder packet loss Decoder G.711 Post-
Processing Enc + Dec G.72x generator G.72x Enc + Dec Processing
source destination
Figure 3: Processing scenario for single codec conditions
Table 3: Tandem Configuration with GSM x FR
(Phone to Phone Scenario in including mobile networks)
Tandem with GSM-FR Packet Loss Speech Frame Nr. of Substitution
Size Frames/
Packet
GSM FR + G.723.1 [18] (5.3) 0 %, 5 %, 10 %, 15 % G.723.1 [18]: 2 G.723.1 [18] internal
30 ms
GSM FR + G.729B [24] 0 %, 5 %, 10 %, 15 % G.729 [22]: 6 G.729 [22] internal
10 ms
GSM EFR + G.723.1 [18] (5.3) 0 %, 5 %, 10 % G.723.1 [18]: 2 G.723.1 [18] internal
30 ms
GSM EFR + G.729B [24] 0 %, 5 %, 10 % G.729 [22]: 6 G.729 [22] internal
10 ms
GSM EFR EP2 + G.723.1 [18] (5.3) 0 %, 5 %, 10 % G.723.1 [18]: 2 G.723.1 [18] internal
30 ms
logical scenario:
ISDN / PSTN ISDN / PSTN
IP
G.711- G.72x
Decoder
Decoder
G.711 G.711
packet loss
Encoder Decoder
G.711-
G.72x
Encoder Encoder
GSM xFR
Enc + Dec
TIPHON network class
simulation scenario:
Pre- GSM xFR G.711 Encoder packet loss Decoder G.711 Post-
Processing Enc + Dec Enc + Dec G.72x generator G.72x Enc + Dec
Processing
source destination
Figure 4: Processing scenario for tandem codec conditions
ETSI
13 ETSI TR 101 329-6 V2.1.1 (2002-02)
5.1.1.3 Results
The following figures illustrate the subjective assessment results. On the y axis the MOS score from 1 (bad) to 5
(excellent) is shown. The x-axis shows the various end-to-end scenarios.
5.1.1.3.1 G.711 + Codec + G.711
5,00
4,50
G.711
G.728
4,00
G.723(5.3)
1x ADPCM32
G.729
3,50
4x ADPCM32
3,00
2,50
2,00
1,50
1,00
0,50
0,00
Packet Loss % 5 10 20 5 10 20 5 10 20 5 10 20 0 0 5 10 20 5 10 20 5 10 20 0 0 5 10 5 10 5 10 5 10 ref ref ref ref 5 10 15 20 25 30 99
IP Packet (ms) 30 60 10 40 60 100
10 40 100 90
Codec
G.711 G.711-G.723(5.3)-G.711 G.711-G.728-G.711 MNRU

Figure 5: Subjective assessment result
The assessment of single codecs under the influence of packet loss leads to assumptions as follows:
• the packet loss rate of 5 % seems to be almost the quality threshold of MOS 3,0;
• in all test cases the evaluation of voice signals with packet loss of >= 10 % the MOS scores are widely below the
quality threshold of MOS 3,0.
5.1.1.3.2 Tandem Conditions with GSM x FR
5,00
4,50
GSM EFR
G.711
4,00
GSM FR
1x ADPCM 32
3,50
3,00
4x ADPCM 32
2,50
2,00
1,50
1,00
0,50
0,00
Packet Loss %
05 10 0 5 10 0005 10 05 10 05 10 refrefref 5 10 15 20 25 30 99
IP Packet: 60ms
GSM FR- GSM FR- GSM EFR- GSM EFR- GSM EFREP2-
MNRU
Codec G.729
G.723(5.3) G.729 G.723(5.3) G.729

Figure 6: Tandem conditions with GSM x FR
ETSI
14 ETSI TR 101 329-6 V2.1.1 (2002-02)
The assessment of tandem codecs under the influence of packet loss for the G.7xx codecs, leads to assumptions as
follows:
• the quality threshold for tandem connection of GSM FR and G.723.1 [18] can be seen with less than or max. 5 %
packet loss, under the precondition of an error free radio channel;
• the quality threshold for tandem connection of GSM FR and G.729 [22] can be seen in the range of << 5 %
packet loss, under the precondition of an error free radio channel;
• the quality threshold for tandem connection of GSM EFR and G.723.1 [18] can be seen in the range of 5 %
packet loss, under the precondition of an error free radio channel.
No acceptable speech transmission quality for tandem connection of GSM EFR and G.723.1 [18] for 0 % packet loss
can be provided if the radio channel induces errors.
5.1.2 Subjective Results on impairment effects of IP packet loss
Source: Nortel Networks; ETSI TIPHON 17TD167, Subjective Results on impairment effects of IP packet
loss.
5.1.2.1 Introduction
With the growing interest in voice transport over Internet (IP) networks (voice over IP or VoIP), it is important to
understand the effects of various impairments on voice quality. One of the important impairments is packet loss, which
can be produced when voice packets are lost or delivered too late to be useful. A subjective experiment was conducted
to investigate the effects of packet loss on voice quality. This experiment included a variety of codecs that are used in
VoIP applications and packet loss rates ranging from 0 % to 5 %. Loss rates greater than 5 % were not included in this
study because based on previous experience such large impairments in voice quality tend to skew subjective data.
5.1.2.2 Measurement Set Up
5.1.2.2.1 Subjects
Sixty-one listeners aged 16 to 68 years (mean age 37 years) participated in the experiment. All listeners were telephony
users with self-reported normal hearing drawn from Nortel Network's Subjective Assessment Lab subject pool.
5.1.2.2.2 Speech Processing
The source speech consisted of high-quality anechoic chamber recordings of North American English sentences spoken
by six talkers (3 male, 3 female). Each speech sample consisted of four sentences uttered by the same talker. All the
speech samples were transmit filtered before encoding and receive-filtered prior to being heard by listeners. The input
level of the speech signals to the codecs was -20 dBm0. For non-G.711 [16] codec conditions, the codecs received
G.711 [16] encoded/decoded speech as the input speech. The codecs tested in this experiment were G.711 [16],
G.729 [22], G.729A [23], G.723.1 [18], and GSM EFR (06.60) [38]. For G.711 [16] and G.729 [22], speech frame sizes
of 10 ms and 20 ms were tested, while G.723.1 [18] and GSM EFR were tested with their standard frame sizes (30 ms
and 20 ms respectively). For all the codecs with a Voice Activity Detection (VAD) feature (G.729 [22], G.729A [23].
GSM EFR, G.723.1 [18] ) the VAD was set to "OFF". In addition, two Packet Loss Concealment (PLC) schemes for
G.711 [16] were tested: one scheme described by AT&T in a submission to SG 16 [35] and proposed as an ANSI
standard and a second proprietary scheme developed by Nortel Networks.
For the packet loss conditions, frames were removed from the speech samples randomly with a frequency determined
by the test condition (e.g. 1 % of the frames). A voice activity detector was used to ensure that losses always occurred
during an active speech period. It is important to note that this technique for simulating packet loss may be different
from other studies, so cross-experiment comparisons should be done with caution. A different random mask file was
applied to the speech samples for each group of 3 listeners in order to randomize where the losses were occurring
during the 4-sentence sample.
ETSI
15 ETSI TR 101 329-6 V2.1.1 (2002-02)
5.1.2.2.3 Procedure
Samples were played back over one channel (one side) of high-fidelity headphones to simulate handset listening. The
samples were played back at 79 dB SPL, measured at the ear reference plane. Listeners heard one sample during each
trial, and entered their ratings by pressing a button on a response box. The order of presentation of the samples was
randomized and the ratings were stored in a file for later statistical analysis. Eight additional samples, which were not
counted in the data, were presented at the beginning of the session as a warm-up. All subjects heard and rated all the
conditions presented by six talkers, resulting in a total of 366 observations per condition.
Listeners rated the processed speech samples in an Absolute Category Rating (ACR) test. Samples were rated according
to the telephony 5-point scale (excellent, good, fair, poor, bad). Mean-opinion-scores (MOS) were computed from the
ratings assigned to each test case.
5.1.2.2.4 MOS Data and MNRU Conditions
The MOS and MNRU equivalence data for each of the conditions are summarized in clause 5.1.2.3. The results of
various conditions of interest are also summari
...