VoLTE KPIs

I have never planned to talk about such an operator specific matter as KPIs. But since I posted NEWS: Telco Monitoring I’m receiving many questions related to this topic and I guess we can discuss at least the basic principles and standards.

Inside AT&T’s Network Operations Center by PCWorldVideos

If you have read the VoLTE standards such as GSMA IR.92 or VoLTE Service Description and Implementation Guidelines, you probably noticed that performance monitoring is more or less ignored. And at the same time all operators are asking about it. What KPIs to watch, how, what are the guidelines?

Btw. I always enjoy being in NOC (Network Operation Centre) or war or crisis rooms. Especially during events like NYE. However mostly it is not allowed to take any photos there, so instead I’ve linked some youtube videos showing the scene. Respect to you bros working day and night to keep the network running!

IMS KPIs

The basic 3GPP specification defining KPI templates is the 3GPP 32.410. This and some other documents define, what we are supposed to understand as “KPI”. So KPI stands for Key Performance Indicator. Each (telco) system provides some measurements/counters which help us to identify the performance and condition of it. These KPIs can be of several types:

  • RATIO: This KPI indicates the percentage of a specific case occurrence to all the cases.
  • MEAN: This KPI shows a mean measurement value based on a number of sample results.
  • CUM: This KPI reflects a cumulative measurement which is always increasing.

The performance is then measured in several categories:

  •    Serveability
  •    Accessibility
  •    Retainability
  •    Integrity
  •    Availability
  •    Reliability
  •    Maintainability
  •    Utilization
  •    Mobility

When a KPI is defined, it is classified into one of the listed category. More in the document ITU-T Recommendation E.800: “Terms and definitions related to quality of service and network performance including dependability”.

Finally KPIs can be calculated for different telco subsystems – e.g.  UTRAN, GERAN, CS/PS Core, IMS.

  • TS 32.401: Performance Management (PM); Concept and requirements
  • TS 52.402: Performance Management (PM); Performance measurements – GSM
  • TS 32.404: Performance Management (PM); Performance measurements – Definitions and template
  • TS 32.405: Performance Management (PM); Performance measurements UTRAN
  • TS 32.406: Performance Management (PM); Performance measurements Core Network (CN) PS domain
  • TS 32.407: Performance Management (PM); Performance measurements Core Network (CN) CS domain
  • TS 32.408: Performance Management (PM); Performance measurements Teleservice
  • TS 32.409: Performance Management (PM); Performance measurements IMS
  • TS 38.913: 5G KPIs

The specification 3GPP 32.409 contains definitions of IMS related KPIs. Howto count a particular KPI is then described in 3GPP 32.454. And these two documents are probably the ones you are looking for.

So let’s have an example. The first KPI described in 32.454 is the Initial Registration Success Rate of S-CSCF (IRSR). This KPI is useful to evaluate accessibility performance provided by IMS and overall network performance. It is defined as a number of Successful Initial Registrations divided by Attempted Initial Registrations.

Initial Registration Success Rate

Sounds logical. However from the practical point of view it has several implications. Firstly a particular S-CSCF not always directly provides such a KPI (differs vendor to vendor). But it has to provide two cumulative counters for Successful Initial Registrations and Attempted Initial Registrations. The exact definition of  these counters is then in the 32.409.

Initial Registration Success Rate Measurements

The list of standardized IMS KPIs:

Accessibility KPIs

  • Initial Registration Success Rate of S-CSCF
    • Useful to evaluate accessibility performance provided by IMS and network performance.
  • Session Setup Time (Mean)
    • Indicates accessibility performance provided by IMS and network performance. This KPI can influence the users’ satisfaction directly and reflect network transaction performance.
  • Session Establishment Success Rate
    • Important in order to evaluate the session establishment performance including user behaviour factors. It is necessary to define session establishment success rate (SESR) with differentiating originating and terminating, otherwise when the operator wants to consider SESR from whole IMS network perspective, the value of SESR will be much bigger than real situation due to one success session being counted twice (both at originating side and terminating side).
  • Third Party Registration Success Rate
    • It is defined to fulfil the need of the operator to evaluate the service and network accessibility performance.
  • Re-registration Success Rate of S-CSCF
    • This KPI is useful for evaluate accessibility of the IMS network, including the user behaviour factors.
  • Session Setup Time Originated from IMS (Mean)
  • Session Setup Time Originated from CS (Mean)
  • Immediate Messaging Success Rate
  • Session Establishment Network Success Rate 
    • KPI is focusing on network view different from the KPI Session Establishment Success Rate. Therefore the measurements on the number of release before ringing, UE not found, UE address incomplete and UE busy should be excluded from the failed measurements. This KPI is helpful to evaluate the real network session establishment success rate.

Retainability KPI

  • Call Drop Rate of IMS Sessions
    • It is important in order to evaluate retainability performance of IMS including user causes.

Utilization KPI

  •  Mean Session Utilization 
    • The mean number of simultaneous online and answered sessions together with Maximum Number of Sessions can reflect system resource utilization. If the value of this KPI is very high, it indicates system capacity is not sufficient and should be increased.

Inside Verizon’s Super Bowl Command Center by IDG.tv

VoLTE KPIs

VoLTE defines parameters for Global Roaming Quality (GRQ). They are described in IR.42 Definition of Quality of Service parameters and their computation and IR.81 GRQ Measurement Implementation. Similarly to 3gpp specifications, IR.81 defines KPIs and IR.42 the formulas how to calculate them. Althought they are defined primarily for Roaming scenarios, they can be a good lead for non-roaming cases too.

VoLTE GRQ parameters

QoS Aspects GRQ Id KPI description
VoLTE service Accessibility 201
202
IMS Registration success ratio
IMS Registration time
VoLTE service integrity and retainability 203
204
205
206
207
208
211
212
213
210
214
215
216
217
Voice MO accessibility (NER-MO or SAT-MO success ratio)
Voice MT accessibility (NER -MT or SAT-MT success ratio)
Voice MO session setup time (PDD-MO or STT-MO duration)
Voice MT session setup time (PDD-MT or STT-MT duration)
Voice MO session setup ratio (CSSR-MO)
Voice MT session setup ratio (CSSR-MT)
Voice MO session duration
Voice MT session duration
OIP transparency MO (CLI transparency)
OIP transparency MT (CLI transparency)
Speech quality on call basis at R-party
Speech quality on call basis at H-party
Speech quality R-factor at R-party
Speech quality R-factor at H-party
VoLTE service mobility 230
231
232
233
SRVCC MO success ratio
SRVCC MT success ratio
SRVCC MO time
SRVCC MT time
LTE network quality for
VoLTE service
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
Default EPS bearer context activation success ratio
Default EPS bearer context activation time
Default EPS bearer QCI
Default EPS bearer UL AMBR
Default EPS bearer DL AMBR
Dedicated EPS bearer context activation success ratio (audio)
Dedicated EPS bearer context activation time (audio)
Dedicated EPS bearer QCI (audio)
Dedicated EPS bearer UL GBR (audio)
Dedicated EPS bearer DL GBR (audio)
IP data volume received on QCI5 bearer at R-party
IP data volume transmitted on QCI5 bearer at R-party
IP data volume received on QCI5 bearer at H-party
IP data volume transmitted on QCI5 bearer at H-party
IP data volume received on QCI1 bearer at R-party
IP data volume transmitted on QCI1 bearer at R-party
IP data volume received on QCI1 bearer at H-party
IP data volume transmitted on QCI1 bearer at H-party
Voice media transport
quality for VoLTE service
260
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
261
262
RTP max packet delay variation R2H (A-B)
RTP mean packet delay variation H2R (B-A)
RTP mean interarrival jitter of incoming streaming R2H (A-B)
RTP mean interarrival jitter of incoming streaming H2R (B-A)
RTP mean data rate transmitted R2H (A-B)
RTP mean data rate received H2R (B-A)
RTP mean data rate transmitted H2R (B-A)
RTP mean data rate received R2H (A-B)
RTP packets lost R2H (A-B)
RTP packets lost H2R (B-A)
RTP packet lost ratio R2H (A-B)
RTP packet lost ratio H2R (B-A)
RTP round-trip delay (RTD RHR A-B-A)
RTP round-trip delay (RTD HRH B-A-B)
RTP one-way delay (OWD R2H A-B)
RTP one-way delay (OWD H2R B-A)
RTP max packet delay variation H2R (B-A)
RTP mean packet delay variation R2H (A-B)

VoLTE KPIs

GRQ Test Code Parameter (KPI) Test Method for Implementation
201AH,
201AV
IMS Registration success
ratio
Initiate an IMS registration in VPMN for IMS
multimedia telephony service (VoLTE) at test probe.
Observe whether the registration is successfully
completed by receiving NOTIFY for registration event
package from P-CSCF. The test probe shall
subscribe to its registration event package (ref. to TS
34.229-1, C.2).
To ensure a full IMS registration, initiate an (UE)
deregistration if it is already IMS registered.
202AH,
202AV
IMS Registration time Initiate an IMS registration in VPMN for IMS
multimedia telephony service (VoLTE). Measure the
time between the test probe sending the initial
REGISTER request for IMS registration and
receiving NOTIFY for registration event package
from P-CSCF. The test probe shall subscribe to its
registration event package (ref. to 3GPP TS 34.229-
1, C.2 ).
To ensure a full IMS registration, initiate an (UE)
deregistration if it is already IMS registered.
203AH,
203AV
Voice MO accessibility Initial a VoLTE MO call from a test probe at VPMN to
HPMN. The call is successful if 180 Ringing is
received at VPMN. The test probe at HPMN shall
send 180 Ringing unreliably i.e. not containing
“Require: 100rel”.
204AH,
204AV
Voice MT accessibility Initial a VoLTE MT call from HPMN to a test probe at
VPMN. The call is successful if 180 Ringing is
received at HPMN from the test probe at VPMN. The
test probe at VPMN shall send 180 Ringing
unreliably, i.e. not containing “Require: 100rel”.
205AH,
205AV
Voice MO session setup
time
Make a successful VoLTE MO call from a test probe
at VPMN to HPMN. Measure the time between
sending INVITE and receiving 200 OK (INVITE) (ref.
to 3GPP TS 34.229-1, C.21 [13]). The time for the
HPMN user accepting the incoming call is excluded
in the calculation.
206AH,
206AV
Voice MT session setup
time
Make a successful VoLTE MT call from HPMN to a
test probe at VPMN. Measure the time between
receiving INVITE and receiving ACK for 200 OK (INVITE) (ref. to 3GPP TS 34.229-1, C.11 ). The
time for the VPMN user accepting the incoming call
is excluded in the calculation.
207AH,
207AV
Voice MO session setup
ratio
Initial a VoLTE MO call from a test probe at VPMN to
HPMN. The session is successfully established if 200
OK (INVITE) is received at VPMN.
208AH,
208AV
Voice MT session setup
ratio
Initial a VoLTE MT call from HPMN to a test probe at
VPMN. The call is successfully established if ACK for
200 OK (INVITE) is received by the test probe at
VPMN.
211AH,
211AV
Voice MO session duration Make a successful VoLTE MO call from a test probe
at VPMN to HPMN. Measure the time at the test
probe of VPMN between receiving 200 OK (INVITE)
and receiving 200 OK (BYE) – using the MO call
release in the test (ref. to 3GPP TS 34.229-1, C.21,
C.32).
212AH,
212AV
Voice MT session duration Make a successful VoLTE MT call from HPMN to a
test probe at VPMN. Measure the time at the test
probe of HPMN between receiving ACK for 200 OK
(INVITE) and receiving BYE – using the MT call
release in the test (ref. to 3GPP TS 34.229-1, C.11,
C.33).
213AH,
213AV
OIP transparency MO Make a successful VoLTE MO call from a test probe
at VPMN to HPMN and check at the test probe of
HPMN if the OIP results in a dial able format (ref. to
GSMA IR.92, 2.3.12) to call back the call
originator at VPMN.
210AH,
210AV
OIP transparency MT Make a successful VoLTE MT call from HPMN to a
test probe at VPMN and check at the test probe of
VPMN if the OIP results in a dial able format to call
back the call originator at HPMN.
214AH,
214AV
SpQ on call basis at Rparty Make a successful VoLTE MO call from a test probe
at VPMN to a test probe at HPMN. Play a standard
file in the test probe at HPMN and record this file in
the test probe at VPMN and calculate the voice
quality per POLQA as specified in ITU-T P.863.
Recommended duration of the audio reference file: 8
– 32s. The sample is played/analysed. The end-result
of the test is a pre-aggregation of the measured
MOS-LQO values to one value per call.
215AH,
215AV
SpQ on call basis at Hparty Make a successful VoLTE MO call from a test probe
at VPMN to a test probe at HPMN. Play a standard
file in the probe at VPMN and record this file in the
probe at HPMN and calculate the voice quality per
POLQA as specified in ITU-T P.863.
Recommended duration of the audio reference file: 8
– 32s. The sample is played/analysed. The end-result
of the test is a pre-aggregation of the measured
MOS-LQO values to one value per call.
216AH,
216AV
SpQ R-factor at R-party Make a successful VoLTE MO call from a test probe
at VPMN to a test probe at HPMN. Play a standard
file in the test probe at HPMN and record this file in
the test probe at VPMN and calculate the voice quality. The sample is played/analysed.
Recommended duration of the audio reference file: 8
– 32s. The end-result of the test is the R-factor LQ
(Listening Quality) calculated per ITU-T G.107.
217AH,
217AV
SpQ R-factor at H-party Make a successful VoLTE MO call from a test probe
at VPMN to a test probe at HPMN. Play a standard
file in the probe at VPMN and record this file in the
probe at HPMN and calculate the voice quality. The
sample is played/analysed. Recommended duration
of the audio reference file: 8 – 32s. The end-result of
the test is the R-factor LQ (Listening Quality)
calculated per ITU-T G.107 .
230AH,
230AV
SRVCC MO success ratio Make a VoLTE MO call from a test probe at VPMN to
a test probe at HPMN and trigger an SRVCC PS to
CS event at VPMN. The test is successful if a TMSI
REALLOCATION COMMAND message is received
by the test probe in the target 2G/3G cell of VPMN,
and
– In case of SRVCC pre-alerting phase, CS call
establishment is continued, CC_ALERTING and
CONNECT are received (ref. to 3GPP TS
36.523-1, 13.4.3.7 ).
– In case of SRVCC alerting phase, CS call
establishment is continued, CONNECT is
received (ref. to 3GPP TS 36.523-1, 13.4.3.21). The voice channel is through connected in
the target 2G/3G cell of VPMN.
– In case of SRVCC mid-call phase, the voice
channel is through connected in the target
2G/3G cell of VPMN.
231AH,
231AV
SRVCC MT success ratio Make a VoLTE MT call from HPMN to a test probe at
VPMN and trigger an SRVCC PS to CS event at
VPMN. The test is successful if a TMSI
REALLOCATION COMMAND message is received
by the test probe in the target 2G/3G cell of VPMN,
and
– In case of SRVCC pre-alerting or alerting phase,
CS call establishment is continued, CONNECT
ACKNOWLEDGE is received (ref. to 3GPP TS
36.523-1, 13.4.3.10). The voice channel is
through connected in the target 2G/3G cell of
VPMN.
– In case of SRVCC mid-call phase, the voice
channel is through connected in the target
2G/3G cell of VPMN.
232AH,
232AV
SRVCC MO success time Make a VoLTE MO call from a test probe at VPMN to
a test probe at HPMN and trigger an SRVCC PS to
CS event at VPMN.
– In case of SRVCC pre-alerting or alerting phase,
measure the time between receiving
MobilityFromEUTRACommand in the E-UTRAN
cell and receiving CONNECT in the target
2G/3G cell (ref. to 3GPP TS 36.523-1, 13.4.3.7).
– In case of SRVCC mid-call phase, two test methods are applied.
a) Measure the time at the test probe of VPMN
between receiving
MobilityFromEUTRACommand in the EUTRAN
cell and receiving TMSI
REALLOCATION COMMAND in the target
2G/3G cell.
b) An average of downlink voice interruption
time at the two probes of VPMN and HPMN.
The time for the HPMN user accepting the incoming
call is excluded in the calculation.
233AH,
233AV
SRVCC MT success time Make a VoLTE MT call from a test probe HPMN to a
test probe at VPMN and trigger an SRVCC PS to CS
event at VPMN.
– In case of SRVCC pre-alerting or alerting phase,
measure the time between receiving
MobilityFromEUTRACommand in the E-UTRAN
cell and receiving CONNECT ACKNOWLEDGE
in the target 2G/3G cell (ref. to TS 36.523-1,
13.4.3.10).
– In case of SRVCC mid-call phase, two test
methods are applied.
a) Measure the time at the test probe of VPMN
between receiving
MobilityFromEUTRACommand in the EUTRAN
cell and receiving TMSI
REALLOCATION COMMAND in the target
2G/3G cell.
b) An average of downlink voice interruption
time at the two probes of VPMN and HPMN.
The time for the VPMN user accepting the incoming
call is excluded in the calculation.

SMS over IP

QoS Aspects GRQ
Id
KPI description
1. SMSoIP Accessibility 201
202
IMS Registration success ratio
IMS Registration success time
2. Service accessibility 221
222
223
224
SMSoIP-MO accessibility
SMSoIP-MT accessibility
SMSoIP-MO access delay
SMSoIP-MT access delay
3. Connection
establishment
225
226
SMSoIP-MO e2e delay
SMSoIP-MT e2e delay

ViLTE

QoS Aspects GRQ
Id
KPI description
ViLTE service
Accessibility
301
302
IMS Registration success ratio
IMS Registration time
ViLTE service integrity
and retainability
303
304
305
306
307
308
311
312
313
310
314
315
216
217
318
319
ViLTE MO accessibility (NER-MO or SAT-MO success ratio)
ViLTE MT accessibility (NER -MT or SAT-MT success ratio)
ViLTE MO session setup time (PDD-MO or STT-MO duration)
ViLTE MT session setup time (PDD-MT or STT-MT duration)
ViLTE MO session setup ratio (CSSR-MO)
ViLTE MT session setup ratio (CSSR-MT)
ViLTE MO session duration
ViLTE MT session duration
OIP transparency MO (CLI transparency)
OIP transparency MT (CLI transparency)
Speech quality on sample basis at R-party
Speech quality on sample basis at H-party
Speech quality R-factor at R-party
Speech quality R-factor at H-party
Video quality on sample basis at R-party
Video quality on sample basis at H-party
ViLTE service mobility 230
231
232
233
SRVCC MO success ratio
SRVCC MT success ratio
SRVCC MO time
SRVCC MT time
LTE network quality
for ViLTE service
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
RTP max packet delay variation R2H (A-B)
RTP max packet delay variation H2R (B-A)
RTP mean packet delay variation R2H (A-B)
RTP mean packet delay variation H2R (B-A)
RTP mean interarrival jitter of incoming streaming R2H (A-B)
RTP mean interarrival jitter of incoming streaming H2R (B-A)
RTP mean data rate transmitted R2H (A-B)
RTP mean data rate received H2R (B-A)
RTP mean data rate transmitted H2R (B-A)
RTP mean data rate received R2H (A-B)
RTP packets lost R2H (A-B)
RTP packets lost H2R (B-A)
RTP packet lost ratio R2H (A-B)
RTP packet lost ratio H2R (B-A)
RTP round-trip delay (RTD RHR A-B-A)
RTP round-trip delay (RTD HRH B-A-B)
RTP one-way delay (OWD R2H A-B)
RTP one-way delay (OWD H2R B-A)
RTP transport quality
for ViLTE service –
video
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
RTP max packet delay variation R2H (A-B)
RTP max packet delay variation H2R (B-A)
RTP mean packet delay variation R2H (A-B)
RTP mean packet delay variation H2R (B-A)
RTP mean interarrival jitter of incoming streaming R2H (A-B)
RTP mean interarrival jitter of incoming streaming H2R (B-A)
RTP mean data rate transmitted R2H (A-B)
RTP mean data rate received H2R (B-A)
RTP mean data rate transmitted H2R (B-A)
RTP mean data rate received R2H (A-B)
RTP packets lost R2H (A-B)
RTP packets lost H2R (B-A)
RTP packet lost ratio R2H (A-B)
RTP packet lost ratio H2R (B-A)
RTP round-trip delay (RTD RHR A-B-A)
RTP round-trip delay (RTD HRH B-A-B)
RTP one-way-delay (OWD R2H A-B)
RTP one-way-delay (OWD H2R B-A)

 

ITU Guidelines

The standardized KPIs are useful if we need to compare two solutions from different vendors. But in daily practice we don’t limit ourselves only to standard set for KPIs and counters. Each VoLTE/VoWifi/RCS/…  solution is very specific, each operator does use a different set of features, has a different network architecture, different infrastructure etc. Some best practices or recommendations are provided by ITU’s Telecommunication Standardization Sector (ITU-T) – e.g. the ITU-T G.1028 gives us End-to-end quality of service for voice over 4G mobile networks recommendations, including high-level troubleshooting guidelines.

VoLTE QoS measurement points – monitoring as per ITU-T

Anyway to go a more in detail we need to understand a particular network, we have to know what are the supported and expected callflows, applied functionalities and other operator specific stuff. That’s why the monitoring is often designed or customized directly by mobile operators/service providers. They have to decide the importance of individual counters, KPIs (as well as billing/log records used as inputs for statistics). What is also extremely important, the data is stored in database and operators can base their analysis/decisions on comparison of actual values with the numbers from the past (e.g. to compare Registration Success Rate with the data from yesterday, previous week, previous month, ..). The values can be also used as feeds for Artificial Intelligence Systems providing decision making support.

Related articles:

Print Friendly, PDF & Email

Realtimecommunication.info

4G/5G, VoLTE, RCS, IMS, SIP, WebRTC, IoT/M2M Descrition, CallFlows, Illustrated Guides, Tests and other stuff for telco egineers

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.