Liaison statement
LS/r to ITU-T SG11 on “QoS-related work in ITU-T SG11” (SG11-LS 105) [from ITU-T SG12]

State Posted
Posted Date 2016-02-02
From Group ITU-T-SG-12
From Contact Judit Kiss
To Group ippm
To Contacts Brian Trammell
Bill Cerveny
Scott Mansfield
Bill Cerveny
Brian Trammell
Spencer Dawkins
IP Performance Metrics Discussion List
Martin Stiemerling
Response Contact
Purpose For information
Attachments oLS 095-td863R1
oLS 095att-td862R1
Study Group 12 thanks you for your liaison reply concerning the development of Recommendations closely related to our mandate, as lead Study Group on Quality of Service and Quality of Experience.

We recognize the revisions to the Scope of Q.3960 (the former Q.FW_Int_sp_test) to change the emphasis to end users of an IP packet transfer service and supporting infrastructure, and to “estimate the access speed to the Internet and to the Internet resources” using measurements that “can be established at the national or international level” (from the current scope). However, the terms “Internet Access”, Internet Connection”, and “Internet Service” are not currently defined in the ITU terms and definitions, so it is necessary to de-scope aspects describing international applicability. National definitions of these terms may exist, but with considerable variability, again putting the applicability of this draft text into question.

In our prior liaison reply, we asked that you take advantage of in-force Recommendation Y.1540, which covers many critical aspects of the performance of IP-based Networks (beyond highlighting the requirements of  a single section which we considered particularly relevant to the draft text you shared with us). This Recommendation provides a wide range of fundamental and secondary performance parameters – all defined at the IP layer. The IP-layer is the layer with end-to-end significance to Internet service providers. Higher layers are implemented in hosts beyond the control of service providers, and the headers of higher layers are part of the payload octets conveyed in an IP packet transfer service. Thus, the availability of IP packet transfer and the performance of IP packet transfer attempts determine the quality of the service, according to the metrics defined in Y.1540. Other metrics are under study and should be removed from the scope from any ITU-T Recommendation (Q.FW_Int_sp_test) until on-going study is complete.

Your Liaison also mentions a new Recommendation planned for development in collaboration with ETSI INT, and which overlaps with the existing Recommendations of Q13/12. The true nature of this Recommendation was not clear from the description in the liaison, and was only clarified by your counsellor in response to questions.  Again, this planned effort is apparently un-aware of in-force Recommendations of the lead SG on the topics where you intend to design tests. The literature of SG 12 Question 13 must be studied and appreciated before you proceed:
        G.1010    End-user multimedia QoS categories
        G.1031    QoE factors in web-browsing
        G.1080    Quality of experience requirements for IPTV services
        G.1091    Quality of Experience requirements for telepresence services

In the brief interval allowed for us to prepare a liaison response, we list the following additional issues with your proposed text under Consent in Q15/11:

•       SG 11 believes that Y.1540 and IPPM RFCs are primarily relevant to the second phase of their Internet Speed development on test methodology. The SG 12 LR did not ask for references to be added to the list, it asked for the development of text to take advantage of the in-force specifications. Although Recommendation Y.1541 is an existing reference, there is no citation in the text. The same is true for BBF Technical Report 304 (2015), which references many IETF RFCs, and the TSB Editors could rightly remove these references from the list (since they are not cited in any way).
•       All Figures in the Consented Internet Speed Test Framework (Q. 3960) are now designated as examples. Examples or not, the Figures lack the detail to describe service scope, and so all discussion of Service Level Agreements (SLAs) is inappropriate.
•       In a brief review of the text of Q. 3960, there was strong opposition expressed to use of a “single Internationally recognized entity” to provide Internet Speed test facilities. AT&T and DT representatives expressed that the idea of a single entity having control of this testing was completely unacceptable (and probably not feasible, due to the multiplicity of peering arrangements required). Every Recommendation must allow for multiple independent implementations.
•       The measurement test definitions seek to measure the “absolute value” of transmission speed between different measurement points. However, there is no known approach which provides such a value. In fact, the surveys of Internet performance on which this framework is partly based have been informative as relative measures comparing different technologies of several service providers, but do not pretend to quantify anything in absolute terms. It must be made clear that any tests used to judge the performance of IP network service providers necessarily exclude factors beyond the service scope (e.g., the User-Network Interface, UNI, and Network-Network Interface, NNI).

Other comments are embedded in the text of Q. 3960, attached.

We reiterate that descriptions of popular measurement studies do not constitute a basis for an international standard, as they do not possess the necessary specificity to guarantee equivalent results from multiple independent implementations, or necessarily use performance parameters that exhibit important attributes such as repeatability.

We urge you to continue study of the relevant Recommendations, IETF RFCs, and other work-in-progress, and to suspend plans to seek approval for all related activities in SG 11. We invite you to join SG12 at our meeting in June 2016, where we will be able to share additional background on IP-based network performance parameters and methods of measurement.

We are unable to send a representative to the Joint meeting you have arranged with ETSI INT, and we apologize for our absence. If we are to coordinate and collaborate on this topic, then the first step is to coordinate among the calendars of key organizations and their interested participants.

Attachment: TD862 Rev.1 (Comments on the Last Call Text of Draft new Recommendation ITU-T Q. 3960)