Quality of experience

Quality of Experience (QoE) is a measure of the delight or annoyance of a customer's experiences with a service (e.g., web browsing, phone call, TV broadcast).[1] QoE focuses on the entire service experience; it is a holistic concept, similar to the field of user experience, but with its roots in telecommunication.[2] QoE is an emerging multidisciplinary field based on social psychology, cognitive science, economics, and engineering science, focused on understanding overall human quality requirements.

Definition and concepts

In 2013, within the context of the COST Action QUALINET, QoE has been defined as:[1]

The degree of delight or annoyance of the user of an application or service. It results from the fulfillment of his or her expectations with respect to the utility and / or enjoyment of the application or service in the light of the user’s personality and current state.

This definition has been adopted in 2016 by the International Telecommunication Union in Recommendation ITU-T P.10.[3] Before, various definitions of QoE had existed in the domain, with the above-mentioned definition now finding wide acceptance in the community.

QoE has historically emerged from Quality of Service (QoS), which attempts to objectively measure service parameters (such as packet loss rates or average throughput). QoS measurement is most of the time not related to a customer, but to the media or network itself. QoE however is a purely subjective measure from the user’s perspective of the overall quality of the service provided, by capturing people’s aesthetic and hedonic needs.[4]

QoE looks at a vendor's or purveyor's offering from the standpoint of the customer or end user, and asks, "What mix of goods, services, and support, do you think will provide you with the perception that the total product is providing you with the experience you desired and/or expected?" It then asks, "Is this what the vendor/purveyor has actually provided?" If not, "What changes need to be made to enhance your total experience?" In short, QoE provides an assessment of human expectations, feelings, perceptions, cognition and satisfaction with respect to a particular product, service or application.[5]

QoE is a blueprint of all human subjective and objective quality needs and experiences arising from the interaction of a person with technology and with business entities in a particular context.[4] Although QoE is perceived as subjective, it is an important measure that counts for customers of a service. Being able to measure it in a controlled manner helps operators understand what may be wrong with their services and how to improve them.

QoE factors

QoE aims at taking into consideration every factor that contributes to a user's perceived quality of a system or service. This includes system, human and contextual factors.[6] The following so-called "influence factors" have been identified and classified by Reiter et al.:[6]

• Human Influence Factors
  • Low-level processing (visual and auditory acuity, gender, age, mood, …)
  • Higher-level processing (cognitive processes, socio-cultural and economic background, expectations, needs and goals, other personality traits…)
• System Influence Factors
  • Content-related
  • Media-related (encoding, resolution, sample rate, …)
  • Network-related (bandwidth, delay, jitter, …)
  • Device-related (screen resolution, display size, …)
• Context Influence Factors
  • Physical context (location and space)
  • Temporal context (time of day, frequency of use, …)
  • Social context (inter-personal relations during experience)
  • Economic context
  • Task context (multitasking, interruptions, task type)
  • Technical and information context (relationship between systems)

Studies in the field of QoE have typically focused on system factors, primarily due to its origin in the QoS and network engineering domains. Through the use of dedicated test laboratories, the context is often sought to be kept constant.

QoE versus User Experience

QoE is strongly related to but different from the field of User Experience (UX), which also focuses on users' experiences with services. Historically, QoE has emerged from telecommunication research, while UX has its roots in Human–Computer Interaction.[2] Both fields can be considered multi-disciplinary. In contrast to UX, the goal of improving QoE for users was more strongly motivated by economic needs.[7]

Wechsung and De Moor identify the following key differences between the fields:[2]

	QoE	UX
Origins	Telecommunication	Human–Computer Interaction
Driving Force	Technology-centered	Human-centered
Theoretical Basis	Measurement and instrumentation approaches Historical lack of theoretical frameworks	Non-instrumental research Theoretic background in hedonic psychology
Measurement and Evaluation	Predominantly quantitative research Empirical–positivist research	Predominantly qualitative methods Interpretative and constructivist research
Experience and Perceptions	Focus on “quality formation” and perception of quality	Focus on “experience” concept

QoE measurement

As a measure of the end-to-end performance at the service level from the user's perspective, QoE is an important metric for the design of systems and engineering processes. This is particularly relevant for video services because – due to their high traffic demands –, bad network performance may highly affect the user's experience.[8][9] So, when designing systems, the expected output, i.e. the expected QoE, is often taken into account – also as a system output metric and optimization goal.

To measure this level of QoE, human ratings can be used. The mean opinion score (MOS) is a widely used measure for assessing the quality of media signals. It is a limited form of QoE measurement, relating to a specific media type, in a controlled environment and without explicitly taking into account user expectations. The MOS as an indicator of experienced quality has been used for audio and speech communication, as well as for the assessment of quality of Internet video, television and other multimedia signals,[10] and web browsing.[11] Due to inherent limitations in measuring QoE in a single scalar value, the usefulness of the MOS is often debated.[12]

Subjective quality evaluation requires a lot of human resources, establishing it as a time-consuming process. Objective evaluation methods can provide quality results faster, but require dedicated computing resources. Since such instrumental video quality algorithms are often developed based on a limited set of subjective data, their QoE prediction accuracy may be low when compared to human ratings.

QoE metrics are often measured at the end devices and can conceptually be seen as the remaining quality after the distortion introduced during the preparation of the content and the delivery through the network, until it reaches the decoder at the end device. There are several elements in the media preparation and delivery chain, and some of them may introduce distortion. This causes degradation of the content, and several elements in this chain can be considered as ”QoE-relevant“ for the offered services. The causes of degradation are applicable for any multimedia service, that is, not exclusive to video or speech. Typical degradations occur at the encoding system (compression degradation), transport network, access network (e.g., packet loss or packet delay), home network (e.g. WiFi performance) and end device (e.g. decoding performance).

QoE management

Several QoE-centric network management and bandwidth management solutions have been proposed, which aim to improve the QoE delivered to the end-users.[13][14][15][16]

When managing a network, QoE fairness may be taken into account in order to keep the users sufficiently satisfied (i.e., high QoE) in a fair manner. From a QoE perspective, network resources and multimedia services should be managed in order to guarantee specific QoE levels instead of classical QoS parameters, which are unable to reflect the actual delivered QoE. A pure QoE-centric management is challenged by the nature of the Internet itself, as the Internet protocols and architecture were not originally designed to support today's complex and high demanding multimedia services.

As an example for an implementation of QoE management, network nodes can become QoE-aware by estimating the status of the multimedia service as perceived by the end-users.[17] This information can then be used to improve the delivery of the multimedia service over the network and proactively improve the users' QoE.[18] This can be achieved, for example, via traffic shaping. QoE management gives the service provider and network operator the capability to minimize storage and network resources by allocating only the resources that are sufficient to maintain a specific level of user satisfaction.

As it may involve limiting resources for some users or services in order to increase the overall network performance and QoE, the practice of QoE management requires that net neutrality regulations are considered.[19]

References

1. Qualinet White Paper on Definitions of Quality of Experience (2012). European Network on Quality of Experience in Multimedia Systems and Services (COST Action IC 1003), Patrick Le Callet, Sebastian Möller and Andrew Perkis, eds., Lausanne, Switzerland, Version 1.2, March 2013
2. Wechsung, Ina; Moor, Katrien De (2014). Möller, Sebastian; Raake, Alexander (eds.). Quality of Experience. T-Labs Series in Telecommunication Services. Springer International Publishing. pp. 35–54. doi:10.1007/978-3-319-02681-7_3. ISBN 9783319026800.
3. ITU-T Recommendation P.10: Vocabulary for performance and quality of service, Amendment 5 (07/16). https://www.itu.int/rec/T-REC-P.10
4. Laghari, K. U. Rehman; Connelly, K. (April 2012). "Toward total quality of experience: A QoE model in a communication ecosystem". IEEE Communications Magazine. 50 (4): 58–65. doi:10.1109/MCOM.2012.6178834.
5. Laghari, Khalil ur Rehman; Crespi, Noel; Molina, B.; Palau, C.E. (2011-03-25). "QoE Aware Service Delivery in Distributed Environment". 2011 IEEE Workshops of International Conference on Advanced Information Networking and Applications (PDF). Ieeexplore.ieee.org. pp. 837–842. doi:10.1109/WAINA.2011.58. ISBN 978-1-61284-829-7. Retrieved 2014-03-03.
6. Reiter, Ulrich; Brunnström, Kjell; Moor, Katrien De; Larabi, Mohamed-Chaker; Pereira, Manuela; Pinheiro, Antonio; You, Junyong; Zgank, Andrej (2014-01-01). Möller, Sebastian; Raake, Alexander (eds.). Factors Influencing Quality of Experience. T-Labs Series in Telecommunication Services. Springer International Publishing. pp. 55–72. doi:10.1007/978-3-319-02681-7_4. ISBN 978-3-319-02680-0.
7. Reichl, Peter; Tuffin, Bruno; Maillé, Patrick (2012). Hadjiantonis, Antonis M.; Stiller, Burkhard (eds.). Telecommunication Economics. Lecture Notes in Computer Science. Springer Berlin Heidelberg. pp. 158–166. doi:10.1007/978-3-642-30382-1_21. ISBN 9783642303814.
8. Dobrian, Florin; Awan, Asad; Joseph, Dilip; Ganjam, Aditya; Zhan, Jibin; Sekar, Vyas; Stoica, Ion; Zhang, Hui (2013-03-01). "Understanding the Impact of Video Quality on User Engagement". Commun. ACM. 56 (3): 91–99. doi:10.1145/2428556.2428577. ISSN 0001-0782.
9. Krishnan, S. S.; Sitaraman, R. K. (2013-12-01). "Video Stream Quality Impacts Viewer Behavior: Inferring Causality Using Quasi-Experimental Designs". IEEE/ACM Transactions on Networking. 21 (6): 2001–2014. CiteSeerX 10.1.1.649.7730. doi:10.1109/TNET.2013.2281542. ISSN 1063-6692.
10. Winkler, S. (2009-12-01). Video quality measurement standards #x2014; Current status and trends. 2009 7th International Conference on Information, Communications and Signal Processing (ICICS). pp. 1–5. doi:10.1109/ICICS.2009.5397585. ISBN 978-1-4244-4656-8.
11. Egger, S.; Hossfeld, T.; Schatz, R.; Fiedler, M. (2012-07-01). Waiting times in quality of experience for web based services. 2012 Fourth International Workshop on Quality of Multimedia Experience. pp. 86–96. doi:10.1109/QoMEX.2012.6263888. ISBN 978-1-4673-0726-0.
12. Hoßfeld, Tobias; Heegaard, Poul E.; Varela, Martín; Möller, Sebastian (2016-12-01). "QoE beyond the MOS: an in-depth look at QoE via better metrics and their relation to MOS". Quality and User Experience. 1 (1): 2. arXiv:1607.00321. doi:10.1007/s41233-016-0002-1. ISSN 2366-0139.
13. Baraković, Sabina; Skorin-Kapov, Lea (2013). "Survey and Challenges of QoE Management Issues in Wireless Networks". Journal of Computer Networks and Communications. 2013: 1–28. doi:10.1155/2013/165146.
14. Hoßfeld, T.; Schatz, R.; Varela, M.; Timmerer, C. (2012-04-01). "Challenges of QoE management for cloud applications". IEEE Communications Magazine. 50 (4): 28–36. doi:10.1109/MCOM.2012.6178831. ISSN 0163-6804.
15. Seufert, M.; Egger, S.; Slanina, M.; Zinner, T.; Hobfeld, T.; Tran-Gia, P. (2015-01-01). "A Survey on Quality of Experience of HTTP Adaptive Streaming". IEEE Communications Surveys and Tutorials. 17 (1): 469–492. doi:10.1109/COMST.2014.2360940. ISSN 1553-877X.
16. Baraković, Sabina; Skorin-Kapov, Lea (2013-03-23). "Survey and Challenges of QoE Management Issues in Wireless Networks". Journal of Computer Networks and Communications. 2013: 1–28. doi:10.1155/2013/165146. ISSN 2090-7141.
17. Huysegems, R.; De Vleeschauwer, B.; De Schepper, K.; Hawinkel, C.; Wu, Tingyao; Laevens, K.; Van Leekwijck, W. (2012-06-01). Session reconstruction for HTTP adaptive streaming: Laying the foundation for network-based QoE monitoring. 2012 IEEE 20th International Workshop on Quality of Service (IWQoS). pp. 1–9. doi:10.1109/IWQoS.2012.6245987. ISBN 978-1-4673-1298-1.
18. Petrangeli, S.; Wauters, T.; Huysegems, R.; Bostoen, T.; De Turck, F. (2015-05-01). Network-based dynamic prioritization of HTTP adaptive streams to avoid video freezes. 2015 IFIP/IEEE International Symposium on Integrated Network Management (IM). pp. 1242–1248. doi:10.1109/INM.2015.7140475. hdl:1854/LU-7016579. ISBN 978-1-4799-8241-7.
19. Krämer, Jan; Wiewiorra, Lukas; Weinhardt, Christof (2013). "Net neutrality: A progress report". Telecommunications Policy. 37 (9): 794–813. CiteSeerX 10.1.1.258.5878. doi:10.1016/j.telpol.2012.08.005.