SS7: Test methodologies for better ecological validity

Motivation and objectives 

Test methodologies are a crucial aspect of the evaluation of multimedia systems and services. Conventional test methods often rely on simplified or artificial scenarios that may not accurately reflect real-world use cases. This can lead to results that lack ecological validity, i.e. they may not generalize to the way in which users actually experience the systems being tested. 

The special session at QoMEX will focus on addressing this issue by exploring research that aims to improve the realism of test methodologies, with the goal of obtaining results that are more applicable to real-world settings. We invite papers that address any aspect of test design and methodology that can contribute to this goal. 

Topics of interest

Possible topics of interest include but are not limited to: 

• Joint evaluation of technical (e.g. video quality) and non-technical (e.g. sense of presence) features. 

• Designing test sessions to reflect real-world consumption habits. 

• Methodologies for assessing the quality of systems designed for professional or industrial use cases. 

• Addressing shortcomings of existing methodologies, such as generalizing subjective scores or avoiding bias in test results. 

Organizers

• Pablo Pérez, Nokia, Spain

• Marta Orduna, UPM, Spain

• Carlos Cortés, UPM, Spain

• Femi Adeyemi-Ejeye, University of Surrey, UK

• Pablo César, CWI, The Netherlands