Limitations of Emotion Recognition in Software User Experience Evaluation Context
Agnieszka Landowska, Jakub Miler
Citation: Proceedings of the 2016 Federated Conference on Computer Science and Information Systems, M. Ganzha, L. Maciaszek, M. Paprzycki (eds). ACSIS, Vol. 8, pages 1631–1640 (2016)
Abstract. This paper concerns how affective-behavioral-cognitive approach applies in software user experience evaluation context. Although it may seem, that affect recognition solutions are accurate in determining user experience, there are several challenges in practical settings. This paper aims at exploration of the limitations of automatic affect recognition applied in usability context as well as proposing set of criteria to choose input channels for affect recognition. The results are revealed through a semi-experiment based on case study of educational game. As a result, a number of concerns were identified, providing a list of pros and cons for affective computing methods applied in usability testing context. The lessons learned might be interesting for both researchers that develop emotion recognition algorithms and for practitioners, who apply them in diverse contexts
- ISO. 1998, Norm 9241: Ergonomics of human-system interaction.
- N. Bevan, 2009. What is the difference between the purpose of usability and user experience evaluation methods. In: Proceedings of the Workshop UXEM’09 (INTERACT’09), Uppsala, Sweden.
- H. I. Ahn, and R. Picard, 2014. Measuring Affective-Cognitive Experience and Predicting Market Success. IEEE Transactions on Affective Comp. 5(2):173-186, http://dx.doi.org/10.1109/TAFFC.2014.2330614
- P. Lew, L. Olsina, P. Becker and L. Zhang, 2012, An integrated strategy to systematically understand and manage quality in use for web applications. Requirements Engineering, 17(4): 299-330 http://dx.doi.org/10.1007/s00766-011-0128-x.
- W. Albert and T. Tullis. 2013. Measuring the user experience: collecting, analyzing, and presenting usability metrics. Morgan Kaufmann, USA.
- M. Szwoch and P. Pieniążek. 2015. Facial Emotion Recognition Using Depth Data, The 8th Int. Conf. on Human System Interaction, pp. 271-277, IEEE, http://dx.doi.org/10.1109/HSI.2015.7170679
- A. Kołakowska. 2015, Recognizing emotions on the basis of keystroke dynamics, Proc. of the 8th International Conference on Human System Interaction, Poland, http://dx.doi.org/10.1109/HSI.2015.7170682
- Z. Zeng, M. Pantic, G. Roisman, and T.S. Huang, 2009. A survey of affect recognition methods: Audio, visual, and spontaneous expressions. IEEE Transactions on Pattern Analysis and Machine Intelligence, , 31(1): 39-58, http://dx.doi.org/10.1109/TPAMI.2008.52.
- H.Gunes and B. Schuller, 2013. Categorical and dimensional affect analysis in continuous input: Current trends and future directions, Image and Vision Computing, 31:120-136, http://dx.doi.org/10.1016/j.imavis.2012.06.016
- A. Kołakowska, A. Landowska, M. Szwoch, W. Szwoch and M. R. Wrobel. 2015. Modeling emotions for affect-aware applications. In Information Systems Development and Applications, University of Gdańsk, Poland, pp. 55–69
- H. Gunes and M. Piccardi. 2005. Affect Recognition from Face and Body: Early Fusion versus Late Fusion, in Proc. IEEE International Conference on Systems, Man and Cybernetics, pp. 3437-3443. http://dx.doi.org/10.1109/ICSMC.2005.1571679
- I. Hupont, S. Ballano, S. Baldassarri and E. Cerezo. 2011. Scalable multimodal fusion for continuous affect sensing, IEEE Workshop on Affective Computational Intelligence, pp. 1,8, 11-15, http://dx.doi.org/10.1109/WACI.2011.5953150
- J. N. Bailenson, E. D. Pontikakis, I. B. Mauss, J. J. Gross, M. E. Jabon, C. A. C. Hutcherson, C. Nass and O. John. 2008. Real-time classification of evoked emotions using facial feature tracking and physiological responses, International Journal of Human-Computer Studies, 66(5): 303-317, http://dx.doi.org/doi:10.1016/j.ijhcs.2007.10.011.
- A. Kołakowska, A. Landowska, M. Szwoch, W. Szwoch, M. R. Wróbel. 2013. Emotion recognition and its application in software engineering, Proc. of 6th International Conference on Human-System Interaction, Poland, pp. 532 - 539, http://dx.doi.org/10.1109/HSI.2013.6577877
- A. Kołakowska, A. Landowska, M. Szwoch, W. Szwoch, M. R. Wróbel. 2014. Emotion recognition and its applications, HumanComputer Systems Interaction: Backgrounds and Applications 3. pp. 51-62, Springer. http://dx.doi.org/10.1007/978-3-319-08491-6_5
- T. Partala, A. Kallinen. 2012. Understanding the Most Satisfying and Unsatisfying User Experiences: Emotions, Psychological Needs, and Context. Interacting with Computers, 24(1):25–34. http://dx.doi.org/10.1016/j.intcom.2011.10.001.
- R. Hazlett, J. Benedek 2007. Measuring emotional valence to understand the user’s experience of software, Int. J. Human-Computer Studies, 65:306-314. http://dx.doi.org/10.1016/j.ijhcs.2006.11.005
- A. Landowska, M.R. Wróbel 2015. Affective reactions to playing digital games, 8th International Conference on Human System Interaction, IEEE, pp.264-270. http://dx.doi.org/10.1109/HSI.2015.7170678
- L. Chittaro, R. Sioni 2014. Affective Computing vs. Affective Placebo: Study of a Biofeedback-Controlled Game for Relaxation Training. International Journal of Human-Computer Studies, 72, 8–9, pp. 663–73. http://dx.doi.org/10.1016/j.ijhcs.2014.01.007.
- W. Szwoch. 2015. Model of emotions for game players, 8th International Conference on Human System Interactions, IEEE, pp.285-290. 10.1109/HSI.2015.7170681
- P. Zimmermann, P. Gomez, B. Danuser, S. Schar. 2006. Extending usability: putting affect into the user-experience, in Proc. of Nordic Conf. on Human-Computer Interaction, Oslo, pp 27-32.
- A. Landowska. 2015. Towards Emotion Acquisition in IT Usability Evaluation Context, Proceedings of the Multimedia, Interaction, Design and Innovation, 5, http://dx.doi.org/10.1145/2814464.2814470
- GraPM website, http://grapm.html-5.me.
- J. Miler, A. Landowska. 2016. Designing effective educational games - a case study of a project management game, FedCSIS, Gdansk, Poland (accepted)
- A. Landowska, 2014. Emotion monitoring - verification of physiological characteristics measurement procedures, Metrology and Measurement Systems Journal, Vol XXI, 4:719-732. http://dx.doi.org/10.2478/mms-2014-0049
- A. Landowska, 2015. Emotion monitor-concept, construction and lessons learned, in Proc. of Computer Science and Information Systems (FedCSIS), Łódź, Poland, pp.75-80. http://dx.doi.org/10.15439/2015F264