EVALUATING THE USABILITY OF MOBILE APPLICATIONS WITHOUT AFFECTING THE USER AND THE USAGE CONTEXT

Abstract

Reliable usability evaluation results are crucial for the success of mobile applications. A high level of usability implies reduced development costs, reduced maintenance costs and higher customer retention. State-of-the-art usability evaluations produce distorted results for mobile applications, as they are mainly tailored for desktop applications and mostly take place in artificial environments: The test user is isolated from the normal usage context and subject to observation processes which significantly affect the user. This paper presents work in progress and proposes an architecture for a usability evaluation framework which preserves the natural mobile usage context, reduces the interference introduced by observations and reduces the overall costs of the evaluation procedure

Keywords

• Mobile Applications, Usability Testing, Non-Intrusive Evaluation, Software Development, Innovation Management, Field Tests

References

1. Bulling, A. and H. Gellersen (2010), “Toward Mobile Eye-Based Human
2. Computer Interaction”, IEEE Pervasive Computing, Vol.9, pp. 8-12. Bias, R. G. and D. J. Mayhew (2005), Cost-justifying usability: An update for the Internet age, Morgan Kaufmann, USA:San Fransico.
3. Clements, P., R. Kazman and M. Klein (2011), Evaluating Software Architectures
4. Methods and Case Studies, Addison-Wesley:Amsterdam. Coursaris, C. K. and D. J. Kim (2011), “A Meta-Analytical Review of Empiriacal
5. Mobile Usability Studies”, Journal of Usability Studies, Vol. 6, No. 3, pp. 117- Goldmedia Custom Research (2011), Wichtigste Ansprüche der Nutzer an ihre
6. Smartphones: Gute Bedienbarkeit und viele Apps, http://www.goldmedia.com/presse/newsroom/mobile-monitor-2011.html, Accessed 16.04.2013]
7. Gregor, S. and A. R. Hevner (2011), “Introduction to the special issue on design science”, Inf. Syst. E-bus. Manag., Vol. 9, No.1, pp. 1–9.
8. Kjeldskov, J. and J. Stage (2004), “New techniques for usability evaluation of mobile systems”, International Journal of Human-Computer Studies, Vol. 60, No. –6, pp. 599–620.

9. Knöpfel, A., B. Groene and P. Tabeling (2006), Fundamental Modeling Concepts,
10. West Sussex, UK: Wiley. Kruchten, P. (2004), The rational unified process: an introduction, Addison- Wesley:Amsterdam.
11. Liang, H., H. Song, Y. Fu, X. Ca and Z. Zhang (2011), "A remote usability testing platform for mobile phones," (in: IEEE International Conference on Computer
12. Science and Automation Engineering (CSAE) 2011, Vol.2), IEEE, pp. 312-316.
13. Lissoboi, I. and H. Kasai (2012), “Development of an Efficient Method for Eye
14. Detection on Mobile CE Devices”, (in: International Symposium on Computer, Consumer and Control (IS3C), 2012) , IEEE, pp. 337-340.
15. Ma, X., B. Yan, G. Chen, C. Zhang, K. Huang, J. Drury and L. Wang (2013),
16. “Design and Implementation of a Toolkit for Usability Testing of Mobile Apps”, Mobile Networks and Applications, Vol. 18, No. 1, pp. 81-97. Malý, I., Z. Míkovec, J. Vystrčil, J. Franc and P. Slavík (2011) “An evaluation tool for research of user behavior in a realistic mobile environment”, Personal and Ubiquitous Computing, pp. 1–12.
17. Miluzzo, E., T. Wang and A. T. Campbell (2010), “Eyephone: activating mobile phones with your eyes”, (in: Proceedings of the second ACM SIGCOMM workshop on Networking, systems, and applications on mobile handhelds), ACM, pp. 15-20.
18. Ryu, Y. S. (2005), ”Development of usability questionnaires for electronic mobile products and decision making methods”, Doctoral dissertation, Virginia
19. Polytechnic Institute and State University. Sá, M. de, L. Carriço, L. Duarte and T. Reis Prototyping Tool for Mobile Devices”. (in: Proceedings of the working conference on Advanced visual interfaces. AVI ‘08, Italy), ACM:New York, pp. 232. (2008), “A Mixed-Fidelity
20. Tamminen, S., A. Oulasvirta, K. Toiskallio and A. Kankainen (2004), “Under- standing mobile contexts”, Personal and Ubiquitous Computing, Vol. 8, No. 2, pp. 135-143.
21. Tullis, T., S. Fleischman, M. McNulty, C. Cianchett and M. Bergel (2002), “An empirical comparison of lab and remote usability testing of web sites”, Usability
22. Professionals Association Conference. Vargas, A., H. Weffers and H. V. da Rocha, (2010) “A Method for Remote and Semi-Automatic Usability Evaluation of Web-based Applications Through Users
23. Behavior Analysis”, (in: Proceedings of the 7th International Conference on Methods and Techniques in Behavioral Research, Netherland), ACM:New York, pp. 19:1-19:5. Väätäjä, H. and Roto V. (2010), “Mobile questionnaires for user experience evaluation”, (in: CHI '10 Extended Abstracts on Human Factors in Computing Systems), ACM, pp. 3361-3366.