Location of Repository

Meeting the Challenges in Evaluating Mobile Learning: A 3-level Evaluation Framework

By Giasemi N. Vavoula and Mike Sharples


This is the final publisher edited version of the paper published as International Journal of Mobile and Blended Learning, 2009, 1 (2), pp. 54-75. This version was first published at http://www.igi-global.com/Bookstore/Article.aspx?TitleId=4058, Doi: 10.4018/jmbl.2009040104.We propose six challenges in evaluating mobile learning: capturing and analyzing learning in context and across contexts, measuring mobile learning processes and outcomes, respecting learner/participant privacy, assessing mobile technology utility and usability, considering the wider organizational and socio-cultural context of learning, and assessing in/formality. A three-level framework for evaluating mobile learning is proposed, comprising a micro level concerned with usability, a meso level concerned with the learning experience, and a macro level concerned with integration within existing educational and organizational contexts. The article concludes with a discussion of how the framework meets the evaluation challenges and with suggestions for further extensions

Topics: Mobile learning evaluation, learning context, evaluation framework, learning outcomes, ethics, informality and formality of learning, requirements for evaluation
Publisher: IGI Global
Year: 2009
DOI identifier: 10.4018/jmbl.2009040104
OAI identifier: oai:lra.le.ac.uk:2381/8123

Suggested articles



  1. (2009). 73 Copyright © 2009, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
  2. (2009). 75 Copyright © 2009, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
  3. (2001). A Briefing on Key Concepts: Formative and summative, criterion and norm-referenced assessment. LTSN Generic Centre.
  4. (2007). A Framework for Conceptualising the Impact of
  5. (2006). A lifecycle approach to evaluating MyArtSpace. In doi
  6. (2004). A task-centred approach to evaluating a mobile learning environment for pedagogical soundness.
  7. (2007). A Theory of Learning for the Mobile Age. In doi
  8. (2006). Adopting a Lifecycle Approach to the Evaluation of Computers and Information Technology.
  9. An Evaluation of MyArtSpace: a Mobile Learning Service for School Museum Trips.
  10. (2007). Analysis of an informal mobile learning activity based on activity theory.
  11. (1995). Assessment and learning: contradictory or complementary? In
  12. (2001). Building the Virtual State: Information Technology and Institutional Change, doi
  13. (2007). Conceptualizing Learning from the Everyday Activities of Digital Kids. doi
  14. (2008). Designing Learning Activities with Mobile Technologies. In doi
  15. (2005). Emerging research methods for understanding mobile technology use. doi
  16. (2005). Evaluating Mobile Learning: Reflections on Current Practice.
  17. (2005). Evaluation of a Mobile Learning Organiser for University Students. doi
  18. (2004). Examining mobile phone use in the wild with quasi-experimentation.
  19. (1998). Finding Evidence of Learning in Museum Settings.
  20. (2007). From ReplayTool to Digital Replay System. In
  21. (2008). Harnessing Technology: Next Generation Learning
  22. (2007). Have You Got Your PDA With You?... Denials and Accusations.
  23. (2008). Health and the Mobile Phone. doi
  24. (2009). IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited.
  25. (2003). Informality and formality doi
  26. (1998). Inside the black box: raising standards through classroom assessment. doi
  27. (2003). Iterative and Incremental Development: A Brief History. doi
  28. (2007). Learning Bridges: a role for mobile technologies in education. doi
  29. (1970). Managing the Development of Large Software Systems.
  30. (2001). Manifesto for Agile Software Development. Available online: www.agilemanifesto.org. Accessed on 22/01/09.
  31. (2004). Mobile Learning - The Ethical and Legal Challenges. doi
  32. (2000). Motivating the Academically Unmotivated: A Critical Issue for the 21st Centrury. doi
  33. (2006). Museum explorations in physical, personal and virtual space through MyArtSpace. doi
  34. (2004). New Techniques for Usability Evaluation of Mobile Systems. doi
  35. Proceedings of mLearn 2008: The bridge from text to context,
  36. (2007). Reconstructing an informal mobile learning experience with multiple data streams.
  37. (2002). Socio-cognitive engineering: a methodology for the design of humancentred technology. doi
  38. (2008). Sustainable Wireless Computing – How Public Information Systems Can Be Designed to Reduce Exposure.
  39. (1954). The critical incident technique. doi
  40. (1970). The development, field test and validation of an inventory of scientific attitudes. doi
  41. (1967). The methodology of evaluation. In doi
  42. (2000). The Six Sigma Way: How GE, Motorola, and Other Top Companies are Honing Their Performance. doi
  43. (2006). The Uses of PDAs and Smartphones in Informal learning. doi
  44. (2006). Towards a Task Model for Mobile Learning: a Dialectical Approach. doi
  45. (2004). User-Centered Internet Research: The Ethical Challenge. doi
  46. (1996). Using the balanced scorecard as a strategic management system. Harvard Business Review, doi
  47. (2002). What does ‘impact’ mean in the evaluation of learning technology?
  48. (2005). WP4: A Study of Mobile Learning Practices. MOBIlearn deliverable D4.4. Available online: http://

To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.