Location of Repository

Challenges in Evaluating Mobile Learning

By Giasemi N. Vavoula and Mike Sharples


This paper was presented at mLearn 2008 Conference, 7-10 Oct, Shropshire, UK and published in the proceedings. The published version is available at http://mlearning.noe-kaleidoscope.org/public/mlearn2008/www.mlearn2008.wlv.ac.uk/index.html.We propose six challenges in evaluating mobile learning: capturing and analysing learning in context and across contexts, measuring mobile learning processes and outcomes, respecting learner/participant privacy, assessing mobile technology utility and usability, considering the wider organisational and socio-cultural context of learning, and assessing in/formality. A three-level framework for evaluating mobile learning is presented, comprising a micro level concerned with usability, a meso level concerned with the learning experience, and a macro level concerned with integration within existing educational and organisational contexts. The paper concludes with a discussion of how the framework meets the evaluation challenges and with suggestions for further extensions

Topics: Mobile learning evaluation, learning context, evaluation framework
Publisher: School of Computing and Information Technology, University of Wolverhampton
Year: 2008
OAI identifier: oai:lra.le.ac.uk:2381/8162

Suggested articles



  1. (2001). A Briefing on Key Concepts: Formative and summative, criterion and norm-referenced assessment. Assessment Series, LTSN Generic Centre.
  2. (2007). A Framework for Conceptualising the Impact of Technology on Teaching and Learning."
  3. (2006). Adopting a Lifecycle Approach to the Evaluation of Computers and Information Technology.
  4. (2007). Analysis of an informal mobile learning activity based on activity theory.
  5. (1998). Assessment and Classroom Learning." Assessment doi
  6. (1995). Assessment and learning: contradictory or complementary? Assessment for Learning doi
  7. (2007). Educational Technology and Practice: Types and Timescales of Change."
  8. (2005). Emerging research methods for understanding mobile technology use. doi
  9. (2004). Examining mobile phone use in the wild with quasi-experimentation.
  10. (1998). Finding Evidence of Learning in Museum Settings. Evaluation and Visitor Research Special Interest Group Conference 'Visitors Centre Stage: Action for the Future',
  11. (2007). Have You Got Your PDA With You?...Denials and Accusations.
  12. (1998). Inside the black box: raising standards through classroom assessment." doi
  13. (2007). Learning Bridges: a role for mobile technologies in education." doi
  14. (2004). New Techniques for Usability Evaluation of Mobile Systems." doi
  15. (2006). Philips (2006b). Museum explorations in physical, personal and virtual space through MyArtSpace. mLearn
  16. (2007). Re-viewing the museum visitor’s view.
  17. (2007). Reconstructing an informal mobile learning experience with multiple data streams.
  18. Rudman (2006a). A lifecycle approach to evaluating MyArtSpace. doi
  19. (1967). The methodology of evaluation. Perspectives of Curriculum Evaluation. doi
  20. Vavoula (2007a). An Evaluation of MyArtSpace: a Mobile Learning Service for School Museum Trips. mLearn 2007, doi
  21. Vavoula (2007b). A Theory of Learning for the Mobile Age. The Sage Handbook of Elearning doi
  22. (2002). What does 'impact' mean in the evaluation of learning technology?"
  23. (2005). WP4: A Study of Mobile Learning Practices.

To submit an update or takedown request for this paper, please submit an Update/Correction/Removal Request.