This paper describes initial work on developing a semantic annotation system for the augmentation of skills-based learning for Healthcare. Scenario driven skills-based learning takes place in an augmented hospital ward simulation involving a patient simulator known as SimMan. The semantic annotation software enables real-time annotations of these simulations for debriefing of the students, student self study and better analysis of the learning approaches of mentors. A description of the developed system is provided with initial findings and future directions for the work.<br/