The Impact of Interpretation Problems on Tutorial Dialogue

Supporting natural language input may improve learning in intelligent tutoring systems. However, interpretation errors are unavoidable and require an effective recovery policy. We describe an evaluation of an error recovery policy in the BEE-TLE II tutorial dialogue system and discuss how different types of interpretation problems affect learning gain and user satisfaction. In particular, the problems arising from student use of non-standard terminology appear to have negative consequences. We argue that existing strategies for dealing with terminology problems are insufficient and that improving such strategies is important in future ITS research.

Exploring User Satisfaction in a Tutorial Dialogue System

Abstract User satisfaction is a common evaluation metric in task-oriented dialogue systems, whereas tutorial dialogue systems are often evaluated in terms of student learning gain. However, user satisfaction is also important for such systems, since it may predict technology acceptance. We present a detailed satisfaction questionnaire used in evaluating the BEETLE II system (REVU-NL), and explore the underlying components of user satisfaction using factor analysis. We demonstrate interesting patterns of interaction between interpretation quality, satisfaction and the dialogue policy, highlighting the importance of more finegrained evaluation of user satisfaction

Threshold concepts: Impacts on teaching and learning at tertiary level

This project explored teaching and learning of hard-to-learn threshold concepts in first-year English, an electrical engineering course, leadership courses, and in doctoral writing. The project was envisioned to produce disciplinary case studies that lecturers could use to reflect on and refine their curriculum and pedagogy, thereby contributing to discussion about the relationship between theory and methodology in higher education research (Shay, Ashwin, & Case, 2009). A team of seven academics investigated lecturers’ awareness and emergent knowledge of threshold concepts and associated pedagogies and how such pedagogies can afford opportunities for learning. As part of this examination the lecturers also explored the role of threshold concept theory in designing curricula and sought to find the commonalities in threshold concepts and their teaching and learning across the four disciplines. The research highlights new ways of teaching threshold concepts to help students learn concepts that are fundamental to the disciplines they are studying and expand their educational experiences. Given that much of the international research in this field focuses on the identification of threshold concepts and debates their characteristics (Barradell, 2013; Flanagan, 2014; Knight, Callaghan, Baldock, & Meyer, 2013), our exploration of what happens when lecturers use threshold concept theory to re-envision their curriculum and teaching helps to address a gap within the field. By addressing an important theoretical and practical approach the project makes a considerable contribution to teaching and learning at the tertiary level in general and to each discipline in particular

Building self-evaluation skills through criterion-referenced assessment in public relations

Although technical skills in public relations are essential to practice, skills in self-evaluation, critical thinking, and problem solving are required when new practitioners move to management roles (Van Leuven, 1999). Public relations courses integrate specialist subject knowledge with graduate skill sets and capabilities in non-technical areas (Butcher & Stefani, 1995). Given that autonomy in learning is a skill valued by employers (Clifford, 1999) and advocated by accrediting professional bodies (Anderson, 1999), this study explores how public relations students build skills in and perceive the practice of self-evaluation. Currently, the public relations education literature presents a limited treatment of self-evaluation. Therefore, this study is guided mostly by the education literature and uses criterion-referenced assessment to determine how more than 150 students understand assessment requirements, assess their strengths and weaknesses, and interpret the differences between their self and their tutor's judgement of performance. The results indicate strong support for student understanding of assessment requirements and self-evaluation techniques but lower than expected support for understanding the differences between their self and tutor judgements. These findings are significant to educators, practitioners and professional bodies as they have implications for lifelong learning for public relations professionals

From classroom tutor to hypertext adviser: An evaluation

This paper describes a three‐year experiment to investigate the possibility of making economies by replacing practical laboratory sessions with courseware while attempting to ensure that the quality of the student learning experience did not suffer. Pathology labs are a central component of the first‐year medical undergraduate curriculum at Southampton. Activities in these labs had been carefully designed and they were supervised by lab demonstrators who were subject domain experts. The labs were successful in the eyes of both staff and students but were expensive to conduct, in terms of equipment and staffing. Year by year evaluation of the introduction of courseware revealed that there was no measurable difference in student performance as a result of introducing the courseware, but that students were unhappy about the loss of interaction with the demonstrators. The final outcome of this experiment was a courseware replacement for six labs which included a software online hypertext adviser. The contribution of this work is that it adds to the body of empirical evidence in support of the importance of maintaining dialogue with students when introducing courseware, and it presents an example of how this interaction might be achieved in software

Sociocultural theory and the teaching of process writing: The scaffolding of learning in a university context

This paper considers how independent and interdependent learning can be fostered through a process approach to the teaching of writing. It does so by presenting the theoretical rational which underlies a university academic skills programme. Drawing on reports of this programme which have been published elsewhere (e.g., Brine & Campbell, 2002), it is a case study illustrating how scaffolding can be effected by teachers and students. The paper begins by briefly reviewing three central concepts of sociocultural theory: the zone of proximal development, scaffolding, and appropriation. Attention is then turned to a consideration of writing as a collaborative process rather than as a product of solitary endeavour. Details are provided about a university course which applies sociocultural concepts to the adoption of a process approach to EAP writing. Attention is then given to the ways by which six principles of scaffolding (Van Lier, 1996) are applied throughout the course. Firstly, various forms of tutor scaffolding are outlined, and then a short sample of transcript data illustrates how students on this course can work collaboratively to co-construct texts and scaffold each other's learning. The paper concludes with a brief discussion of the broader pedagogical implications of sociocultural theory to the teaching of writing

Embedding generic employability skills in an accounting degree: development and impediments

This paper explores and analyses the views of, and effects on, students of a project that integrated the development of employability skills within the small group classes of two compulsory courses in the first year of an accounting degree at a UK university. The project aimed to build, deliver and evaluate course materials designed to encourage the development of a broad range of employability skills: skills needed for life-long learning and a successful business career. By analysing students' opinions gathered from a series of focus groups spread throughout the year, three prominent skill areas of interest were identified: time management, modelling, and learning to learn. Further analysis highlighted the complex nature of skills development, and brought to light a range of impediments and barriers to both students' development of employability skills and their subject learning. The analysis suggests the need for accounting educators to see skills development as being an essential element of the path to providing a successful accounting education experience

Strategic Approaches to the Development and Management of Personal Tutorial Systems in UK Higher Education

This chapter explores our experience, over nine years, of taking a strategic approach to personal tutoring within University of the Arts London. University of the Arts London comprises (at time of writing) five colleges and is specialist within the disciplines of art, design and communication. It is the biggest art and design educational institution in Europe and possibly the world. The chapter outlines the positive development we have been able to achieve in some colleges of the university and the tensions and difficulties encountered

Supporting undergraduate students’ acquisition of academic argumentation strategies through computer conferencing

Executive Summary Background This research grows out of work on the importance of argumentation in developingstudents’ critical abilities. It focuses attention on how students argue in computer mediated conferences as opposed to traditionalwritten assignments, investigating the way in which argumentation is realised within the relatively new context of computer conferencing which allows extended written discussions to take place overa period of weeks. Such text-based asynchronous conferencing is typically characterised by features of both spoken and written modes. Aims The main aims of the project were: • to investigate the argumentation strategies used in asynchronous text-based computer conferences; • to compare the argumentation strategies developed through conferencing with those used in the writing of academic assignments; • to examine the strategies used by tutors to encourage and facilitate argumentation in text-based computer conferences. Methods Data was collected over two years for the distance undergraduate course ‘Perspectives on Complementary and Alternative Medicine’ at the Open University.Qualitative data was obtained through interviews with the course chair, tutors and students, and through a student questionnaire. Assignments and computer-mediated tutorials were collected for textual analysis, although the timing of the assignments meant that analysis has only just begun on the essay data. To analyse the argumentation in the computer conferences and assignments a method of categorising, coding and tracking argumentative discourse was developed building on earlier work by the authors. In addition, computational searches were carried out to compare linguistic features across conference and assignment data. Results In tutorial conferences, student discussion tended to take the form of collaborative co-construction of an argument through exchanging information and experience to substantiate a position. However, students were also prepared to challenge other viewpoints. In both cases, they frequently drew on personal and professional experience to support argument claims. The use of these strategies suggests that text-based conferencing lends itself to the collective combining of diverse sources of information, experiences and ideas. Conference discussions were often personalised with fewer explicit logical links marking argument structure. They were also marked by complexity of argument strands, many of which reached no conclusion. Preliminary analysis of argumentation in assignments suggests that this did not, however, adversely affect students’ ability to create a more traditional, linear argument in their essays. Further analysis will be undertaken to compare argumentation strategies across the two sets of data. Tutors expressed concern about levels of participation in the tutorial conferences, which varied quite considerably. They also felt uncertain about their own knowledge of appropriate pedagogic strategies which would encourage students to participate in a collaborative yet critical way, and tended to rely on strategies from face-to-face teaching. Analysis of the conference discussion showed that tutors made fewer claims than students and were also less likely to provide information in support of their claims. There was, therefore, little modelling by tutors of the basic type of argumentation that would be expected in formal written assignments.Despite these concerns, student responses indicated that having a tutor and a group of peers to interact with, or just to observe, was valued as a supportive feature of this form of distance learning. No clear picture arose of how to make conferencing more interactive for more students, and this reinforces the sense gained from the tutor interviews of the difficulty of proposing a model of tutoring in computer conferences that will necessarily engage all students or raise the level of discussion and debate. Conclusions Our study suggests that text-based conferencing has an important role to play in developing students’ argumentation strategies and understanding of academic discourse and conventions. In view of its hybrid nature, somewhere between spontaneous speech and formal academic writing, course designers and tutors should aim to take advantage of both aspects – on the one hand, the informal dialogic exchange of opinions and co-construction of knowledge, and on the other,the opportunity for consolidation, reflection and re-positioning. Our findings reinforce the view that students’ willingness to exchange ideas freely and openly is partly a consequence of how personally engaged, at ease and confident students feel with one another and their tutor. In particular, it seems that there is a role for the interpersonal and, to some extent, the chat and the frivolity, which in some other studies discussed in the literature review have been regarded as negative influences. Recommendations To facilitate students’ development of argumentation and learning more generally,tutors need greater awareness of the ways in which academic argumentation operates in computer conferencing as compared to written assignments. Since pedagogic strategies developed in other contexts may not transfer well to computer conferencing, there is a need for targeted professional development, focussing in particular on: • Choosing topics for discussion and designing effective task prompts; • Supporting weaker students; • Encouraging challenging of ideas; • Finding the right tone to facilitate peer discussions. Some specific suggestions are made within the report, but our recommendations at this stage remain tentative as we still have to complete the analysis of the assignment data and draw conclusions about the impact of the computer conferencing on the quality of written argumentation within this more formal context