Performance enhancement of the task assessment process through the application of an electronic performance support system

Higher education in Australia, as for many other countries, has changed greatly over the last 20 years at all levels and in many areas of operation including teaching, learning and assessment. The driving forces for these changes have been both internal and external, and have included factors such as: the increasing student population; the increasing use of part-time staff; a reduction in government funding; an increased expectation of institutional accountability; and the growing access and use of information and communication technologies (ICT) in teaching and learning. Assessment has not escaped these changes but in many cases has not kept up with exemplary and recommended practice. This is especially so in the specific area of task assessment that involves professional judgement, where assessment is a timeconsuming, expensive, poorly managed, and a stressful professional activity, and is often a negative emotional experience for both learners and educators

Using hypermedia to improve the dissemination and accessibility of syllabus documents with particular reference to primary mathematics

The fundamental question that this study set out to investigate was: Can the advantages of hypermedia be extended to curriculum materials that are for the sole use of teachers? To consider this question, three areas needed to be investigated: hypermedia (the medium); teachers (the target) and curriculum documents (the content). Hypermedia has a long history dating back to Bush (1986) who in 1945 imagined his Memex system as building information trails between ideas. However, it was not until the mid 1980s that technology caught up with the theory and hypermedia came of age. The evaluation of hypermedia documents is still in its infancy and design standards are still being formlulated. Social acceptability and usability will be of major concern in the evaluation process of hypermedia. Therefore this study needed to investigate whether this medium of presentation is socially acceptable to teachers? Advances in Information Technology (IT), both in hardware and software in the last few years have brought the potential of hypermedia to the personal computer (PC). Information, be it text, sound, graphics or video, or a mixture of these, can now be presented on the same screen and the movement between screens can be seamless. The movement between screens is no longer limited to sequential movement as it is when the information is presented in a hard copy form, but can be randomly accessed. This access allows the user to move about the information as they would move about within their own minds, that is, by association. Already commercial hypermedia products are being produced for the education and leisure markets. Teachers\u27 work loads are increasing, as they take on more curriculum responsibilities, while at the same time, information is expanding at a rapid rate. The challenge today is to encourage teachers to use new information technology to overcome these problems. However, since their inception into schools fifteen years ago, computers have not delivered the results that had been expected of them. Can the access to hypermedia curriculum documents help teachers to lessen their work load and encourage them to use IT? Firstly, it is important to consider whether curriculum materials for teacher use are suitable for hypermedia presentation. The literature indicated that textual materials that are not meant to be read sequentially like a novel, arc suitable to be presented in hypermedia form. At present, curriculum materials for teachers contain the content in hard copy form but the presentation is lacking in quality. This hard copy material is expensive, hard to correct and slow to update. Hypermedia offers the potential to overcome these limitations and to provide easy access to much more information. This new medium could allow teachers for the first ti.me to truly integrate their teaching programme by enabling them to access multiple curriculum documents. The methodology used in this study was based on two types of descriptive research, survey and correlation methods. The target population for this study was all K-7 teachers using the Western Australia Mathematics syllabus within Western Australia. The instrument was a mailed survey questionnaire that consisted of five parts. The first part consisted of collecting personal data such as age and gender. The second part was the Computer Attitude Scale (CAS), designed by Loyd and Gressard (1984), and was used to measure attitudes towards learning and using computers. The third part consisted of questions that asked teachers for their views and impressions on the social acceptability and utility of the present hard copy. The fourth part consisted of questions on computer experience and use, both in and outside the classroom. The final part consisted questions on the likely acceptance and usefulness of a hypermedia copy of the syllabus. This study found that the likely medium-based anxiety for this type of application is low for the teachers sampled, with 70 percent indicating that they were likely to accept this type of application. The findings indicated that the acceptance rate increased as the teachers\u27 positive attitude towards computers increased. Teachers that rated themselves competent at using a computer were also more likely to accept this type of application. Time spent using a computer at school showed that teachers who frequently use them at least several times a week were more likely to accept this type of application. The study also found that the majority of teachers sampled considered the ability to link the syllabus to other teaching material was very useful. Many of the problems identified by the teachers sampled concerning the usability of the present hard copy could be overcome using a hypermedia version

The Digital Filing Cabinet: Using Web-2 Technologies To Collaborate, Create, And Manage Student Information

This paper discusses the development and piloting of a digital filing system. It promotes the idea of the paperless office or teaching space of the future. This is a tool to improve efficiency, accountability and collaboration when managing data. This system allows you to have access to all your records wherever you are. It allows joint authoring and input of data. Teachers, managers and students, who need to store and manage important data, can customize the toll o meet their needs. It supports a vision of a learning community of the future where all the needed information is right at your fingertips where ever you are working. Key players can access records over the web at any time or place. There is also a stand-alone version on the desktop when you are not connected to a network. The system is designed to improve efficiency and effectiveness of staff. This paper documents the participatory action research model used in the development and evaluation of the system in which an ICT specialist and an educator worked collaboratively over two years to trial and pilot the system. This system is currently being applied in a tertiary setting, managing a course that runs over three campuses. The technology and principles of this interactive web-based student management system can be used in school settings for digital portfolios, managing assessment records or office settings for data management

I Hate Marking: An innovative use of technology to ease the Marking Day Blues

“I hate marking” reflects the feelings of many educators when faced with large piles of work to assess. This paper discusses the challenges and tensions in the assessment process, and considers ways of improving it, such as accommodating the learning styles of all stakeholders and incorporating technology. Two case studies are presented that include examples of how technology can be used in the assessment process to improve efficiency, streamline the administrative processes, and support the learning styles of students. The development of e-marking rubrics, podcasts and vodcasts are described as innovative ways to promote effective teaching and learning practices. Although these strategies are trialed in a tertiary setting, the methodology and technologies used can be adapted for any educational setting

Assessment in senior secondary physical education. Questions of judgement

The ways in which various aspects of senior physical education courses should be assessed and whether some can, or indeed should be incorporated in external examinations, are matters of longstanding professional debate across Australia and internationally. Differences in current practice across Australasia reflect an ongoing lack of consensus about how assessment requirements and arrangements and particularly, examinations in senior physical education, can best address concerns to ensure validity, reliability, equity and feasibility. An issue never far from such debates is that of ‘professional judgement’ and more specifically, whether and how professional judgement does and/or should ‘come into play’ in assessment. This paper reports on research that has explored new approaches to examination assessment and marking in senior physical education, using digital technologies. It focuses specifically on the ways in which ‘professional judgement’ can be deemed to be inherent to two contrasting methods of assessment used in the project: ‘analytical standardsbased’ assessment and ‘comparative pairs’ assessment. Details of each method of assessment are presented. Data arising directly from assessors’ comments and from analysis which explored intermarker reliability for each method of assessment and compared results generated by internal teacher assessment, standards-based and comparative pairs assessment, is reported. Discussion explores whether the data arising can be seen as lending weight to arguments for (i) more faith to be placed in professional judgement and (ii) for the comparative pairs methods to be more widely employed in examination assessment in senior physical education

Using iPad2 to Assess Students\u27 Live Performances and Actively Engage Students With Tutor and Peer Feedback

Assessing student live performances can be challenging because markers need to make quick and often complex judgements about the learning while at the same time record information and watch the performance. This is further challenged where multiple markers are involved and moderation between markers is required. Maintaining fairness and validity throughout the assessment process can consequently become a significant issue. Moderation of assessment can cause a delay in the turnaround time for student feedback because markers need to meet and review. In addition, the ‘busy type of work’ associated with compiling and sorting individual marks and distributing them to students, often further delays this process. This paper describes a two phase, qualitative, action research project which trialled the use of an innovative digital tool to streamline the assessment process when assessing live performances. Phase one involved the assessment of arts performances of 170 Bachelor of Education students and phase two involved 200 students. For each phase, the students were enrolled in a 12 week Arts Education unit in the third year of their course and were assessed in groups of 5 or 6 students. The digital assessment tool enabled each marker wireless access to a customised database during marking and moderation. Markers used laptops in phase one of the study, then iPad2 in phase two, as it enabled mobility during assessment. Each group’s performance video was embedded into their marking key. This made it quick and easy to locate and view. The digital tool automatically saved and collated data. At the completion of the marking and moderation period, the marking key with markers’ feedback as well as the embedded video of performance were automatically emailed to individuals as a pdf attachment. Individuals only received the feedback pertaining to their own group’s performance. The markers reported that the digital tool significantly enhanced the way in which they were able to capture and record their observations of complex learning. They felt that the assessment was more accurate and that the paperless process was far more efficient. The students reported that they became more engaged with the assessment process and that they engaged with their feedback on multiple occasions

Improving assessment outcomes through the application of innovative digital technologies

Assessing students’ live performances is challenging because the marker needs to make complex judgements, often very quickly, while at the same time recording information and watching the performance. This is further challenged when multiple markers are involved and moderation of marks is required. It can be difficult to maintain good assessment principles, such as fairness and validity and to offer students quality and timely feedback. This paper describes a two phase, qualitative, action research project that trialled the use of an innovative, digital technology supported, assessment tool designed to improve the efficiency and effectiveness of assessment and moderation of live performances. The digital assessment tool enabled students to engage with the assessment and feedback from tutors and peers multiple times. The project was initially trialled with 170 pre-service teachers (in phase one) and then 200 pre-service teachers (in phase two) enrolled in an arts education unit in the third year of their Bachelor of Education course. Literature is abundant with references of digital technology which is used to automate scoring and marks (Clarke-Midura & Dede, 2010), however, use of digital technology in this project does not replace the marker. Instead, it provides the marker with a tool with which to conduct and easily record rich observations of complex learning and it does so in a paperless, highly efficient and engaging way

Risk algorithm using serial biomarker measurements doubles the number of screen-detected cancers compared with a single-threshold rule in the United Kingdom collaborative trial of ovarian cancer screening

PURPOSE: Cancer screening strategies have commonly adopted single-biomarker thresholds to identify abnormality. We investigated the impact of serial biomarker change interpreted through a risk algorithm on cancer detection rates. PATIENTS AND METHODS: In the United Kingdom Collaborative Trial of Ovarian Cancer Screening, 46,237 women, age 50 years or older underwent incidence screening by using the multimodal strategy (MMS) in which annual serum cancer antigen 125 (CA-125) was interpreted with the risk of ovarian cancer algorithm (ROCA). Women were triaged by the ROCA: normal risk, returned to annual screening; intermediate risk, repeat CA-125; and elevated risk, repeat CA-125 and transvaginal ultrasound. Women with persistently increased risk were clinically evaluated. All participants were followed through national cancer and/or death registries. Performance characteristics of a single-threshold rule and the ROCA were compared by using receiver operating characteristic curves. RESULTS: After 296,911 women-years of annual incidence screening, 640 women underwent surgery. Of those, 133 had primary invasive epithelial ovarian or tubal cancers (iEOCs). In all, 22 interval iEOCs occurred within 1 year of screening, of which one was detected by ROCA but was managed conservatively after clinical assessment. The sensitivity and specificity of MMS for detection of iEOCs were 85.8% (95% CI, 79.3% to 90.9%) and 99.8% (95% CI, 99.8% to 99.8%), respectively, with 4.8 surgeries per iEOC. ROCA alone detected 87.1% (135 of 155) of the iEOCs. Using fixed CA-125 cutoffs at the last annual screen of more than 35, more than 30, and more than 22 U/mL would have identified 41.3% (64 of 155), 48.4% (75 of 155), and 66.5% (103 of 155), respectively. The area under the curve for ROCA (0.915) was significantly (P = .0027) higher than that for a single-threshold rule (0.869). CONCLUSION: Screening by using ROCA doubled the number of screen-detected iEOCs compared with a fixed cutoff. In the context of cancer screening, reliance on predefined single-threshold rules may result in biomarkers of value being discarded

Bayesian inference reveals positive but subtle effects of experimental fishery closures on marine predator demographics

Global forage-fish landings are increasing, with potentially grave consequences for marine ecosystems. Predators of forage fish may be influenced by this harvest, but the nature of these effects is contentious. Experimental fishery manipulations offer the best solution to quantify population-level impacts, but are rare. We used Bayesian inference to examine changes in chick survival, body condition and population growth rate of endangered African penguins Spheniscus demersus in response to 8 years of alternating time-area closures around two pairs of colonies. Our results demonstrate that fishing closures improved chick survival and condition, after controlling for changing prey availability. However, this effect was inconsistent across sites and years, highlighting the difficultly of assessing management interventions in marine ecosystems. Nevertheless, modelled increases in population growth rates exceeded 1% at one colony; i.e. the threshold considered biologically meaningful by fisheries management in South Africa. Fishing closures evidently can improve the population trend of a forage-fish-dependent predator-we therefore recommend they continue in South Africa and support their application elsewhere. However, detecting demographic gains for mobile marine predators from small no-take zones requires experimental time frames and scales that will often exceed those desired by decision makers

The institutional shaping of management: in the tracks of English individualism

Globalisation raises important questions about the shaping of economic action by cultural factors. This article explores the formation of what is seen by some as a prime influence on the formation of British management: individualism. Drawing on a range of historical sources, it argues for a comparative approach. In this case, the primary comparison drawn is between England and Scotland. The contention is that there is a systemic approach to authority in Scotland that can be contrasted to a personal approach in England. An examination of the careers of a number of Scottish pioneers of management suggests the roots of this systemic approach in practices of church governance. Ultimately this systemic approach was to take a secondary role to the personal approach engendered by institutions like the universities of Oxford and Cambridge, but it found more success in the different institutional context of the USA. The complexities of dealing with historical evidence are stressed, as is the value of taking a comparative approach. In this case this indicates a need to take religious practice as seriously as religious belief as a source of transferable practice. The article suggests that management should not be seen as a simple response to economic imperatives, but as shaped by the social and cultural context from which it emerges