Measuring and comparing the reliability of the structured walkthrough evaluation method with novices and experts

Effective evaluation of websites for accessibility remains problematic. Automated evaluation tools still require a significant manual element. There is also a significant expertise and evaluator effect. The Structured Walkthrough method is the translation of a manual, expert accessibility evaluation process adapted for use by novices. The method is embedded in the Accessibility Evaluation Assistant (AEA), a web accessibility knowledge management tool. Previous trials examined the pedagogical potential of the tool when incorporated into an undergraduate computing curriculum. The results of the evaluations carried out by novices yielded promising, consistent levels of validity and reliability. This paper presents the results of an empirical study that compares the reliability of accessibility evaluations produced by two groups (novices and experts). The main results of this study indicate that overall reliability of expert evaluations was 76% compared to 65% for evaluations produced by novices. The potential of the Structured Walkthrough method as a useful and viable tool for expert evaluators is also examined. Copyright 2014 ACM

Becoming Psychotherapists: Experiences of Novice Trainees in a Beginning Graduate Class

The authors investigated the experiences related to becoming psychotherapists for 5 counseling psychology doctoral trainees in their first prepracticum course. Qualitative analyses of weekly journals indicated that trainees discussed challenges related to becoming psychotherapists (e.g., being self-critical, having troubling reactions to clients, learning to use helping skills), gains made during the semester related to becoming psychotherapists (e.g., using helping skills more effectively, becoming less self-critical, being able to connect with clients), as well as experiences in supervision and activities that helped them cope with their anxieties. Results are discussed in 5 broad areas: feelings about self in role of psychotherapist, awareness of reactions to clients, learning and using helping skills, reactions to supervision, and experiences that fostered growth. Implications for training and research are provided

Motorcycle rider safety project: summary report

Motorcycle trauma is a serious issue in Queensland and throughout Australia; the fatality rate per 100 million kilometres travelled for motorcycle riders in Australia is nearly 30 times the rate for drivers of other vehicles (Australian Transport Safety Bureau, 2002). In 2009, the then Queensland Transport (later the Department of Transport and Main Roads or TMR) appointed CARRS-Q to provide a three-year program of Road Safety Research Services for Motorcycle Rider Safety. Funding for this research originated from the Motor Accident Insurance Commission. This program of research was undertaken to produce knowledge to assist TMR to improve motorcycle safety by further strengthening the licensing and training system to make learner riders safer by developing a pre-learner package (Deliverable 1), and by evaluating the Q-Ride CAP program to ensure that it is maximally effective and contributes to the best possible training for new riders (Deliverable 2), and identifying potential new licensing components that will reduce the incidence of risky riding and improve higher-order cognitive skills in new riders (Deliverable 3)

Comparative Judgment of Novice and Expert on Internal Control Tasks: Assessment on Work Effort and Ethical Orientation

AbstractAuditors’ judgment performances are being questioned and the public has become less trusting on profession's ability and willingness in protecting the interests of investor and stakeholders. There are three objectives addressed in this study with the first two objectives is to analyze the relationship between effort and personal ethical orientation on audit judgment performance. The third objective is to examine the differences in the audit judgment performance between audit trainees and auditors. A research instrument was developed and included a set of audit case and Forsyth's (1980) Ethics Position Questionnaire. The results show that there is a significant relationship between the respondent's effort and audit judgment performance

The role of presentation format on decision-makers' behaviour in accounting

The recent increase in researching presentation format area is resulting in an increase in awareness of the importance of presentation format on decision-makers' behaviour. This paper presents a synthesis of prior research on presentation format in the accounting literature which could be used as bases and references for future research. It reviews and evaluates existing accounting literature that examines the linkages of presentation format on decision-makers behaviour. Finally, future research opportunities in this area are made

Building Schemata for Tax Provision Learning Based On Cognitive Load Theory and Constructivism

This study investigated whether different instructional methodologies have an impact on learning a complex accounting task: tax provision work. To become proficient in making tax provision judgments, an accountant must understand the rules and principles of GAAP and the rules and regulations governing income tax reporting. However, these two sets of rules are often in direct opposition. Using cognitive load theory and constructivist learning theory as a framework, this study predicted how schema acquisition, a key component of learning, could improve tax provision performance. Greater schema acquisition should in turn lead to more accurate performance. Hypotheses include the following: (a) participants who learn using a systems instructional method will perform better than participants who learn using traditional instructional method, (b) participants who practice actively should perform better than participants who practice passively, and (c) the relationship between instructional method and performance will be moderated by the practice method. These hypotheses were tested using a 2 x 2 between-subjects experiment, manipulating instructional method and practice method as independent variables. The results of this study are inconclusive. The statistically significant findings are invalid due to potentially unequal pre-tax knowledge among the subjects, and several hypotheses were not supported. The results of this study had the potential to benefit theory, practice, and education by identifying the most effective combination of instructional method and practice method to build a tax provision schema in a novice learner. However, due to design flaws, this study did not realize that potential

Implementing STARTALK-endorsed principles in an intensive summer LCTL program: Challenges and implications

This paper discusses how STARTALK-endorsed principles were adopted in a first-year STARTALK summer program in three LCTLs: Arabic, Persian and Turkish. Our 34 high school and college participants were enrolled in seven week, six-credit intensive introductory college courses. All students began at the Novice-low proficiency level in each LCTL. The program effectiveness was assessed by students’ learning outcomes (ACTFL-based proficiency test results and LinguaFolio evidence) and evaluated by a site visit team’s observation report, program exit surveys and student project work. Students’ oral proficiency scores showed a significant improvement ranging from the Novice level to Intermediate low/high. Triangulation of the qualitative data also revealed overall positive attitudes toward this immersive and learner-centered program that integrated language, culture, and content in LCTLs. Positive results did not come without some challenges in curriculum design and technological integration. Lessons learned and pedagogical implications are provided for others interested in implementing similar programs

An examination of methods of improving audit judgment

This study examined three methods of improving audit judgment, namely inoculation, the group process, and counter explanation. Prior research found that auditors\u27 judgments were not always optimum. An important question this study asked was whether the use of inoculation, the group process and counter explanation leads to more effective judgments. In Experiment One, I proposed that inoculation will reduce the effect of framing so that participants exposed to the inoculation treatment will not display a framing effect, while participants not exposed to the treatment will. In addition, I hypothesised that the relation between audit experience and responsiveness to training has an inverted-U shaped value function. The results showed this to be the case. No interaction effect between framing and inoculation was observed, but the order of writing supporting and opposing arguments as required by the inoculation treatment led to a primacy effect. The second experiment posited that the use of audit groups will improve individual auditors\u27 judgments in a going-concern evaluation task. The results showed that not only were audit group judgments more conservative, but consensus was also higher among audit group judgment

CPAS/CCM experiences: Perspectives for AI/ES research in accounting

https://egrove.olemiss.edu/dl_proceedings/1111/thumbnail.jp