Framework for evaluating water quality information system performance

1994 Fall.Includes bibliographic references (pages 280-308).Water resource and water quality managers are being held increasingly accountable for the programs they manage. Much progress has been made in applying total systems perspectives to the design and operation of water quality monitoring and information programs, and towards rationalizing those programs with respect to management objectives and information needs. A recent example of that progress is the development of data analysis protocols to enhance the information system design process. However, further work is necessary to develop approaches which can help managers confront the water quality management environment of the future, which will be characterized by: (1) fewer purely technical questions, (2) more complex problems with social, economic, political and legal ramifications, and (3) actively managed and continuously improved water quality information systems. This research concludes that the management of water quality information systems for continuous improvement requires: (1) a competent system design process, (2) comprehensive documentation of system design and operation, and (3) a routine and thorough performance measurement and evaluation process. The framework for evaluating water quality information system performance presented in this dissertation integrates the experience of several disciplines into an instrument to help water quality managers accomplish these requirements. The framework embodies four phases: (1) evaluation planning, (2) watershed and management system analyses, (3) information system analysis, and (4) information system performance evaluation. The application of the framework is demonstrated in the evaluation of water quality monitoring programs associated with a unique municipal water transfer project. Water quality professionals of the U.S. Environmental Protection Agency and the U.S. Geological Survey are surveyed as to its potential application to large (e.g., regional or national) systems. Those exercises indicate the framework to be a convenient, economic, and flexible instrument useful towards enhancing water quality information system performance. Recommendations for future research to refine the framework and to extend its scope and utility are also presented

A dual approach to course evaluation (a case study)

The appraisal of teaching and the evaluation of courses in New Zealand polytechnics is topical as it has become a requirement of managers in polytechnics to implement staff appraisal and provide for staff development. All polytechnics have staff development personnel who are likely to be involved in either supervising staff appraisal and course evaluations or providing training in response to staff appraisal outcomes. In addition there are currently three regional training units in the country, providing initial level training to new staff. Trainers report that there has been an increase in the demand for information and advice on strategies for examining the effectiveness of both teaching and learning, with a focus on the improvement of course design and delivery. A request from the Media Centre within Christchurch Polytechnic to explore such strategies and a personal interest of the researcher to pilot the role of an external evaluator led to this research project. The department agreed to the presence of the external evaluator (who was known to all staff, and a staff member volunteered a 20 week course in Radio Journalism as the subject of the evaluation. This case study was aimed at designing an evaluation strategy, which used a variety of data collection methods, both quantitative and qualitative, to gain information about the course. The evaluation emphasised staff support and development, to facilitate course improvement and was to be primarily diagnostic and formative rather than judgemental and summative

Basic Communication Course Annual Vol. 14

Full issue (262 pages, 9.457 MB

Assessing Clinical Judgment Utilizing Embedded Patients In A Baccalaureate Nursing Program’s Simulation Laboratory

The purpose of this study was to determine if clinical judgment could be assessed in baccalaureate nursing students during a 15 to 20-minute embedded (live patient) simulation scenario using role play. Graduate nurses are expected to apply critical reasoning and be clinically competent when caring for patients when they enter the nursing profession. The use of simulation scenarios is an approved and effective supplement to the clinical experience of nursing students. This study was conducted due to the sparsity of evidence referencing clinical judgment in live patient simulation scenarios. This study used a convenience sample (n=22) of baccalaureate nursing students in their final semester of a nursing program. The study used a mixed-method (quantitative/qualitative) approach to assess clinical judgment. It was framed on Khalili’s clinical simulation practise framework. The clinical judgment section of Creighton’s Competency Evaluation Instrument (C-CEI®) measured observed clinical decision making based on nine dimensions or competencies. A self-assessment pretest/posttest utilized a five-point Likert scale and an open response to record participant’s perceptions of applied clinical judgment. Fisher’s exact test returned a p-value of .02 (α = .05) for the C-CEI dimensions and was considered statistically significant. Only two of the nine observed competencies met or exceeded a score of 77% (the passing grade for this program). The Likert scale mode of ‘4’ was self-reported on the pretest, while the posttest reported a mode of ‘3’ for clinical judgment. A Wilcoxon Signed-Ranks test showed a statistically-significant difference between the two exams (W Statistic = 8 and W Critical = 66). Open and axial coding identified two major themes (judgment, and scenario time) and three subcategories (emotions, task priorities, and missed opportunities). For this study, clinical judgment could not adequately be observed during short, live patient, simulation scenarios. Students did not report an increase in perceived clinical judgment following the scenario. Longer scenario times and the use of the entire C-CEI instrument (or other valid rubric) should be considered for future studies

Implementation Evaluation of a Critical Online Resource Evaluation (CORE) Program for High School Students in Quebec

Being literate in the 21st century goes beyond the abilities of reading and writing; new skills and methods are required to evaluate the credibility of the information we find online. It is frequently assumed that young students, the so-called 'digital born generation,’ can discern credible information from mis/dis information or biased information; however, evidence suggests they have limited critical evaluation skills. Aware of this gap, researchers from the University of Concordia and practitioners in three high schools in Quebec implemented a Critical Online Resource Evaluation (CORE) intervention program to improve students' ability to evaluate online resources. With the vision to scale up the intervention in Canada and worldwide, it was essential to understand how the program was implemented and whether it was delivered as intended by the research team, to make informed decisions about modifications, adaptations, or refinements to improve the program's effectiveness. Implementation fidelity was evaluated using a pragmatic mixed methods design, concurrently collecting and analyzing quantitative and qualitative data from three sources: teachers, researchers and students. The CORE intervention was implemented with high fidelity from both teacher and researcher perspectives (79.9%, 89.0% ). While teachers adhered to the program content as planned, they adapted it to meet their classroom context. Teachers conducted the program with a high level of quality, with a mean score of 90.3% from the teacher’s perspective and 96.7% from the researcher’s perspective, suggesting they were well-prepared and confident. Students' engagement and motivation varied from the different views. From the teacher's perspective, students were highly engaged, with a mean score of 80.9%, while researchers and students perceived they were moderately engaged and motivated (77%, 69.6%). Teachers, researchers, and students concurred that the program was long, complex and the COVID topic fatiguing. Finally, the intervention highlighted group work and interactivity as the most engaging and motivating factors

Linking Teacher Evaluation, Professional Growth, and Motivation: A Multiple-site Case Study

The primary purpose of this study was to investigate the nature of the activities, processes, and structures used to link a teacher evaluation program to professional growth and motivation. A total of 52 teachers in the Johnson City School System, who were scheduled for evaluation during the 1994-95 school year, were selected by both random and purposeful sampling techniques to participate. Data were collected through both quantitative and qualitative methods. Principals of the nine schools involved also participated in the naturalistic inquiry component of the study. Data were analyzed both deductively and inductively. The analysis revealed attitudes, behaviors, and perceptions of those involved in the implementation of a growth-oriented approach to teacher evaluation. What were the reasons for the success or failure of the program? Through data analysis the investigator identified 12 critical elements within four major categories that influence the linking of teacher evaluation, professional growth, and motivation. The four major categories are: characteristics of the culture, characteristics of the administrator, characteristics of the teacher, and characteristics of the process. The 12 critical elements were identified as follows: a trusting environment; collaborative relationships; high expectations of growth; administrators who are facilitators or coaches and resource providers; teachers who are mature, responsible, and self-directed; and a continuous process that is individualized, formative, and structured. Based on the findings, the following recommendations were suggested: (1) assess the culture of the school before implementing the Professional Teacher Evaluation Model, (2) provide training for teachers and administrators, (3) allow teachers to choose professional growth options, (4) identify teachers\u27 level of readiness for self-directive learning, (5) emphasize importance of the principal\u27s role, (6) ensure presence of the 12 critical elements identified, and (7) implement the Professional Teacher Evaluation Model as the professional growth option of the proposed Tennessee State Model for Local Evaluation

Parent Satisfaction with Staff Interactions at a Community Based Mental Health Center

This study investigates an important issue in mental health service provision, the level of parental satisfaction with staff interactions at a community based mental health center. A small target population, which included all families who had received services within a one year time frame, were surveyed via mail to determine levels of satisfaction. Parents responded to a questionnaire and data were collected to determine overall satisfaction, areas of service in need of improvement and areas of interaction which were most unsatisfactory. Additionally, data were compared to past study material to determine either an increase or decrease in parent satisfaction within a two year period. The results suggested that little change existed within the two year time frame. However, a marginal decline in satisfaction was found among the parents. Results indicated a need to evaluate the parents\u27 perception of staff helpfulness and the staff\u27s need to educate the parents in appropriate ways to live with their severely emotionally disturbed child. Additional recommendations included other important research areas within a community based mental health system, the application of individual interview procedures and improvements to instrumentation

A critical analysis of current methods of public sector project and programme evaluation in regeneration

This thesis presents a critique of public sector regeneration project and programme evaluation and develops an alternative and original evaluation framework. The framework has been successfully applied, in part, to a major national evaluation, demonstrating both its feasibility and significant added value. An important priority for the UK Government is regenerating areas of concentrated deprivation. The thesis provides, in Chapter 2, a review of the inter-linked problems facing such areas. It goes on to consider the nature and characteristics of the regeneration policies and programmes that have been developed to tackle these problems - noting their complex and multi-domain nature. The thesis demonstrates, in Chapter 3, that the public sector intervenes in the allocation of resources to secure certain societal objectives including, in particular, economic efficiency and equity. The thesis argues that regeneration programmes will often seek to meet a number of these objectives to varying degrees. If the public sector is to ensure that net social benefits of regeneration are maximised effective evaluation is critical. The thesis shows that the evaluation of regeneration programmes is extremely complex. It proposes in Chapter 4, a typology of evaluation methodologies and techniques. The thesis reviews current guidance on evaluations and identifies a number of key issues. The thesis presents a critique of various, recent evaluations of regeneration programmes and identifies significant weaknesses in them and their coverage of the key evaluation issues. It argues that complex regeneration programmes need a multi-faceted framework to evaluate them. The approach proposed is novel and combines multi-criteria value for money, cost benefit account and what works analyses, using both macro (top-down) and micro (bottom-up) approaches. Overall, the thesis sets out the results of distinctive and original research into the evaluation of regeneration projects and programmes. The alternative framework proposed represents a significant improvement on past practice