Contextual adaptation of the Personnel Evaluation Standards for assessing faculty evaluation systems in developing countries: the case of Iran

<p>Abstract</p> <p>Background</p> <p>Faculty evaluations can identify needs to be addressed in effective development programs. Generic evaluation models exist, but these require adaptation to a particular context of interest. We report on one approach to such adaptation in the context of medical education in Iran, which is integrated into the delivery and management of healthcare services nationwide.</p> <p>Methods</p> <p>Using a triangulation design, interviews with senior faculty leaders were conducted to identify relevant areas for faculty evaluation. We then adapted the published checklist of the Personnel Evaluation Standards to fit the Iranian medical universities' context by considering faculty members' diverse roles. Then the adapted instrument was administered to faculty at twelve medical schools in Iran.</p> <p>Results</p> <p>The interviews revealed poor linkages between existing forms of development and evaluation, imbalance between the faculty work components and evaluated areas, inappropriate feedback and use of information in decision making. The principles of Personnel Evaluation Standards addressed almost all of these concerns and were used to assess the existing faculty evaluation system and also adapted to evaluate the core faculty roles. The survey response rate was 74%. Responses showed that the four principles in all faculty members' roles were met <it>occasionally </it>to <it>frequently</it>. Evaluation of teaching and research had the highest mean scores, while clinical and healthcare services, institutional administration, and self-development had the lowest mean scores. There were statistically significant differences between small medium and large medical schools (p < 0.000).</p> <p>Conclusion</p> <p>The adapted Personnel Evaluation Standards appears to be valid and applicable for monitoring and continuous improvement of a faculty evaluation system in the context of medical universities in Iran. The approach developed here provides a more balanced assessment of multiple faculty roles, including educational, clinical and healthcare services. In order to address identified deficiencies, the evaluation system should recognize, document, and uniformly reward those activities that are vital to the academic mission. Inclusion of personal developmental concerns in the evaluation discussion is essential for evaluation systems.</p

Can programme theory be used as a 'translational tool’ to optimise health service delivery in a national early years’ initiative in Scotland: a case study

Background Theory-based evaluation (TBE) approaches are heralded as supporting formative evaluation by facilitating increased use of evaluative findings to guide programme improvement. It is essential that learning from programme implementation is better used to improve delivery and to inform other initiatives, if interventions are to be as effective as they have the potential to be. Nonetheless, few studies describe formative feedback methods, or report direct instrumental use of findings resulting from TBE. This paper uses the case of Scotland’s, National Health Service, early years’, oral health improvement initiative (Childsmile) to describe the use of TBE as a framework for providing feedback on delivery to programme staff and to assess its impact on programmatic action.&lt;p&gt;&lt;/p&gt; Methods In-depth, semi-structured interviews and focus groups with key stakeholders explored perceived deviations between the Childsmile programme 'as delivered’ and its Programme Theory (PT). The data was thematically analysed using constant comparative methods. Findings were shared with key programme stakeholders and discussions around likely impact and necessary actions were facilitated by the authors. Documentary review and ongoing observations of programme meetings were undertaken to assess the extent to which learning was acted upon.&lt;p&gt;&lt;/p&gt; Results On the whole, the activities documented in Childsmile’s PT were implemented as intended. This paper purposefully focuses on those activities where variation in delivery was evident. Differences resulted from the stage of roll-out reached and the flexibility given to individual NHS boards to tailor local implementation. Some adaptations were thought to have diverged from the central features of Childsmile’s PT, to the extent that there was a risk to achieving outcomes. The methods employed prompted national service improvement action, and proposals for local action by individual NHS boards to address this.&lt;p&gt;&lt;/p&gt; Conclusions The TBE approach provided a platform, to direct attention to areas of risk within a national health initiative, and to agree which intervention components were 'core’ to its hypothesised success. The study demonstrates that PT can be used as a 'translational tool’ to facilitate instrumental use of evaluative findings to optimise implementation within a complex health improvement programme.&lt;p&gt;&lt;/p&gt

people centred production design

This case reports on a worldwide operating SME producing floor cleaning machines. The SME distinguishes itself from its competitors by providing highly customizable high-quality products. Employees are one of the "most-valuable resources" to the management. However, the initial situation reveals significant improvement opportunities related to the employee involvement and empowerment concerning workplace re-design. The proposed subject-oriented solution aims to involve shop floor workers in workplace (re-)design by providing them structural empowerment means such as social media for suggestion proposals, discussions and negotiations. Furthermore, the newly introduced features are designed to allow for context-sensitive reporting of suggestions and errors. Context-sensitive elicitation provides the basis for analysing impacts of changes (e.g. the affected location or worker) and visualizing potential improvement areas within the shop floor. The generic suggestion and error handling process can be tailored to different organizations. The S-BPM process handling has been integrated with a semantic wiki allowing for context-sensitive workplace improvement elicitation and change propagation analysis. The evaluation reports on findings in developer workshops, focus groups and user tests conducted in parallel to the design and implementation to ensure a user-centred approach (formative part), and on findings related to the outcome of the case implementations at the given SME (summative part)

The design and evaluation of EKE, a semi-automated email knowledge extraction tool

This is a post-peer-review, pre-copyedit version of an article published in Knowledge Management Research and Practice. The definitive publisher-authenticated version, TEDMORI, S. and JACKSON, T., 2012. The design and evaluation of EKE, a semi-automated email knowledge extraction tool. Knowledge Management Research and Practice, 10 (1), pp. 79 - 88 is available online at: http://dx.doi.org/10.1057/kmrp.2011.40This paper presents an approach to locating experts within organisations through the use of the indispensable communication medium and source of information, email. The approach was realised through the email expert locator architecture developed by the authors, which uses email content in the modelling of individuals' expertise profiles. The approach has been applied to a real-world application, EKE, and evaluated using focus group sessions and system trials. In this work, the authors report the findings obtained from the focus groups sessions. The aim of the sessions was to obtain information about the participants' perceptions, opinions, underlying attitudes, and recommendations with regard to the notion of exploiting email content for expertise profiling. The paper provides a review of the various approaches to expertise location that have been developed and highlights the end-users' perspectives on the usability and functionality of EKE and the socio-ethical challenges raised by its adoption from an industrial perspective. © 2012 Operational Research Society. All rights reserved

The DARE study of relapse prevention in depression: design for a phase 1/2 translational randomised controlled trial involving mindfulness-based cognitive therapy and supported self monitoring

<p>Abstract</p> <p>Background</p> <p>Depression is a common condition that typically has a relapsing course. Effective interventions targeting relapse have the potential to dramatically reduce the point prevalence of the condition. Mindfulness-based cognitive therapy (MBCT) is a group-based intervention that has shown efficacy in reducing depressive relapse. While trials of MBCT to date have met the core requirements of phase 1 translational research, there is a need now to move to phase 2 translational research - the application of MBCT within real-world settings with a view to informing policy and clinical practice. The aim of this trial is to examine the clinical impact and health economics of MBCT under real-world conditions and where efforts have been made to assess for and prevent resentful demoralization among the control group. Secondary aims of the project involve extending the phase 1 agenda to an examination of the effects of co-morbidity and mechanisms of action.</p> <p>Methods/Design</p> <p>This study is designed as a prospective, multi-site, single-blind, randomised controlled trial using a group comparison design between involving the intervention, MBCT, and a self-monitoring comparison condition, Depression Relapse Active Monitoring (DRAM). Follow-up is over 2 years. The design of the study indicates recruitment from primary and secondary care of 204 participants who have a history of 3 or more episodes of Major Depression but who are currently well. Measures assessing depressive relapse/recurrence, time to first clinical intervention, treatment expectancy and a range of secondary outcomes and process variables are included. A health economics evaluation will be undertaken to assess the incremental cost of MBCT.</p> <p>Discussion</p> <p>The results of this trial, including an examination of clinical, functional and health economic outcomes, will be used to assess the role that this treatment approach may have in recommendations for treatment of depression in Australia and elsewhere. If the findings are positive, we expect that this research will consolidate the evidence base to guide the decision to fund MBCT and to seek to promote its availability to those who have experienced at least 3 episodes of depression.</p> <p>Trial Registration</p> <p>Australian New Zealand Clinical Trials Registry: <a href="http://www.anzctr.org.au/ACTRN12607000166471.aspx">ACTRN12607000166471</a></p

Princípios e padrões em metaavaliação: diretrizes para os programas de saúde Principles and standards in metaevaluation: guidelines for health programs

Este artigo apresenta questões imperativas para o uso de princípios e padrões em avaliação, permitindo destacar diretrizes para estudos de metaavaliação, tendo como exemplo os Estudos de Linha de Base (ELB) do Projeto de Expansão e Consolidação do Saúde da Família (Proesf). Essas questões são importantes em países em que a Política Nacional de Avaliação ainda é incipiente, como é o caso do Brasil, onde se observa a ausência de dispositivos legais, diretrizes, ou parâmetros que busquem claramente garantir a qualidade, credibilidade e reconhecimento da prática da avaliação. Apresenta-se uma breve revisão da literatura sobre princípios e normas que fundamentam as diretrizes gerais formuladas para uma metaavaliação formativa dos estudos de linha de base, orientada pela aplicação dos quatro padrões internacionais para avaliação das políticas sociais: utilidade, exeqüibilidade, propriedade e precisão, que têm igualmente servido de guia para o desenho dos estudos de avaliação de programas de promoção e proteção à saúde, aos quais se associam padrões de especificidade definidos por uma técnica de consenso mista baseada nas prescrições do Termo de Referência dos ELB. O principal interesse é incentivar o amplo uso dessas ferramentas na avaliação em saúde como um ponto crucial para aperfeiçoamento e desenvolvimento do SUS.<br>This article poses questions imperative for the use of principles and standards of evaluation allowing for pointing out directives for meta-evaluation on the example of the Baseline Studies (BLS) of PROESF. These questions are important in countries like Brazil, where a National Evaluation Policy is still in its beginning, and where one observes absence of legal devices, directives or parameters clearly seeking to ensure the quality, credibility and recognition of the practice of evaluation. We present a brief review of the literature about principles and norms constituting the bases for the general directives formulated for a formative metaevaluation of the baseline studies, oriented by the four international standards for evaluation of social policies &shy; utility, feasibility, propriety and accuracy. The same standards were used for guiding the design of studies for evaluating health promotion and protection programs, together with specificity standards defined by a mixed consensus technique based on the prescriptions of the Terms of Reference of the BLS. The main interest rests in encouraging a broad use of these evaluation tools in the field of health, as a crucial point for its improvement and the development of the Unified Health System