Evaluating Services Computing Systems Engineering Framework using An Acceptance Model

The gap between business services and IT services becomes a major concern in services computing. As an approach for service-based IT solution, services computing systems are promised to be able to bridge the gap between these services. The implementation will require an engineering framework as a guide to building the systems. The framework needs to be evaluated to provide important feedback to the framework development. This paper outlines the evaluation of SCSE framework through an acceptance model. The study develops an acceptance model based on the experiences of a group of engineers after using the framework to build smart campus services systems. A survey involving 54 systems engineers with various engineering backgrounds was conducted to assess the experiences of the engineers in using the framework. The results of the acceptance model show that both perceived ease of use, represented by the level of agreement (υ1) and perceived usefulness, represented by the level of importance (υ2) deliver good results almost for the entire stages of the proposed framework. In addition, the user experiences of using the proposed framework are in the acceptable levels. The contribution of this paper is an enrichment of the engineering methodologies for the service-oriented system from the perspective of services computing

Simulation Use in Pre-Licensure Nursing Programs: Assuring Excellence in New Nurse Competence and Confidence

Purpose: This program evaluation project aims were to review the current state of simulation experiences in pre-licensure undergraduate nursing programs in Washington State and determine policy recommendations related to the future use of simulation experiences in clinical nursing education. The evaluation compared student outcomes of NCLEX pass rates associated with clinical simulation versus traditional clinical nursing experiences. Programs were evaluated for compliance with INACSL Simulation Standards of Best Practice. Conceptual Framework: The NLN/Jeffries Theory provided the framework for an analysis of program evaluation data regarding using the INASCL Simulation Standards of Best Practice. Design/Method: The study design was a descriptive mixed method using a compilation of survey questions from the National Council State Boards of Nursing (NCSBN) Survey of Simulation Use in Pre-licensure Nursing Program Changes and Advancements and the Program Assessment Survey for Simulation (PASS). Ten completed surveys provided data for qualitative and quantitative analysis. Results: Variation exists between nursing programs related to clinical hours per course. All programs offer a variety of simulation experiences as part of their pedagogy. Variation was also noted in program use of simulation activities substituted for traditional clinical hours, with 1:1 ratio being used when hours were substituted. All programs were aware of the INACSL Standards and were in varying stages of full implementation of those best practices. All programs met and most programs exceeded the minimum passing standard for NCLEX. Conclusions: The simulation experience of the past year of Covid 19 supports the role of simulation in substitution for traditional clinical hours at both 1:1 and 2:1 ratio

Towards Systemic Evaluation in Turbulent Times – Second-order practice shift

The need for, and possibilities of, a second-order shift in evaluation practice are explored. Second-order evaluation practice enables an evaluator to improve practice as a skilled practitioner, acknowledging her embeddedness within an evaluand. The paper explores evaluation practice as experienced by professional evaluators, using ideas from developmental evaluation coupled with systemic evaluation in the tradition of systems thinking in practice (STiP). Systemic evaluation aims to capture systemic sensibilities – the bigger picture – of complex turbulent situations of change underpinning evaluands. Attributes of second-order practice with systemic evaluation are understood as being aligned with both systemic and systematic modes of evaluation praxis. Personal experiences are provided where this juxtaposing praxis has been found wanting. By example, a STiP framework is explored as heuristic support for making a second-order practice shift. The paper concludes with a discussion of some implications for developments in professionalising evaluation practice and research

Co-design of augmented reality textbook for children’s collaborative learning experience in primary schools

Augmented Reality (AR) is a recent technology that allows a seamless composition between virtual objects and the real world. This practice-based research uses the affordances of AR to design an AR textbook for collaborative learning experience. It identifies the key concepts of children s AR textbooks for the designing and evaluation of collaborative learning experiences. These concepts were used to develop a conceptual framework for the AR textbook that considers collaborative experience, learning and usability. Informed by these concepts, the research also has identified the design features which are unique to AR affordances which can be integrated in the school textbooks to develop a collaborative AR textbook for primary school children. The research follows a participatory design approach to involve the users of the AR textbook in the design process. The researcher has conducted three co-design studies involving primary school children and adults using cooperative inquiry techniques. The first study uses low-tech prototyping to find the overall direction of designing the AR textbook. After the development of the first AR textbook prototype, two formative evaluations have been conducted using cooperative inquiry critiquing, and layered elaboration techniques. Throughout these studies, a conceptual framework has been developed namely, Experience, Learn and Use (ELU) for the designing and evaluation of children s AR textbooks for collaborative learning experience. This framework is based on the adaption of Janet Read s Play, Learn, Use (PLU) model that defines children s relationships with the interactive technologies. The research proposes the ELU framework as a useful classification framework in the evaluation process, which informs the design features of the AR textbook which are related to the concepts of collaborative experience, learning and usability. The practical component of the thesis proposes a sample of an AR textbook that is integrated in the regular school curriculum. It demonstrates the design features which can be implemented in other textbooks to support collaborative learning experiences for primary school children. The documentation of the co-design process provides a practical framework for co-designing an AR textbook with children, as well as an evidence of using the ELU framework in practice. 4 This research also contributes in bridging the gap between AR and Child-Computer Interaction (CCI) communities, through the use of common CCI methods in the AR development. This research has resulted in key design principles which contribute original knowledge to the literature of the AR for children s education considering the CCI perspective. These important principles are informed by the collaborative experiences, learning and usability aspects that establish a framework for the design and evaluation of collaborative AR textbook for children. The eight identified principles by this research are, Joint Textbooks, Personalised AR Experience, Interactive AR Book, Communication-Based Learning, Rewarding AR feedback, Audio AR Textbook, Intuitive AR Markers, and Mutual AR Display. The research introduces the definition for each of the concepts and a demonstration of the related design features in the outcome of the AR textbook prototype

The View from across the Table: A Qualitative Study of Parent Experiences with the School Psychologist During Initial Evaluation Feedback Conferences

This qualitative study focuses on the experiences of 11 parents whose child received an evaluation at school by the school psychologist. By using a semi-structured interview, the author examined the initial evaluation process, or the first time a parent had his or her child evaluated by a school psychologist. A special focus was placed on the meeting in which the parent received results from the school psychologist. The results indicated that there is a lack of connection between parents and the evaluation process. This disconnect becomes wider over the course of the evaluation and at the end of the process, these parents are still searching for help for their children. This dissertation will discuss ideas for an improvement in reframing this process around a cognitive-behavioral framework; this will include developing a therapeutic alliance using empathy, learning the existing schema that a parent has for his or her child, and treating the evaluation feedback conference as an activating event for thoughts and feelings. Finally, practice guidelines for school psychologists to use during the evaluation process with parents will be suggested

Evaluation Policy and Organizational Evaluation Capacity Building: Application of an Ecological Framework across Cultural Contexts

Background: Research on the role and effects of evaluation policy is limited. Some research on the policy’s role in enhancing organizational evaluation capacity (EC) is beginning to accrue but to date it has been limited largely to global Western evaluation contexts. Purpose: We employed an ecological conceptual framework arising from our own empirical research to explore the interface between evaluation policy and EC in non-western contexts. We asked—To what extent does this framework resonate across these contexts? In the selected non-Western context, what are the salient variables moderating the relationship between policy and EC in the selected contexts? Are there differences across countries? Setting: The present research is focused on perceptions about evaluation culture and experiences in two countries situated in the Middle East and North Africa (MENA) region, namely Turkey and Jordan. Intervention: Not applicable. Research design: We conducted focus groups within the respective countries with a combined total of 18 participants associated with country-level voluntary organizations for professional evaluation (VOPE). Participants worked in government, non-governmental aid agencies, universities and private sector organizations. Data collection and analysis: We introduced the focus group participants to our ecological framework and then guided the conversation using semi-structured questions. Data were audio-recorded, transcribed and subsequently thematically analyzed using NVivo. Findings: The ecological framework was found to resonate well but the findings were weighted heavily toward macro-level contextual variables. Even though important contextual and cultural differences between Turkey and Jordan were evident, leadership emerged as a significant meso-level moderating variable in both settings. The discussion of the results included implications for ongoing research. Keywords: Evaluation capacity building; evaluation policy; program evaluation; cultural contex

Constructive eHealth evaluation:Lessons from evaluation of EHR development in 4 Danish hospitals

Abstract Background Information and communication sources in the healthcare sector are replaced with new eHealth technologies. This has led to problems arising from the lack of awareness of the importance of end-user involvement in eHealth development and of the difficulties caused by using traditional summative evaluation methods. The Constructive eHealth evaluation method (CeHEM) provides a solution to these problems by offering an evaluation framework for supporting and facilitating end-user involvement during all phases of eHealth development. The aim of this paper is to support this process by sharing experiences of the eHealth evaluation method used in the introduction of electronic health records (EHR) in the North Denmark Region of Denmark. It is the first time the fully developed method and the experiences on using the CeHEM in all five phases of a full lifecycle framework is presented. Methods A case study evaluation of the EHR development process in the North Denmark Region was conducted from 2004 to 2010. The population consisted of clinicians, IT professionals, administrators, and vendors. The study involved 4 hospitals in the region. Data were collected using questionnaires, observations, interviews, and insight gathered from relevant documents. Results The evaluation showed a need for a) Early involvement of clinicians, b) The best possible representation of clinicians, and c) Workload reduction for those involved. The consequences of not providing this were a lack of ownership of decisions and negative attitudes towards the clinical benefits related to these decisions. Further, the result disclosed that by following the above recommendations, and by providing feedback to the 4 actor groups, the physicians’ involvement was improved. As a result they took ownership of decisions and gained a positive attitude to the clinical benefits. Conclusions The CeHEM has proven successful in formative evaluation of EHR development and can point at important issues that need to be taken care of by management. The method provides a framework that takes care of feedback and learning during eHealth development. It can thus support successful eHealth development in a broader context while building on a well-known success factor: end-user involvement in eHealth development

A pragmatic cluster randomised trial evaluating three implementation interventions

Background Implementation research is concerned with bridging the gap between evidence and practice through the study of methods to promote the uptake of research into routine practice. Good quality evidence has been summarised into guideline recommendations to show that peri-operative fasting times could be considerably shorter than patients currently experience. The objective of this trial was to evaluate the effectiveness of three strategies for the implementation of recommendations about peri-operative fasting. Methods A pragmatic cluster randomised trial underpinned by the PARIHS framework was conducted during 2006 to 2009 with a national sample of UK hospitals using time series with mixed methods process evaluation and cost analysis. Hospitals were randomised to one of three interventions: standard dissemination (SD) of a guideline package, SD plus a web-based resource championed by an opinion leader, and SD plus plan-do-study-act (PDSA). The primary outcome was duration of fluid fast prior to induction of anaesthesia. Secondary outcomes included duration of food fast, patients' experiences, and stakeholders' experiences of implementation, including influences. ANOVA was used to test differences over time and interventions. Results Nineteen acute NHS hospitals participated. Across timepoints, 3,505 duration of fasting observations were recorded. No significant effect of the interventions was observed for either fluid or food fasting times. The effect size was 0.33 for the web-based intervention compared to SD alone for the change in fluid fasting and was 0.12 for PDSA compared to SD alone. The process evaluation showed different types of impact, including changes to practices, policies, and attitudes. A rich picture of the implementation challenges emerged, including inter-professional tensions and a lack of clarity for decision-making authority and responsibility. Conclusions This was a large, complex study and one of the first national randomised controlled trials conducted within acute care in implementation research. The evidence base for fasting practice was accepted by those participating in this study and the messages from it simple; however, implementation and practical challenges influenced the interventions' impact. A set of conditions for implementation emerges from the findings of this study, which are presented as theoretically transferable propositions that have international relevance. Trial registration ISRCTN18046709 - Peri-operative Implementation Study Evaluation (POISE

Neural Network Based Reinforcement Learning for Audio-Visual Gaze Control in Human-Robot Interaction

This paper introduces a novel neural network-based reinforcement learning approach for robot gaze control. Our approach enables a robot to learn and to adapt its gaze control strategy for human-robot interaction neither with the use of external sensors nor with human supervision. The robot learns to focus its attention onto groups of people from its own audio-visual experiences, independently of the number of people, of their positions and of their physical appearances. In particular, we use a recurrent neural network architecture in combination with Q-learning to find an optimal action-selection policy; we pre-train the network using a simulated environment that mimics realistic scenarios that involve speaking/silent participants, thus avoiding the need of tedious sessions of a robot interacting with people. Our experimental evaluation suggests that the proposed method is robust against parameter estimation, i.e. the parameter values yielded by the method do not have a decisive impact on the performance. The best results are obtained when both audio and visual information is jointly used. Experiments with the Nao robot indicate that our framework is a step forward towards the autonomous learning of socially acceptable gaze behavior.Comment: Paper submitted to Pattern Recognition Letter

Dissecting assessment: A paradigm shift towards technology-enhanced assessments

This study proposes a framework for making a paradigm shift from traditional (teacher-centred) to technology-enhanced (student-centred) assessment, using an example of an intelligent tutor. Informed by Situated Learning Theory that addresses students’ needs and concerns in timely learning experiences, the proposed ‘dissecting assessment’ framework has two primary variables: Students’ Expectations and Assessment Deliverable with positive and negative secondary variables such as inbuilt fear to handle failures and exposures and comparison phobia. Employing a case study approach, a purposeful sample of 14 U.S. College students were supported by an intelligent tutoring system in monitoring their learning with prompt corrective feedback in their physics course. This formative assessment prepares students for succeeding on their summative assessments, which is the final outcome of learning with feedback. The analysis of the proposed dissecting assessment framework led to the conclusion that concentrating efforts on the positives in the framework, such as unbiased evaluation, would eventually reduce the negatives, such as comparison phobia