Design and Implementation of Performance Metrics for Evaluation of Assessments Data

The objective of this paper is to design performance metrics and respective formulas to quantitatively evaluate the achievement of set objectives and expected outcomes both at the course and program levels. Evaluation is defined as one or more processes for interpreting the data acquired through the assessment processes in order to determine how well the set objectives and outcomes are being attained. Even though assessment processes for accreditation are well documented but existence of an evaluation process is assumed. This paper focuses on evaluation process to provide insights and techniques for data interpretation. It gives a complete evaluation process from the data collection through various assessment methods, performance metrics, to the presentations in the form of tables and graphs. Authors hope that the articulated description of evaluation formulas will help convergence to high quality standard in evaluation process

Identifying, Monitoring, and Assessing Promising Innovations: Using Evaluation to Support Rapid-Cycle Change

Examines the new Center for Medicare and Medicaid Innovation's mission, critical issues, and challenges in finding effective ways to raise healthcare quality and lower costs, documenting innovation, and providing evidence to support broad policy change

Providing behaviour awareness in collaborative project courses

Several studies show that awareness mechanisms can contribute to enhance the collaboration process among students and the learning experiences during collaborative project courses. However, it is not clear what awareness information should be provided to whom, when it should be provided, and how to obtain and represent such information in an accurate and understandable way. Regardless the research efforts done in this area, the problem remains open. By recognizing the diversity of work scenarios (contexts) where the collaboration may occur, this research proposes a behaviour awareness mechanism to support collaborative work in undergraduate project courses. Based on the authors previous experiences and the literature in the area, the proposed mechanism considers personal and social awareness components, which represent metrics in a visual way, helping students realize their performance, and lecturers intervene when needed. The trustworthiness of the mechanisms for determining the metrics was verified using empirical data, and the usability and usefulness of these metrics were evaluated with undergraduate students. Experimental results show that this awareness mechanism is useful, understandable and representative of the observed scenarios.Peer ReviewedPostprint (published version

Quality Indicators Guiding Secondary Career and Technical Education Programs of Study

The purpose of this study was to examine quality indicators currently guiding the rigor of secondary career and technical education (CTE) programs of study in the United States. Quality indicators are desirable characteristics or expectations for a comprehensive and effective CTE program of study. As of May 2017, we were able to locate publicly accessible secondary CTE quality program standards/guidelines for 38 states. A majority (n=24) updated their secondary CTE quality program standards/guidelines within the last five years (i.e., 2012-2017). Deductive content analysis was conducted to examine the 38 state profiles using the Association of Career and Technical Education (ACTE) Quality CTE Program of Study Framework 4.0 for coding purposes. Common quality elements and key quality indicators were identified from those state documents, which supplements the ACTE Framework. Implications and examples for practice are also discussed

Providing behaviour awareness in collaborative project courses

Several studies show that awareness mechanisms can contribute to enhance the collaboration process among students and the learning experiences during collaborative project courses. However, it is not clear what awareness information should be provided to whom, when it should be provided, and how to obtain and represent such information in an accurate and understandable way. Regardless the research efforts done in this area, the problem remains open. By recognizing the diversity of work scenarios (contexts) where the collaboration may occur, this research proposes a behaviour awareness mechanism to support collaborative work in undergraduate project courses. Based on the authors previous experiences and the literature in the area, the proposed mechanism considers personal and social awareness components, which represent metrics in a visual way, helping students realize their performance, and lecturers intervene when needed. The trustworthiness of the mechanisms for determining the metrics was verified using empirical data, and the usability and usefulness of these metrics were evaluated with undergraduate students. Experimental results show that this awareness mechanism is useful, understandable and representative of the observed scenarios.Peer ReviewedPostprint (published version

Towards evaluation design for smart city development

Smart city developments integrate digital, human, and physical systems in the built environment. With growing urbanization and widespread developments, identifying suitable evaluation methodologies is important. Case-study research across five UK cities - Birmingham, Bristol, Manchester, Milton Keynes and Peterborough - revealed that city evaluation approaches were principally project-focused with city-level evaluation plans at early stages. Key challenges centred on selecting suitable evaluation methodologies to evidence urban value and outcomes, addressing city authority requirements. Recommendations for evaluation design draw on urban studies and measurement frameworks, capitalizing on big data opportunities and developing appropriate, valid, credible integrative approaches across projects, programmes and city-level developments

Model-driven performance evaluation for service engineering

Service engineering and service-oriented architecture as an integration and platform technology is a recent approach to software systems integration. Software quality aspects such as performance are of central importance for the integration of heterogeneous, distributed service-based systems. Empirical performance evaluation is a process of measuring and calculating performance metrics of the implemented software. We present an approach for the empirical, model-based performance evaluation of services and service compositions in the context of model-driven service engineering. Temporal databases theory is utilised for the empirical performance evaluation of model-driven developed service systems

Implementing Observation Protocols: Lessons for K-12 Education From the Field of Early Childhood

Examines issues for implementing standardized observation protocols for teacher evaluations. Makes recommendations based on lessons from preschool, such as the need to show empirical links between teacher performance and student learning and development