Design and Implementation of Performance Metrics for Evaluation of Assessments Data

The objective of this paper is to design performance metrics and respective formulas to quantitatively evaluate the achievement of set objectives and expected outcomes both at the course and program levels. Evaluation is defined as one or more processes for interpreting the data acquired through the assessment processes in order to determine how well the set objectives and outcomes are being attained. Even though assessment processes for accreditation are well documented but existence of an evaluation process is assumed. This paper focuses on evaluation process to provide insights and techniques for data interpretation. It gives a complete evaluation process from the data collection through various assessment methods, performance metrics, to the presentations in the form of tables and graphs. Authors hope that the articulated description of evaluation formulas will help convergence to high quality standard in evaluation process

An evaluation framework for data competitions in TEL

This paper presents a study describing the development of an Evaluation Framework (EF) for data competitions in TEL. The study applies the Group Concept Method (GCM) to empirically depict criteria and their indicators for evaluating software applications in TEL. A statistical analysis including multidimensional scaling and hierarchical clustering on the GCM data identified the following six evaluation criteria: 1.Educational Innovation, 2.Usability, 3.Data, 4.Performance, 5.Privacy, and 6.Audience. Each of them was operationalized through a set of indicators. The resulting Evaluation Framework (EF) incorporating these criteria was applied to the first data competition of the LinkedUp project. The EF was consequently improved using the results from reviewers' interviews, which were analysed qualitatively and quantitatively. The outcome of these efforts is a comprehensive EF that can be used for TEL data competitions and for the evaluation of TEL tools in general. The final publication is available at Springer via http://dx.doi.org/10.1007/978-3-319-11200-8_6.EC/FP7/LinkedUpEC/FP7/DURAAR

A Tripartite Framework for Leadership Evaluation

The Tripartite Framework for Leadership Evaluation provides a comprehensive examination of the leadership evaluation landscape and makes key recommendations about how the field of leadership evaluation should proceed. The chief concern addressed by this working paper is the use of student outcome data as a measurement of leadership effectiveness. A second concern in our work with urban leaders is the absence or surface treatment of race and equity in nearly all evaluation instruments or processes. Finally, we call for an overhaul of the conventional cycle of inquiry, which is based largely on needs analysis and leader deficits, and incomplete use of evidence to support recurring short cycles within the larger yearly cycle of inquiry

Empowering manufacturing personnel through functional understanding

A growing interest in organizational knowledge management, along with increasingly widespread adoption of Quality Standards such as ISO 9001, has increasingly led organizations to implement training programs for all employees. Training for the manufacturing workforce, however, remains limited to informal “On-the-Job” training, administered by peer colleagues or supervisors - particularly in Small and Medium Enterprises (SMEs) where economic, educational, cognitive and cultural constraints to training are often deeply embedded. This paper proposes a methodology for training the manufacturing workforce on the functions of products and their constituent parts, and presents a case study conducted in a UK-based manufacturing SME - aiming to verify our two research hypotheses: Functional Analysis Diagrams (FAD) of the company’s products and parts would assist in knowledge assimilation; and, the knowledge assimilation has a positive effect on work quality and productivity levels. This intervention provided training on the purpose of the processes the participants are involved, aiming to empower them in supporting the optimization of these same processes. By using surveys and applying statistical inference on long-term quantitative data, the study confirmed subjective observations of substantial improvements in work quality (scrap reduction of 63%) and increased productivity (setup time reduced by 67%). To our knowledge, we were the first to examine the effect of functional modelling methods for workforce training in a manufacturing setup. Although this paper presents a single case study, the results suggest that the proposed methodology can be a promising solution for the industry

MANAGING POLICY NETWORKS: A SOCIAL MARKETING- AND COLLECTIVE INTELLIGENCE SYSTEMS-DRIVEN VIEW

This research contributes a new view of Policy Networks (PN) management. The research object is a successful PN practice in the Basque Country (BC) over an 8-year period, in relation to Local Agenda 21 (LA21) promotion. The Basque experience is studied using a qualitative and a quantitative approach. PNs are viewed as social marketing-driven collective intelligence systems built to have an effect on municipality commitment to LA21 (in terms of value, satisfaction and loyalty). The research concludes that by fostering the co-development ‘genome’ (a mix of co-decision, co-creation, love, glory and money ‘genes’) a commitment to the new tool is achieved.

Risk-sensitive Inverse Reinforcement Learning via Semi- and Non-Parametric Methods

The literature on Inverse Reinforcement Learning (IRL) typically assumes that humans take actions in order to minimize the expected value of a cost function, i.e., that humans are risk neutral. Yet, in practice, humans are often far from being risk neutral. To fill this gap, the objective of this paper is to devise a framework for risk-sensitive IRL in order to explicitly account for a human's risk sensitivity. To this end, we propose a flexible class of models based on coherent risk measures, which allow us to capture an entire spectrum of risk preferences from risk-neutral to worst-case. We propose efficient non-parametric algorithms based on linear programming and semi-parametric algorithms based on maximum likelihood for inferring a human's underlying risk measure and cost function for a rich class of static and dynamic decision-making settings. The resulting approach is demonstrated on a simulated driving game with ten human participants. Our method is able to infer and mimic a wide range of qualitatively different driving styles from highly risk-averse to risk-neutral in a data-efficient manner. Moreover, comparisons of the Risk-Sensitive (RS) IRL approach with a risk-neutral model show that the RS-IRL framework more accurately captures observed participant behavior both qualitatively and quantitatively, especially in scenarios where catastrophic outcomes such as collisions can occur.Comment: Submitted to International Journal of Robotics Research; Revision 1: (i) Clarified minor technical points; (ii) Revised proof for Theorem 3 to hold under weaker assumptions; (iii) Added additional figures and expanded discussions to improve readabilit

A new classification approach: improving the regional ecosystem classification system in Queensland, Australia

Eda Addicott designed a quantitative approach for identifying the plant communities in Queensland. She found that plant communities identified using the new approach were more recognisable and more useful for land management planning. The Queensland Herbarium is using the new approach to identify plant communities as a state-wide standard