Systematic evaluation of design choices for software development tools

[Abstract]: Most design and evaluation of software tools is based on the intuition and experience of the designers. Software tool designers consider themselves typical users of the tools that they build and tend to subjectively evaluate their products rather than objectively evaluate them using established usability methods. This subjective approach is inadequate if the quality of software tools is to improve and the use of more systematic methods is advocated. This paper summarises a sequence of studies that show how user interface design choices for software development tools can be evaluated using established usability engineering techniques. The techniques used included guideline review, predictive modelling and experimental studies with users

Evaluating Development Co-operation: Summary of Key Norms and Standards

This is a best practice that offers guidance in developing norms and standards for evaluation in an effort to improve humanitarian aid programs and policy. The approach offers a core set of principles, along with five evaluation criteria: relevance, effectiveness, efficiency, impact, sustainability, and a framework for peer reviews and assessment

Efficient Neural Network Implementations on Parallel Embedded Platforms Applied to Real-Time Torque-Vectoring Optimization Using Predictions for Multi-Motor Electric Vehicles

The combination of machine learning and heterogeneous embedded platforms enables new potential for developing sophisticated control concepts which are applicable to the field of vehicle dynamics and ADAS. This interdisciplinary work provides enabler solutions -ultimately implementing fast predictions using neural networks (NNs) on field programmable gate arrays (FPGAs) and graphical processing units (GPUs)- while applying them to a challenging application: Torque Vectoring on a multi-electric-motor vehicle for enhanced vehicle dynamics. The foundation motivating this work is provided by discussing multiple domains of the technological context as well as the constraints related to the automotive field, which contrast with the attractiveness of exploiting the capabilities of new embedded platforms to apply advanced control algorithms for complex control problems. In this particular case we target enhanced vehicle dynamics on a multi-motor electric vehicle benefiting from the greater degrees of freedom and controllability offered by such powertrains. Considering the constraints of the application and the implications of the selected multivariable optimization challenge, we propose a NN to provide batch predictions for real-time optimization. This leads to the major contribution of this work: efficient NN implementations on two intrinsically parallel embedded platforms, a GPU and a FPGA, following an analysis of theoretical and practical implications of their different operating paradigms, in order to efficiently harness their computing potential while gaining insight into their peculiarities. The achieved results exceed the expectations and additionally provide a representative illustration of the strengths and weaknesses of each kind of platform. Consequently, having shown the applicability of the proposed solutions, this work contributes valuable enablers also for further developments following similar fundamental principles.Some of the results presented in this work are related to activities within the 3Ccar project, which has received funding from ECSEL Joint Undertaking under grant agreement No. 662192. This Joint Undertaking received support from the European Union’s Horizon 2020 research and innovation programme and Germany, Austria, Czech Republic, Romania, Belgium, United Kingdom, France, Netherlands, Latvia, Finland, Spain, Italy, Lithuania. This work was also partly supported by the project ENABLES3, which received funding from ECSEL Joint Undertaking under grant agreement No. 692455-2

Lucene4IR: Developing information retrieval evaluation resources using Lucene

The workshop and hackathon on developing Information Retrieval Evaluation Resources using Lucene (L4IR) was held on the 8th and 9th of September, 2016 at the University of Strathclyde in Glasgow, UK and funded by the ESF Elias Network. The event featured three main elements: (i) a series of keynote and invited talks on industry, teaching and evaluation; (ii) planning, coding and hacking where a number of groups created modules and infrastructure to use Lucene to undertake TREC based evaluations; and (iii) a number of breakout groups discussing challenges, opportunities and problems in bridging the divide between academia and industry, and how we can use Lucene for teaching and learning Information Retrieval (IR). The event was composed of a mix and blend of academics, experts and students wanting to learn, share and create evaluation resources for the community. The hacking was intense and the discussions lively creating the basis of many useful tools but also raising numerous issues. It was clear that by adopting and contributing to most widely used and supported Open Source IR toolkit, there were many benefits for academics, students, researchers, developers and practitioners - providing a basis for stronger evaluation practices, increased reproducibility, more efficient knowledge transfer, greater collaboration between academia and industry, and shared teaching and training resources

An evaluation methodology for ergonomic design of electronic consumer products based on fuzzy axiomatic design

This article is posted with permission of OCP Science imprint. Copyright @ 2008 Old City Publishing Group.The development life cycle of software and electronic products has been shortened by the growth of rapid prototyping techniques. The evaluation of electronic consumer products should consider hardware and software as well as the ergonomic usability, emotional appeal and aesthetic integrity of the design. This research follows a systematic approach to develop an evaluation methodology for electronic mobile products on ergonomic design. The proposed methodology is based on fuzzy multi attribute decision making and fuzzy axiomatic design realized in three steps; determination of ergonomic attributes for electronic consumer products, determination of a representative set of alternatives, and selection of the best alternative in terms of ergonomic design by utilizing fuzzy axiomatic design. A case study is also provided to support the proposed methodology

Understanding Public Evaluation: Quantifying Experimenter Intervention

Public evaluations are popular because some research questions can only be answered by turning “to the wild.” Different approaches place experimenters in different roles during deployment, which has implications for the kinds of data that can be collected and the potential bias introduced by the experimenter. This paper expands our understanding of how experimenter roles impact public evaluations and provides an empirical basis to consider different evaluation approaches. We completed an evaluation of a playful gesture-controlled display – not to understand interaction at the display but to compare different evaluation approaches. The conditions placed the experimenter in three roles, steward observer, overt observer, and covert observer, to measure the effect of experimenter presence and analyse the strengths and weaknesses of each approach

Fast design optimization of UWB antenna with WLAN Band-Notch

In this paper, a methodology for rapid design optimization of an ultra-wideband ( UWB) monopole antenna with a lower WLAN band-notch is presented. The band-notch is realized using an open loop resonator implemented in the radiation patch of the antenna. Design optimization is a two stage process, with the first stage focused on the design of the antenna itself, and the second stage aiming at identification of the appropriate dimensions of the resonator with the purpose of allocating the band-notch in the desired frequency range. Both optimization stages are realized using surrogate-based optimization involving variable-fidelity electromagnetic ( EM) simulation models as well as an additive response correction ( first stage), and sequential approximate optimization ( second stage). The final antenna design is obtained at the CPU cost corresponding to only 23 high-fidelity EM antenna simulations

Teacher Evaluations in the Special Education Setting: Voices from the Field

National policy mandates have placed greater accountability on districts and schools to improve the outcomes of every student, including students with disabilities. Teacher evaluation systems that accurately identify effective teachers and their impact on student learning have resulted in the widespread design and implementation of new teacher evaluation systems. Nevertheless, limited exploration related to the evaluation of special education teachers exists. Using a qualitative, multi-case study research design, we sought to gain insight into the perceptions and experiences of special education teachers and administrators to better understand (a) the relationship between teacher evaluation and teacher effectiveness; (b) the ways in which educators approach the challenges of applying teacher evaluation systems for special education teachers; and (c) the ways in which teacher evaluation processes support the professional growth and development of special education teachers. The challenges involved in evaluating special education teachers with a uniform teacher evaluation protocol instrument are presented. The need to be able to accurately appraise special education teachers within an inclusive classroom setting, apply criteria from the observation protocol to special education students, and lack of appraisers� knowledge of the roles and expertise of special education teachers were identified. Recommendations for improvement are provided.Educatio

Lessons Learned on Gender Equality

The purpose of this evaluation study is twofold: Firstly, to compile lessons learned from Danida evaluations, evaluation studies and other evaluation publications, and secondly, to distil a set of recommendations relevant to the roll-out of The Right to a Better Life (2012), more particularly the planned update of Danida's strategy, Gender Equality in Danish Development Cooperation (2004). The evaluation study was carried out between June and September 2013. The methodology is based on a desk review of Danida evaluation publications carried out between 2004 and 2013, and comprises three interconnected phases:1. Screening of a long list of 104 evaluation publications2. Analysis of 26 shortlisted evaluations3. Reporting findings and recommendatio