The excellence in research for Australia Scheme: An evaluation of the draft journal weights for economics

In February 2008, the Australian government announced its intention to develop a new quality and evaluation system for research conducted at the nation’s universities. Although the Excellence in Research for Australia (ERA) scheme will utilize several measures to evaluate institutional performance, we have chosen to focus on one element only: the assessment of refereed journal article output based on ERA’s own journal weighting scheme. The ERA weighting scheme will undoubtedly shape the reward structure facing university administrators and individual academics. Our objective is to explore the nature of the ERA weighting scheme for economics, and to demonstrate how it impacts on departmental and individual researcher rankings relative to rankings generated by alternative schemes employed in the economics literature. In order to do so, we utilize data from New Zealand’s economics departments and the draft set of journal weights (DERA) released in August 2008 by ERA officials. Given the similarities between Australia and New Zealand, our findings should have relevance to the Australian scene. As a result, we hope to provide the reader with a better understanding of the type of research activity that influences DERA rankings at both the departmental and individual level

Revisiting Student Evaluation of Teaching during the pandemic

The pandemic has placed unprecedented pressures upon staff and students alike. Yet performance management of academics including Student Evaluation of Teaching (SET) persists. The American Association of University Professors (AAUP) has intervened on this issue. We develop new methods enabling better treatment of pandemic-era SET. Analysis of UK National Student Survey (NSS) data suggests 85% of institutions meet reasonable performance expectations during the pandemic. Results emphasize the need for a more sensitive treatment of pandemic-era SET

A system for the simulation and evaluation of satellite communication networks

With the emergence of a new era in satellite communications brought about by NASA's thrust into the Ka band with multibeam and onboard processing technologies, new and innovative techniques for evaluating these concepts and systems are required. To this end, NASA, in conjunction with its extensive program for advanced communications technology development, has undertaken to develop a concept for the simulation and evaluation of a complete communications network. Incorporated in this network will be proof of concept models of the latest technologies proposed for future satellite communications systems. These include low noise receivers, matrix switches, baseband processors, and solid state and tube type high power amplifiers. To accomplish this, numerous supporting technologies must be added to those aforementioned proof of concept models. These include controllers for synchronization, order wire, and resource allocation, gain compensation, signal leveling, power augmentation, and rain fade and range delay simulation. Taken together, these will be assembled to comprise a system capable of addressing numerous design and performance questions. The simulation and evaluation system as planned will be modular in design and implementation, capable of modification and updating to track and evaluate a continuum emerging concepts and technologies

A Survey on Deep Learning-based Architectures for Semantic Segmentation on 2D images

Semantic segmentation is the pixel-wise labelling of an image. Since the problem is defined at the pixel level, determining image class labels only is not acceptable, but localising them at the original image pixel resolution is necessary. Boosted by the extraordinary ability of convolutional neural networks (CNN) in creating semantic, high level and hierarchical image features; excessive numbers of deep learning-based 2D semantic segmentation approaches have been proposed within the last decade. In this survey, we mainly focus on the recent scientific developments in semantic segmentation, specifically on deep learning-based methods using 2D images. We started with an analysis of the public image sets and leaderboards for 2D semantic segmantation, with an overview of the techniques employed in performance evaluation. In examining the evolution of the field, we chronologically categorised the approaches into three main periods, namely pre-and early deep learning era, the fully convolutional era, and the post-FCN era. We technically analysed the solutions put forward in terms of solving the fundamental problems of the field, such as fine-grained localisation and scale invariance. Before drawing our conclusions, we present a table of methods from all mentioned eras, with a brief summary of each approach that explains their contribution to the field. We conclude the survey by discussing the current challenges of the field and to what extent they have been solved.Comment: Updated with new studie

Face Recognition: Demystification of Multifarious Aspect in Evaluation Metrics

Face recognition has become an interesting research area in the recent era, and blends knowledge from various disciplines such as neuroscience, psychology, statistics, data mining, computer vision, pattern recognition, image processing, and machine learning. A new opportunity is obtained using the application of statistical methods for evaluating the performance of the system. Evaluation methods are the yardstick to examine the efficiency and performance of any face recognition system. Methods for performance evaluation seek to distinguish, compare, and interpret the various factors such as characteristics of subjects, location, illumination, and images. In this chapter, we show how to adapt popular performance measures commonly used in face recognition research, including—precision, recall, F-measure, fallout, accuracy, efficiency, sensitivity, specificity, error rate, receiver operating characteristics (ROC). This work serves as an introduction to performance measures, and as a practical guide for using them in research

Exploring Educators’ Decisions During the Era of New Professionalism : Teachers and Administrators Dialoguing Together in a Performance-Based Pay School District

The purpose of this action research study was to examine how teachers embedded in cultures of accountability, performance, and rewards, attempted to maintain integrity and professionalism in their instructional choices, with administrators as supportive partners. In addition, I aimed to explore how teachers and administrators balance power relations while negotiating this terrain. The research questions that guided this dissertation study were: 1. When given a supportive space for ongoing dialogue in the current era of new professionalism and neoliberalism, how do we as teachers and administrators describe our educational decisions while functioning in evaluation systems? 2. What kinds of actions might teachers recommend or consider taking regarding how administrators can best support their instructional decision-making in this era of heightened accountability? Teacher accountability policies since 2001 have changed the landscape of education in the United States, with a heightened emphasis on the yearly evaluation scores of educators. Hence, this study took place within the new era of heightened accountability and rewards cultures that prevailed in education. I conducted a participatory action research (PAR) study consisting of eight participants (seven teachers and myself, an administrator) who met weekly in a professional learning community (PLC) and journaled online about these sessions, as well as their everyday experiences. These findings add to research regarding the effects of new professionalism on both teachers and administrators and how an in-depth look at the daily interactions between these groups can inform future legislation and local decisions regarding educator practice and evaluation systems

Advances in mode-stirred reverberation chambers for wireless communication performance evaluation

Reverberation chambers (RC) are a popular tool for laboratory wireless communication performance evaluation, and their sandardization for Over-The-Air (OTA) measurements is underway. Yet, the inherent limitations of singlecavity RCs to emulate isotropic Rayleigh-fading scenarios with uniform phase distribution and high elevation angular spread put their representation of realistic scenarios into jeopardy. Recent advances in the last few years, however, have solved all these limitations by using more general mode-stirred reverberation chambers (MSC), wherein the number of cavities, their stirring and coupling mechanisms, and their software postprocessing algorithms is far from simple, representing a new era for wireless communications research, development, and over-the-air testing. This article highlights recent advances in the development of second-generation mode-stirred chambers for wireless communications performance evaluatio

Comparative analysis of live sports streaming services

In the current era of dynamic development of internet television, there is significant competition among providers of sports transmission services. In an effort to stand out, manufacturers introduce new functionalities, often disregarding the limitations associated with customers' access to high-speed internet. The aim of this article is to conduct a comparative analysis of sports event streaming services available in the Polish market, taking into account their performance under various network conditions. A survey was conducted among a specified research group, and a technical evaluation of the internet and mobile applications of three services was carried out. Both approaches revealed that Polsat Box Go is the service that performs best under any network conditions

Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation

Program synthesis has been long studied with recent approaches focused on directly using the power of Large Language Models (LLMs) to generate code according to user intent written in natural language. Code evaluation datasets, containing curated synthesis problems with input/output test-cases, are used to measure the performance of various LLMs on code synthesis. However, test-cases in these datasets can be limited in both quantity and quality for fully assessing the functional correctness of the generated code. Such limitation in the existing benchmarks begs the following question: In the era of LLMs, is the code generated really correct? To answer this, we propose EvalPlus -- a code synthesis benchmarking framework to rigorously evaluate the functional correctness of LLM-synthesized code. In short, EvalPlus takes in the base evaluation dataset and uses an automatic input generation step to produce and diversify large amounts of new test inputs using both LLM-based and mutation-based input generators to further validate the synthesized code. We extend the popular HUMANEVAL benchmark and build HUMANEVAL+ with 81x additionally generated tests. Our extensive evaluation across 14 popular LLMs demonstrates that HUMANEVAL+ is able to catch significant amounts of previously undetected wrong code synthesized by LLMs, reducing the pass@k by 15.1% on average! Moreover, we even found several incorrect ground-truth implementations in HUMANEVAL. Our work not only indicates that prior popular code synthesis evaluation results do not accurately reflect the true performance of LLMs for code synthesis but also opens up a new direction to improve programming benchmarks through automated test input generation