Providing Validity Evidence for a Speaking Test Using FACETS

Speaking has been considered one of the most important skills in second language teaching and assessment. However, to date there has not been a clear definition of speaking ability. The present study attempts to examine what it means to be able to speak a second language using a newly designed computer-delivered speaking test. The test was created based on a theoretical definition of speaking ability, and then administered to 95 ESL students. A variety of statistical analyses were employed to examine the validity of the test, including reliability analyses, correlation analyses, and a many-facet Rasch measurement (Linacre, 1989) analysis. Results seem to provide some evidence for the validation of the test

Plug-in, Trainable Gate for Streamlining Arbitrary Neural Networks

Architecture optimization, which is a technique for finding an efficient neural network that meets certain requirements, generally reduces to a set of multiple-choice selection problems among alternative sub-structures or parameters. The discrete nature of the selection problem, however, makes this optimization difficult. To tackle this problem we introduce a novel concept of a trainable gate function. The trainable gate function, which confers a differentiable property to discretevalued variables, allows us to directly optimize loss functions that include non-differentiable discrete values such as 0-1 selection. The proposed trainable gate can be applied to pruning. Pruning can be carried out simply by appending the proposed trainable gate functions to each intermediate output tensor followed by fine-tuning the overall model, using any gradient-based training methods. So the proposed method can jointly optimize the selection of the pruned channels while fine-tuning the weights of the pruned model at the same time. Our experimental results demonstrate that the proposed method efficiently optimizes arbitrary neural networks in various tasks such as image classification, style transfer, optical flow estimation, and neural machine translation.Comment: Accepted to AAAI 2020 (Poster

Ambivalence toward bureaucracy: Responses from Korean school principals

Purpose – Given the context of accountability-driven policy environments, research has shown that school leaders perceive bureaucratic rules and protocols in negative ways, but they also utilize organizational structures and routines to lead changes. To better understand both enabling and hindering mechanisms of bureaucracy in schools, this study explores how Korean school principals understand and perceive bureaucratic structures using a lens of ambivalence. The authors draw on Weber’s theory of bureaucracy, with a particular focus on the paradoxical aspect of bureaucracy that might be experienced by individuals within the system. Design/methodology/approach – This study analyzed qualitative data collected from 26 in-depth interviews with 10 Korean school principals between 2013 and 2015. The authors used the multiple cycles of coding to explore patterns and themes that emerged from the participants’ responses. Findings – The analysis of this study showed that the participants’ ambivalent responses toward bureaucracy were particularly salient in three areas where formal organizational structures were changing through policy initiatives: teacher evaluation, electronic approval system and school-based management promoting decentralized decision making. The study participants reflected on how such changes can enable and/or hinder schools to achieve organizational goals and collective values, from the viewpoints of multiple aspects, which led to their ambivalent responses to bureaucratic structures in school settings. Originality/value – This study contributes to the understanding of school organizations by revisiting Weber’s theory of bureaucracy in school settings. Using the lens of ambivalence enabled us to reconcile school principals’ contradictory perceptions toward bureaucracy, which complicates analyses of tensions and paradoxical responses found in leadership practices within school systems

Comparing Sample-wise Learnability Across Deep Neural Network Models

Estimating the relative importance of each sample in a training set has important practical and theoretical value, such as in importance sampling or curriculum learning. This kind of focus on individual samples invokes the concept of sample-wise learnability: How easy is it to correctly learn each sample (cf. PAC learnability)? In this paper, we approach the sample-wise learnability problem within a deep learning context. We propose a measure of the learnability of a sample with a given deep neural network (DNN) model. The basic idea is to train the given model on the training set, and for each sample, aggregate the hits and misses over the entire training epochs. Our experiments show that the sample-wise learnability measure collected this way is highly linearly correlated across different DNN models (ResNet-20, VGG-16, and MobileNet), suggesting that such a measure can provide deep general insights on the data's properties. We expect our method to help develop better curricula for training, and help us better understand the data itself.Comment: Accepted to AAAI 2019 Student Abstrac