Fidelity-Weighted Learning

Training deep neural networks requires many training samples, but in practice training labels are expensive to obtain and may be of varying quality, as some may be from trusted expert labelers while others might be from heuristics or other sources of weak supervision such as crowd-sourcing. This creates a fundamental quality versus-quantity trade-off in the learning process. Do we learn from the small amount of high-quality data or the potentially large amount of weakly-labeled data? We argue that if the learner could somehow know and take the label-quality into account when learning the data representation, we could get the best of both worlds. To this end, we propose "fidelity-weighted learning" (FWL), a semi-supervised student-teacher approach for training deep neural networks using weakly-labeled data. FWL modulates the parameter updates to a student network (trained on the task we care about) on a per-sample basis according to the posterior confidence of its label-quality estimated by a teacher (who has access to the high-quality labels). Both student and teacher are learned from the data. We evaluate FWL on two tasks in information retrieval and natural language processing where we outperform state-of-the-art alternative semi-supervised methods, indicating that our approach makes better use of strong and weak labels, and leads to better task-dependent data representations.Comment: Published as a conference paper at ICLR 201

Mitigating Gender Bias in Machine Learning Data Sets

Artificial Intelligence has the capacity to amplify and perpetuate societal biases and presents profound ethical implications for society. Gender bias has been identified in the context of employment advertising and recruitment tools, due to their reliance on underlying language processing and recommendation algorithms. Attempts to address such issues have involved testing learned associations, integrating concepts of fairness to machine learning and performing more rigorous analysis of training data. Mitigating bias when algorithms are trained on textual data is particularly challenging given the complex way gender ideology is embedded in language. This paper proposes a framework for the identification of gender bias in training data for machine learning.The work draws upon gender theory and sociolinguistics to systematically indicate levels of bias in textual training data and associated neural word embedding models, thus highlighting pathways for both removing bias from training data and critically assessing its impact.Comment: 10 pages, 5 figures, 5 Tables, Presented as Bias2020 workshop (as part of the ECIR Conference) - http://bias.disim.univaq.i

The impact of artificial intelligence in education from teachers’ perspective: A case study for primary and secondary schools

The rapid development of disruptive technologies such as Artificial Intelligence (AI) is offering exciting opportunities for the field of education. Under the premise that AI can help bridge gaps in learning and teaching and create new opportunities for schools, a growing number of practitioners and researchers have sought to understand the potential of these technologies in educational settings. But these innovations also come at a price, and it is critical to investigate whether the benefits of AI in education (AIEd) effectively outweigh its risks. In this regard, this study aims to analyze the impact of Artificial Intelligence in education, identifying the factors that contribute to the possibility of implementing AI in Portuguese primary and secondary schools. Using a mixed research approach and collecting data from a survey answered by 184 Portuguese teachers, this research tested the effects of three factors in particular: respondents’ (1) knowledge and perceptions about the (2) benefits and (3) barriers of AI in education. The results indicated that the perceived benefits of AI in education strongly affect the intention to implement these technologies in the classroom, as opposed to the barriers. Also, knowledge of AIEd proved to be a significant factor, although teachers were more familiar with the potential of AI in theory than in practice. This situation, however, does not seem to compromise the intention to implement AI in Portuguese schools – on the contrary, it is reflected in a tendentially positive attitude towards this phenomenon.O rápido desenvolvimento de tecnologias disruptivas como a Inteligência Artificial (IA) está a trazer oportunidades empolgantes para o campo da educação. Sob a premissa de que a IA pode ajudar a colmatar lacunas na aprendizagem e no ensino e criar novas oportunidades para as escolas, um número crescente de profissionais e investigadores tem procurado compreender o potencial destas tecnologias em contextos educativos. Mas estas inovações vêm também com um preço associado, sendo por isso fundamental investigar se os benefícios da IA na educação (IAEd) compensam efetivamente os seus riscos. Neste sentido, o objetivo deste estudo é analisar o impacto da Inteligência Artificial na educação, identificando os fatores que contribuem para a possibilidade de implementar a IA nas escolas primárias e secundárias portuguesas. Utilizando uma abordagem de investigação mista e recolhendo dados de um inquérito respondido por 184 professores portugueses, esta investigação testou os efeitos de três fatores em particular: (1) os conhecimentos e as perceções dos inquiridos relativamente aos (2) benefícios e (3) às barreiras da IA na educação. Os resultados indicaram que os benefícios percebidos sobre a IAEd impactam significativamente a vontade de implementar estas tecnologias em sala de aula, ao contrário das barreiras. Também o conhecimento sobre AIEd demonstrou ter influência, apesar dos professores estarem mais familiarizados com o potencial da IA na teoria do que na prática. Esta situação, contudo, não parece comprometer a intenção de implementar a IA nas escolas portuguesas – pelo contrário, reflete-se numa atitude tendencialmente positiva em relação a este fenómeno