Interrogating the Explanatory Power of Attention in Neural Machine Translation

Attention models have become a crucial component in neural machine translation (NMT). They are often implicitly or explicitly used to justify the model's decision in generating a specific token but it has not yet been rigorously established to what extent attention is a reliable source of information in NMT. To evaluate the explanatory power of attention for NMT, we examine the possibility of yielding the same prediction but with counterfactual attention models that modify crucial aspects of the trained attention model. Using these counterfactual attention mechanisms we assess the extent to which they still preserve the generation of function and content words in the translation process. Compared to a state of the art attention model, our counterfactual attention models produce 68% of function words and 21% of content words in our German-English dataset. Our experiments demonstrate that attention models by themselves cannot reliably explain the decisions made by a NMT model.Comment: Accepted at the 3rd Workshop on Neural Generation and Translation (WNGT 2019) held at EMNLP-IJCNLP 2019 (Camera ready

Recipes for calibration and validation of agent-based models in cancer biomedicine

Computational models and simulations are not just appealing because of their intrinsic characteristics across spatiotemporal scales, scalability, and predictive power, but also because the set of problems in cancer biomedicine that can be addressed computationally exceeds the set of those amenable to analytical solutions. Agent-based models and simulations are especially interesting candidates among computational modelling strategies in cancer research due to their capabilities to replicate realistic local and global interaction dynamics at a convenient and relevant scale. Yet, the absence of methods to validate the consistency of the results across scales can hinder adoption by turning fine-tuned models into black boxes. This review compiles relevant literature to explore strategies to leverage high-fidelity simulations of multi-scale, or multi-level, cancer models with a focus on validation approached as simulation calibration. We argue that simulation calibration goes beyond parameter optimization by embedding informative priors to generate plausible parameter configurations across multiple dimensions

Advanced optical imaging for the rational design of nanomedicines

Despite the enormous potential of nanomedicines to shape the future of medicine, their clinical translation remains suboptimal. Translational challenges are present in every step of the development pipeline, from a lack of understanding of patient heterogeneity to insufficient insights on nanoparticle properties and their impact on material-cell interactions. Here, we discuss how the adoption of advanced optical microscopy techniques, such as super-resolution optical microscopies, correlative techniques, and high-content modalities, could aid the rational design of nanocarriers, by characterizing the cell, the nanomaterial, and their interaction with unprecedented spatial and/or temporal detail. In this nanomedicine arena, we will discuss how the implementation of these techniques, with their versatility and specificity, can yield high volumes of multi-parametric data; and how machine learning can aid the rapid advances in microscopy: from image acquisition to data interpretation.</p

Advanced optical imaging for the rational design of nanomedicines

Despite the enormous potential of nanomedicines to shape the future of medicine, their clinical translation remains suboptimal. Translational challenges are present in every step of the development pipeline, from a lack of understanding of patient heterogeneity to insufficient insights on nanoparticle properties and their impact on material-cell interactions. Here, we discuss how the adoption of advanced optical microscopy techniques, such as super-resolution optical microscopies, correlative techniques, and high-content modalities, could aid the rational design of nanocarriers, by characterizing the cell, the nanomaterial, and their interaction with unprecedented spatial and/or temporal detail. In this nanomedicine arena, we will discuss how the implementation of these techniques, with their versatility and specificity, can yield high volumes of multi-parametric data; and how machine learning can aid the rapid advances in microscopy: from image acquisition to data interpretation.</p

Brain data:Scanning, scraping and sculpting the plastic learning brain through neurotechnology

Neurotechnology is an advancing field of research and development with significant implications for education. As 'postdigital' hybrids of biological and informational codes, novel neurotechnologies combine neuroscience insights into the human brain with advanced technical development in brain imaging, brain-computer interfaces, neurofeedback platforms, brain stimulation and other neuroenhancement applications. Merging neurobiological knowledge about human life with computational technologies, neurotechnology exemplifies how postdigital science will play a significant role in societies and education in decades to come. As neurotechnology developments are being extended to education, they present potential for businesses and governments to enact new techniques of 'neurogovernance' by 'scanning' the brain, 'scraping' it for data and then 'sculpting' the brain toward particular capacities. The aim of this article is to critically review neurotechnology developments and implications for education. It examines the purposes to which neurotechnology development is being put in education, interrogating the commercial and governmental objectives associated with it and the neuroscientific concepts and expertise that underpin it. Finally, the article raises significant ethical and governance issues related to neurotechnology development and postdigital science that require concerted attention from education researchers

Analysis of explainable artificial intelligence on time series data

In recent years, the interest in Artificial Intelligence (AI) has experienced a significant growth, which has contributed to the emergence of new research directions such as Explainable Artificial Intelligence (XAI). The ability to apply AI approaches to solve various problems in many industrial areas has been mainly achieved by increasing model complexity and the use of various black-box models that lack transparency. In particular, deep neural networks are great at dealing with problems that are too difficult for classic machine learning methods, but it is often a big challenge to answer the question why a neural network made such a decision and not another. The answer to this question is extremely important to ensure that ML models are reliable and can be held liable for the decision-making process. Over a relatively short period of time a plethora of methods to tackle this problem have been proposed, but mainly in the area of computer vision and natural language processing. Few publications have been published so far in the context of explainability in time series. This Thesis aims to provide a comprehensive literature review of the research in XAI for time series data as well as to achieve and evaluate local explainability for a model in time series forecasting problem. The solution involved framing a time series forecasting task as a Remaining Useful Life (RUL) prognosis for turbofan engines. We trained two Bi-LSTM models, with and without attention layer, on the C-MAPSS data set. The local explainability was achieved using two post-hoc explainability techniques SHAP and LIME as well as extracting and interpreting the attention weights. The results of explanations were compared and evaluated. We applied the evaluation metric which incorporates the temporal dimension of the data. The obtained results indicate that LIME technique outperforms other methods in terms of the fidelity of local explanations. Moreover, we demonstrated the potential of attention mechanisms to make a deep learning model for time series forecasting task more interpretable. The approach presented in this work can be easily applied to any time series forecasting or classification scenario in which we want to achieve model interpretability and evaluation of generated explanations