Adapting Visual Question Answering Models for Enhancing Multimodal Community Q&A Platforms

Question categorization and expert retrieval methods have been crucial for information organization and accessibility in community question & answering (CQA) platforms. Research in this area, however, has dealt with only the text modality. With the increasing multimodal nature of web content, we focus on extending these methods for CQA questions accompanied by images. Specifically, we leverage the success of representation learning for text and images in the visual question answering (VQA) domain, and adapt the underlying concept and architecture for automated category classification and expert retrieval on image-based questions posted on Yahoo! Chiebukuro, the Japanese counterpart of Yahoo! Answers. To the best of our knowledge, this is the first work to tackle the multimodality challenge in CQA, and to adapt VQA models for tasks on a more ecologically valid source of visual questions. Our analysis of the differences between visual QA and community QA data drives our proposal of novel augmentations of an attention method tailored for CQA, and use of auxiliary tasks for learning better grounding features. Our final model markedly outperforms the text-only and VQA model baselines for both tasks of classification and expert retrieval on real-world multimodal CQA data.Comment: Submitted for review at CIKM 201

Toward a Neural Semantic Parsing System for EHR Question Answering

Clinical semantic parsing (SP) is an important step toward identifying the exact information need (as a machine-understandable logical form) from a natural language query aimed at retrieving information from electronic health records (EHRs). Current approaches to clinical SP are largely based on traditional machine learning and require hand-building a lexicon. The recent advancements in neural SP show a promise for building a robust and flexible semantic parser without much human effort. Thus, in this paper, we aim to systematically assess the performance of two such neural SP models for EHR question answering (QA). We found that the performance of these advanced neural models on two clinical SP datasets is promising given their ease of application and generalizability. Our error analysis surfaces the common types of errors made by these models and has the potential to inform future research into improving the performance of neural SP models for EHR QA.Comment: Accepted at the AMIA Annual Symposium 2022 (10 pages, 5 tables, 1 figure

A Frame-Based NLP System for Cancer-Related Information Extraction.

We propose a frame-based natural language processing (NLP) method that extracts cancer-related information from clinical narratives. We focus on three frames: cancer diagnosis, cancer therapeutic procedure, and tumor description. We utilize a deep learning-based approach, bidirectional Long Short-term Memory (LSTM) Conditional Random Field (CRF), which uses both character and word embeddings. The system consists of two constituent sequence classifiers: a frame identification (lexical unit) classifier and a frame element classifier. The classifier achieves an

Introducing an ePortfolio into Practicum-Based Units: Pre-service Teachers’ Perceptions of Effective Support

ePortfolios are gaining momentum as a preferred way for graduates to demonstrate current and developing capabilities against industry standards. Effective training is essential for new graduates to produce quality and competitive ePortfolios. This research focused on the perspective of pre-service teachers on the effectiveness of learning opportunities provided to increase confidence and skills in developing an ePortfolio in an Australian four-year undergraduate degree. The initial phase of this research employed a survey to examine the perspective of 132 second-year and 105 third-year pre-service teachers. Results indicated that for the second-year cohort there was a minimal increase in the levels of confidence across all areas. In contrast, the third-year pre-service teachers showed some increase in confidence in developing an ePortfolio and understanding its purpose. While the findings from this study emphasised the pre-service teachers’ need for ongoing hands-on support, it also highlighted their reluctance to seek support at an independent level

Deep Patient Representation of Clinical Notes via Multi-Task Learning for Mortality Prediction.

We propose a deep learning-based multi-task learning (MTL) architecture focusing on patient mortality predictions from clinical notes. The MTL framework enables the model to learn a patient representation that generalizes to a variety of clinical prediction tasks. Moreover, we demonstrate how MTL enables small but consistent gains on a single classification task (e.g., in-hospital mortality prediction) simply by incorporating related tasks (e.g., 30-day and 1-year mortality prediction) into the MTL framework. To accomplish this, we utilize a multi-level Convolutional Neural Network (CNN) associated with a MTL loss component. The model is evaluated with 3, 5, and 20 tasks and is consistently able to produce a higher-performing model than a single-task learning (STL) classifier. We further discuss the effect of the multi-task model on other clinical outcomes of interest, including being able to produce high-quality representations that can be utilized to great effect by simpler models. Overall, this study demonstrates the efficiency and generalizability of MTL across tasks that STL fails to leverage

Testing of linear models for optimal control of second-order dynamical system based on model-reality differences

In this paper, the testing of linear models with different parameter values is conducted for solving the optimal control problem of a second-order dynamical system. The purpose of this testing is to provide the solution with the same structure but different parameter values in the model used. For doing so, the adjusted parameters are added to each model in order to measure the differences between the model used and the plant dynamics. On this basis, an expanded optimal control problem, which combines system optimization and parameter estimation, is introduced. Then, the Hamiltonian function is defined and a set of the necessary conditions is derived. Consequently, a modified model-based optimal control problem has resulted. Follow from this, an equivalent optimization problem without constraints is formulated. During the calculation procedure, the conjugate gradient algorithm is employed to solve the optimization problem, in turn, to update the adjusted parameters repeatedly for obtaining the optimal solution of the model used. Within a given tolerance, the iterative solution of the model used approximates the correct optimal solution of the original linear optimal control problem despite model-reality differences. The results obtained show the applicability of models with the same structures and different parameter values for solving the original linear optimal control problem. In conclusion, the efficiency of the approach proposed is highly verified

A Linearised Hybrid FE-SEA Method for Nonlinear Dynamic Systems Excited by Random and Harmonic Loadings

The present paper proposes a linearised hybrid finite element-statistical energy analysis (FE-SEA) formulation for built-up systems with nonlinear joints and excited by random, as well as harmonic, loadings. The new formulation was validated via an ad-hoc developed stochastic benchmark model. The latter was derived through the combination of the Lagrange-Rayleigh-Ritz method (LRRM) and the Monte Carlo simulation (MCS). Within the build-up plate systems, each plate component was modelled by using the classical Kirchhoff’s thin-plate theory. The linearisation processes were carried out according to the loading-type. In the case of random loading, the statistical linearisation (SL) was employed, while, in the case of harmonic loading, the method of harmonic balance (MHB) was used. To demonstrate the effectiveness of the proposed hybrid FE-SEA formulation, three different case studies, made-up of built-up systems with localized cubic nonlinearities, were considered. Both translational and torsional springs, as joint components, were employed. Four different types of loadings were taken into account: harmonic/random point and distributed loadings. The response of the dynamic systems was investigated in terms of ensemble average of the time-averaged energy.</jats:p