Adapting Visual Question Answering Models for Enhancing Multimodal Community Q&A Platforms

Question categorization and expert retrieval methods have been crucial for information organization and accessibility in community question & answering (CQA) platforms. Research in this area, however, has dealt with only the text modality. With the increasing multimodal nature of web content, we focus on extending these methods for CQA questions accompanied by images. Specifically, we leverage the success of representation learning for text and images in the visual question answering (VQA) domain, and adapt the underlying concept and architecture for automated category classification and expert retrieval on image-based questions posted on Yahoo! Chiebukuro, the Japanese counterpart of Yahoo! Answers. To the best of our knowledge, this is the first work to tackle the multimodality challenge in CQA, and to adapt VQA models for tasks on a more ecologically valid source of visual questions. Our analysis of the differences between visual QA and community QA data drives our proposal of novel augmentations of an attention method tailored for CQA, and use of auxiliary tasks for learning better grounding features. Our final model markedly outperforms the text-only and VQA model baselines for both tasks of classification and expert retrieval on real-world multimodal CQA data.Comment: Submitted for review at CIKM 201

Knowledge will Propel Machine Understanding of Content: Extrapolating from Current Examples

Machine Learning has been a big success story during the AI resurgence. One particular stand out success relates to learning from a massive amount of data. In spite of early assertions of the unreasonable effectiveness of data, there is increasing recognition for utilizing knowledge whenever it is available or can be created purposefully. In this paper, we discuss the indispensable role of knowledge for deeper understanding of content where (i) large amounts of training data are unavailable, (ii) the objects to be recognized are complex, (e.g., implicit entities and highly subjective content), and (iii) applications need to use complementary or related data in multiple modalities/media. What brings us to the cusp of rapid progress is our ability to (a) create relevant and reliable knowledge and (b) carefully exploit knowledge to enhance ML/NLP techniques. Using diverse examples, we seek to foretell unprecedented progress in our ability for deeper understanding and exploitation of multimodal data and continued incorporation of knowledge in learning techniques.Comment: Pre-print of the paper accepted at 2017 IEEE/WIC/ACM International Conference on Web Intelligence (WI). arXiv admin note: substantial text overlap with arXiv:1610.0770

Threshold concepts: Impacts on teaching and learning at tertiary level

This project explored teaching and learning of hard-to-learn threshold concepts in first-year English, an electrical engineering course, leadership courses, and in doctoral writing. The project was envisioned to produce disciplinary case studies that lecturers could use to reflect on and refine their curriculum and pedagogy, thereby contributing to discussion about the relationship between theory and methodology in higher education research (Shay, Ashwin, & Case, 2009). A team of seven academics investigated lecturers’ awareness and emergent knowledge of threshold concepts and associated pedagogies and how such pedagogies can afford opportunities for learning. As part of this examination the lecturers also explored the role of threshold concept theory in designing curricula and sought to find the commonalities in threshold concepts and their teaching and learning across the four disciplines. The research highlights new ways of teaching threshold concepts to help students learn concepts that are fundamental to the disciplines they are studying and expand their educational experiences. Given that much of the international research in this field focuses on the identification of threshold concepts and debates their characteristics (Barradell, 2013; Flanagan, 2014; Knight, Callaghan, Baldock, & Meyer, 2013), our exploration of what happens when lecturers use threshold concept theory to re-envision their curriculum and teaching helps to address a gap within the field. By addressing an important theoretical and practical approach the project makes a considerable contribution to teaching and learning at the tertiary level in general and to each discipline in particular