3,183 research outputs found
Adapting Visual Question Answering Models for Enhancing Multimodal Community Q&A Platforms
Question categorization and expert retrieval methods have been crucial for
information organization and accessibility in community question & answering
(CQA) platforms. Research in this area, however, has dealt with only the text
modality. With the increasing multimodal nature of web content, we focus on
extending these methods for CQA questions accompanied by images. Specifically,
we leverage the success of representation learning for text and images in the
visual question answering (VQA) domain, and adapt the underlying concept and
architecture for automated category classification and expert retrieval on
image-based questions posted on Yahoo! Chiebukuro, the Japanese counterpart of
Yahoo! Answers.
To the best of our knowledge, this is the first work to tackle the
multimodality challenge in CQA, and to adapt VQA models for tasks on a more
ecologically valid source of visual questions. Our analysis of the differences
between visual QA and community QA data drives our proposal of novel
augmentations of an attention method tailored for CQA, and use of auxiliary
tasks for learning better grounding features. Our final model markedly
outperforms the text-only and VQA model baselines for both tasks of
classification and expert retrieval on real-world multimodal CQA data.Comment: Submitted for review at CIKM 201
Social Search: retrieving information in Online Social Platforms -- A Survey
Social Search research deals with studying methodologies exploiting social
information to better satisfy user information needs in Online Social Media
while simplifying the search effort and consequently reducing the time spent
and the computational resources utilized. Starting from previous studies, in
this work, we analyze the current state of the art of the Social Search area,
proposing a new taxonomy and highlighting current limitations and open research
directions. We divide the Social Search area into three subcategories, where
the social aspect plays a pivotal role: Social Question&Answering, Social
Content Search, and Social Collaborative Search. For each subcategory, we
present the key concepts and selected representative approaches in the
literature in greater detail. We found that, up to now, a large body of studies
model users' preferences and their relations by simply combining social
features made available by social platforms. It paves the way for significant
research to exploit more structured information about users' social profiles
and behaviors (as they can be inferred from data available on social platforms)
to optimize their information needs further
Foundational Models in Medical Imaging: A Comprehensive Survey and Future Vision
Foundation models, large-scale, pre-trained deep-learning models adapted to a
wide range of downstream tasks have gained significant interest lately in
various deep-learning problems undergoing a paradigm shift with the rise of
these models. Trained on large-scale dataset to bridge the gap between
different modalities, foundation models facilitate contextual reasoning,
generalization, and prompt capabilities at test time. The predictions of these
models can be adjusted for new tasks by augmenting the model input with
task-specific hints called prompts without requiring extensive labeled data and
retraining. Capitalizing on the advances in computer vision, medical imaging
has also marked a growing interest in these models. To assist researchers in
navigating this direction, this survey intends to provide a comprehensive
overview of foundation models in the domain of medical imaging. Specifically,
we initiate our exploration by providing an exposition of the fundamental
concepts forming the basis of foundation models. Subsequently, we offer a
methodical taxonomy of foundation models within the medical domain, proposing a
classification system primarily structured around training strategies, while
also incorporating additional facets such as application domains, imaging
modalities, specific organs of interest, and the algorithms integral to these
models. Furthermore, we emphasize the practical use case of some selected
approaches and then discuss the opportunities, applications, and future
directions of these large-scale pre-trained models, for analyzing medical
images. In the same vein, we address the prevailing challenges and research
pathways associated with foundational models in medical imaging. These
encompass the areas of interpretability, data management, computational
requirements, and the nuanced issue of contextual comprehension.Comment: The paper is currently in the process of being prepared for
submission to MI
When Brain-inspired AI Meets AGI
Artificial General Intelligence (AGI) has been a long-standing goal of
humanity, with the aim of creating machines capable of performing any
intellectual task that humans can do. To achieve this, AGI researchers draw
inspiration from the human brain and seek to replicate its principles in
intelligent machines. Brain-inspired artificial intelligence is a field that
has emerged from this endeavor, combining insights from neuroscience,
psychology, and computer science to develop more efficient and powerful AI
systems. In this article, we provide a comprehensive overview of brain-inspired
AI from the perspective of AGI. We begin with the current progress in
brain-inspired AI and its extensive connection with AGI. We then cover the
important characteristics for both human intelligence and AGI (e.g., scaling,
multimodality, and reasoning). We discuss important technologies toward
achieving AGI in current AI systems, such as in-context learning and prompt
tuning. We also investigate the evolution of AGI systems from both algorithmic
and infrastructural perspectives. Finally, we explore the limitations and
future of AGI
Artificial Intelligence-Enabled Intelligent Assistant for Personalized and Adaptive Learning in Higher Education
This paper presents a novel framework, Artificial Intelligence-Enabled
Intelligent Assistant (AIIA), for personalized and adaptive learning in higher
education. The AIIA system leverages advanced AI and Natural Language
Processing (NLP) techniques to create an interactive and engaging learning
platform. This platform is engineered to reduce cognitive load on learners by
providing easy access to information, facilitating knowledge assessment, and
delivering personalized learning support tailored to individual needs and
learning styles. The AIIA's capabilities include understanding and responding
to student inquiries, generating quizzes and flashcards, and offering
personalized learning pathways. The research findings have the potential to
significantly impact the design, implementation, and evaluation of AI-enabled
Virtual Teaching Assistants (VTAs) in higher education, informing the
development of innovative educational tools that can enhance student learning
outcomes, engagement, and satisfaction. The paper presents the methodology,
system architecture, intelligent services, and integration with Learning
Management Systems (LMSs) while discussing the challenges, limitations, and
future directions for the development of AI-enabled intelligent assistants in
education.Comment: 29 pages, 10 figures, 9659 word
Survey of the State of the Art in Natural Language Generation: Core tasks, applications and evaluation
This paper surveys the current state of the art in Natural Language
Generation (NLG), defined as the task of generating text or speech from
non-linguistic input. A survey of NLG is timely in view of the changes that the
field has undergone over the past decade or so, especially in relation to new
(usually data-driven) methods, as well as new applications of NLG technology.
This survey therefore aims to (a) give an up-to-date synthesis of research on
the core tasks in NLG and the architectures adopted in which such tasks are
organised; (b) highlight a number of relatively recent research topics that
have arisen partly as a result of growing synergies between NLG and other areas
of artificial intelligence; (c) draw attention to the challenges in NLG
evaluation, relating them to similar challenges faced in other areas of Natural
Language Processing, with an emphasis on different evaluation methods and the
relationships between them.Comment: Published in Journal of AI Research (JAIR), volume 61, pp 75-170. 118
pages, 8 figures, 1 tabl
Instruction Tuning for Large Language Models: A Survey
This paper surveys research works in the quickly advancing field of
instruction tuning (IT), a crucial technique to enhance the capabilities and
controllability of large language models (LLMs). Instruction tuning refers to
the process of further training LLMs on a dataset consisting of
\textsc{(instruction, output)} pairs in a supervised fashion, which bridges the
gap between the next-word prediction objective of LLMs and the users' objective
of having LLMs adhere to human instructions. In this work, we make a systematic
review of the literature, including the general methodology of IT, the
construction of IT datasets, the training of IT models, and applications to
different modalities, domains and applications, along with an analysis on
aspects that influence the outcome of IT (e.g., generation of instruction
outputs, size of the instruction dataset, etc). We also review the potential
pitfalls of IT along with criticism against it, along with efforts pointing out
current deficiencies of existing strategies and suggest some avenues for
fruitful research.Comment: A Survey paper, Pre-prin
On Data-driven systems analyzing, supporting and enhancing usersâ interaction and experience
Tesis doctoral en inglés y resumen extendido en español[EN] The research areas of Human-Computer Interaction and Software Architectures have been traditionally treated separately, but in the literature, many authors made efforts to merge them to build better software systems. One of the common gaps between software engineering and usability is the lack of strategies to apply usability principles in the initial design of software architectures. Including these principles since the early phases of software design would help to avoid later architectural changes to include user experience requirements. The combination of both fields (software architectures and Human-Computer Interaction) would contribute to building better interactive software that should include the best from both the systems and user-centered designs. In that combination, the software architectures should enclose the fundamental structure and ideas of the system to offer the desired quality based on sound design decisions.
Moreover, the information kept within a system is an opportunity to extract knowledge about the system itself, its components, the software included, the users or the interaction occurring inside. The knowledge gained from the information generated in a software environment can be used to improve the system itself, its software, the usersâ experience, and the results. So, the combination of the areas of Knowledge Discovery and Human-Computer Interaction offers ideal conditions to address Human-Computer-Interaction-related challenges. The Human-Computer Interaction focuses on human intelligence, the Knowledge Discovery in computational intelligence, and the combination of both can raise the support of human intelligence with machine intelligence to discover new insights in a world crowded of data.
This Ph.D. Thesis deals with these kinds of challenges: how approaches like data-driven software architectures (using Knowledge Discovery techniques) can help to improve the users' interaction and experience within an interactive system. Specifically, it deals with how to improve the human-computer interaction processes of different kind of stakeholders to improve different aspects such as the user experience or the easiness to accomplish a specific task.
Several research actions and experiments support this investigation. These research actions included performing a systematic literature review and mapping of the literature that was aimed at finding how the software architectures in the literature have been used to support, analyze or enhance the human-computer interaction. Also, the actions included work on four different research scenarios that presented common challenges in the Human-Computer Interaction knowledge area. The case studies that fit into the scenarios selected were chosen based on the Human-Computer Interaction challenges they present, and on the authorsâ accessibility to them. The four case studies were: an educational laboratory virtual world, a Massive Open Online Course and the social networks where the students discuss and learn, a system that includes very large web forms, and an environment where programmers develop code in the context of quantum computing. The development of the experiences involved the review of more than 2700 papers (only in the literature review phase), the analysis of the interaction of 6000 users in four different contexts or the analysis of 500,000 quantum computing programs.
As outcomes from the experiences, some solutions are presented regarding the minimal software artifacts to include in software architectures, the behavior they should exhibit, the features desired in the extended software architecture, some analytic workflows and approaches to use, or the different kinds of feedback needed to reinforce the usersâ interaction and experience.
The results achieved led to the conclusion that, despite this is not a standard practice in the literature, the software environments should embrace Knowledge Discovery and data-driven principles to analyze and respond appropriately to the usersâ needs and improve or support the interaction. To adopt Knowledge Discovery and data-driven principles, the software environments need to extend their software architectures to cover also the challenges related to Human-Computer Interaction. Finally, to tackle the current challenges related to the usersâ interaction and experience and aiming to automate the software response to usersâ actions, desires, and behaviors, the interactive systems should also include intelligent behaviors through embracing the Artificial Intelligence procedures and techniques
On data-driven systems analyzing, supporting and enhancing usersâ interaction and experience
[EN]The research areas of Human-Computer Interaction and Software Architectures have
been traditionally treated separately, but in the literature, many authors made efforts to
merge them to build better software systems. One of the common gaps between software
engineering and usability is the lack of strategies to apply usability principles in the initial
design of software architectures. Including these principles since the early phases of software
design would help to avoid later architectural changes to include user experience
requirements. The combination of both fields (software architectures and Human-Computer
Interaction) would contribute to building better interactive software that should include the
best from both the systems and user-centered designs. In that combination, the software
architectures should enclose the fundamental structure and ideas of the system to offer the
desired quality based on sound design decisions.
Moreover, the information kept within a system is an opportunity to extract knowledge
about the system itself, its components, the software included, the users or the interaction
occurring inside. The knowledge gained from the information generated in a software
environment can be used to improve the system itself, its software, the usersâ experience, and
the results. So, the combination of the areas of Knowledge Discovery and Human-Computer
Interaction offers ideal conditions to address Human-Computer-Interaction-related
challenges. The Human-Computer Interaction focuses on human intelligence, the Knowledge
Discovery in computational intelligence, and the combination of both can raise the support
of human intelligence with machine intelligence to discover new insights in a world crowded
of data.
This Ph.D. Thesis deals with these kinds of challenges: how approaches like data-driven
software architectures (using Knowledge Discovery techniques) can help to improve the users'
interaction and experience within an interactive system. Specifically, it deals with how to
improve the human-computer interaction processes of different kind of stakeholders to
improve different aspects such as the user experience or the easiness to accomplish a specific
task.
Several research actions and experiments support this investigation. These research
actions included performing a systematic literature review and mapping of the literature that
was aimed at finding how the software architectures in the literature have been used to
support, analyze or enhance the human-computer interaction. Also, the actions included work
on four different research scenarios that presented common challenges in the Human-
Computer Interaction knowledge area. The case studies that fit into the scenarios selected
were chosen based on the Human-Computer Interaction challenges they present, and on the
authorsâ accessibility to them. The four case studies were: an educational laboratory virtual world, a Massive Open Online Course and the social networks where the students discuss
and learn, a system that includes very large web forms, and an environment where
programmers develop code in the context of quantum computing. The development of the
experiences involved the review of more than 2700 papers (only in the literature review
phase), the analysis of the interaction of 6000 users in four different contexts or the analysis
of 500,000 quantum computing programs.
As outcomes from the experiences, some solutions are presented regarding the minimal
software artifacts to include in software architectures, the behavior they should exhibit, the
features desired in the extended software architecture, some analytic workflows and
approaches to use, or the different kinds of feedback needed to reinforce the usersâ interaction
and experience.
The results achieved led to the conclusion that, despite this is not a standard practice in
the literature, the software environments should embrace Knowledge Discovery and datadriven
principles to analyze and respond appropriately to the usersâ needs and improve or
support the interaction. To adopt Knowledge Discovery and data-driven principles, the
software environments need to extend their software architectures to cover also the challenges
related to Human-Computer Interaction. Finally, to tackle the current challenges related to
the usersâ interaction and experience and aiming to automate the software response to usersâ
actions, desires, and behaviors, the interactive systems should also include intelligent
behaviors through embracing the Artificial Intelligence procedures and techniques
The use of tools of data mining to decision making in engineering educationâA systematic mapping study
In recent years, there has been an increasing amount of theoretical and applied research that has focused on educational data mining. The learning analytics is a discipline that uses techniques, methods, and algorithms that allow the user to discover and extract patterns in stored educational data, with the purpose of improving the teachingâlearning process. However, there are many requirements related to the use of new technologies in teachingâlearning processes that are practically unaddressed from the learning analytics. In an analysis of the literature, the existence of a systematic revision of the application of learning analytics in the field of engineering education is not evident. The study described in this article provides researchers with an overview of the progress made to date and identifies areas in which research is missing. To this end, a systematic mapping study has been carried out, oriented toward the classification of publications that focus on the type of research and the type of contribution. The results show a trend toward case study research that is mainly directed at software and computer science engineering. Furthermore, trends in the application of learning analytics are highlighted in the topics, such as student retention or dropout prediction, analysis of academic student data, student learning assessment and student behavior analysis. Although this systematic mapping study has focused on the application of learning analytics in engineering education, some of the results can also be applied to other educational areas
- âŠ