56,437 research outputs found
Task-Aware Asynchronous Multi-Task Model with Class Incremental Contrastive Learning for Surgical Scene Understanding
Purpose: Surgery scene understanding with tool-tissue interaction recognition
and automatic report generation can play an important role in intra-operative
guidance, decision-making and postoperative analysis in robotic surgery.
However, domain shifts between different surgeries with inter and intra-patient
variation and novel instruments' appearance degrade the performance of model
prediction. Moreover, it requires output from multiple models, which can be
computationally expensive and affect real-time performance.
Methodology: A multi-task learning (MTL) model is proposed for surgical
report generation and tool-tissue interaction prediction that deals with domain
shift problems. The model forms of shared feature extractor, mesh-transformer
branch for captioning and graph attention branch for tool-tissue interaction
prediction. The shared feature extractor employs class incremental contrastive
learning (CICL) to tackle intensity shift and novel class appearance in the
target domain. We design Laplacian of Gaussian (LoG) based curriculum learning
into both shared and task-specific branches to enhance model learning. We
incorporate a task-aware asynchronous MTL optimization technique to fine-tune
the shared weights and converge both tasks optimally.
Results: The proposed MTL model trained using task-aware optimization and
fine-tuning techniques reported a balanced performance (BLEU score of 0.4049
for scene captioning and accuracy of 0.3508 for interaction detection) for both
tasks on the target domain and performed on-par with single-task models in
domain adaptation.
Conclusion: The proposed multi-task model was able to adapt to domain shifts,
incorporate novel instruments in the target domain, and perform tool-tissue
interaction detection and report generation on par with single-task models.Comment: Manuscript accepted in the International Journal of Computer Assisted
Radiology and Surgery. codes available:
https://github.com/lalithjets/Domain-adaptation-in-MT
Integrating DGSs and GATPs in an Adaptative and Collaborative Blended-Learning Web-Environment
The area of geometry with its very strong and appealing visual contents and
its also strong and appealing connection between the visual content and its
formal specification, is an area where computational tools can enhance, in a
significant way, the learning environments.
The dynamic geometry software systems (DGSs) can be used to explore the
visual contents of geometry. This already mature tools allows an easy
construction of geometric figures build from free objects and elementary
constructions. The geometric automated theorem provers (GATPs) allows formal
deductive reasoning about geometric constructions, extending the reasoning via
concrete instances in a given model to formal deductive reasoning in a
geometric theory.
An adaptative and collaborative blended-learning environment where the DGS
and GATP features could be fully explored would be, in our opinion a very rich
and challenging learning environment for teachers and students.
In this text we will describe the Web Geometry Laboratory a Web environment
incorporating a DGS and a repository of geometric problems, that can be used in
a synchronous and asynchronous fashion and with some adaptative and
collaborative features.
As future work we want to enhance the adaptative and collaborative aspects of
the environment and also to incorporate a GATP, constructing a dynamic and
individualised learning environment for geometry.Comment: In Proceedings THedu'11, arXiv:1202.453
Embedding Multi-Task Address-Event- Representation Computation
Address-Event-Representation, AER, is a communication protocol that is
intended to transfer neuronal spikes between bioinspired chips. There are
several AER tools to help to develop and test AER based systems, which may
consist of a hierarchical structure with several chips that transmit spikes
among them in real-time, while performing some processing. Although these
tools reach very high bandwidth at the AER communication level, they require
the use of a personal computer to allow the higher level processing of the
event information. We propose the use of an embedded platform based on a
multi-task operating system to allow both, the AER communication and
processing without the requirement of either a laptop or a computer. In this
paper, we present and study the performance of an embedded multi-task AER
tool, connecting and programming it for processing Address-Event
information from a spiking generator.Ministerio de Ciencia e Innovación TEC2006-11730-C03-0
Reinforcement Learning for Mobile Robot Collision Avoidance in Navigation Tasks
Collision avoidance is fundamental for mobile robot navigation. In general, its solutions include: {\it map-based} and {\it mapless approaches.} In the map-based approach, robots pre-plan collision-free paths based on an environment map and follow their paths during navigation. On the other hand, the mapless approach requires robots to avoid collisions without referencing to an environment map. This thesis first studies the map-based approach for multiple robots to collectively build environment maps. In this study, a robot following a pre-planned path may encounter unexpected obstacles, such as other moving robots and obstacles inaccurately presented on an environment map. This motivates us to study mapless collision avoidance in the second part of the thesis. Mapless collision avoidance requires a robot to infer an optimal action based on sensor data and operate in real time. Inferring an optimal action in a timely manner is computationally expensive, particularly when a robot has limited on-board computing resources. To avoid the expensive online action inferring, this thesis presents a reinforcement learning approach which learns policies for mapless collision avoidance under real-world settings. We first propose a Real-Time Actor-Critic Architecture (RTAC) to support asynchronous reinforcement learning under real-time constraint. Based on RTAC, we propose asynchronous reinforcement learning methods for mapless collision avoidance of various numbers of robots under different environment configurations. Through extensive experiments, we demonstrate that RTAC serves as a solid foundation to support multi-task and multi-agent learning for mapless collision avoidance under asynchronous settings
- …