1,199 research outputs found
Efficient Fault Tolerance for Pipelined Query Engines via Write-ahead Lineage
Modern distributed pipelined query engines either do not support intra-query
fault tolerance or employ high-overhead approaches such as persisting
intermediate outputs or checkpointing state. In this work, we present
write-ahead lineage, a novel fault recovery technique that combines Spark's
lineage-based replay and write-ahead logging. Unlike Spark, where the lineage
is determined before query execution, write-ahead lineage persistently logs
lineage at runtime to support dynamic task dependencies in pipelined query
engines. Since only KB-sized lineages are persisted instead of MB-sized
intermediate outputs, the normal execution overhead is minimal compared to
spooling or checkpointing based approaches. To ensure fast fault recovery
times, tasks only consume intermediate outputs with persisted lineage,
preventing global rollbacks upon failure. In addition, lost tasks from
different stages can be recovered in a pipelined parallel manner. We implement
write-ahead lineage in a distributed pipelined query engine called Quokka. We
show that Quokka is around 2x faster than SparkSQL on the TPC-H benchmark with
similar fault recovery performance.Comment: ICDE 2024 (copyright IEEE
SATR-DL: Improving Surgical Skill Assessment and Task Recognition in Robot-assisted Surgery with Deep Neural Networks
Purpose: This paper focuses on an automated analysis of surgical motion
profiles for objective skill assessment and task recognition in robot-assisted
surgery. Existing techniques heavily rely on conventional statistic measures or
shallow modelings based on hand-engineered features and gesture segmentation.
Such developments require significant expert knowledge, are prone to errors,
and are less efficient in online adaptive training systems. Methods: In this
work, we present an efficient analytic framework with a parallel deep learning
architecture, SATR-DL, to assess trainee expertise and recognize surgical
training activity. Through an end-to-end learning technique, abstract
information of spatial representations and temporal dynamics is jointly
obtained directly from raw motion sequences. Results: By leveraging a shared
high-level representation learning, the resulting model is successful in the
recognition of trainee skills and surgical tasks, suturing, needle-passing, and
knot-tying. Meanwhile, we explore the use of ensemble in classification at the
trial level, where the SATR-DL outperforms state-of-the-art performance by
achieving accuracies of 0.960 and 1.000 in skill assessment and task
recognition, respectively. Conclusion: This study highlights the potential of
SATR-DL to provide improvements for an efficient data-driven assessment in
intelligent robotic surgery
DPA Load Balancer: Load balancing for Data Parallel Actor-based systems
In this project we explore ways to dynamically load balance actors in a
streaming framework. This is used to address input data skew that might lead to
stragglers. We continuously monitor actors' input queue lengths for load, and
redistribute inputs among reducers using consistent hashing if we detect
stragglers. To ensure consistent processing post-redistribution, we adopt an
approach that uses input forwarding combined with a state merge step at the end
of the processing. We show that this approach can greatly alleviate stragglers
for skewed data.Comment: 7 page
- …