145 research outputs found
A transcriptional factor B paralog functions as an activator to DNA damage-responsive expression in archaea
Negative Frames Matter in Egocentric Visual Query 2D Localization
The recently released Ego4D dataset and benchmark significantly scales and
diversifies the first-person visual perception data. In Ego4D, the Visual
Queries 2D Localization task aims to retrieve objects appeared in the past from
the recording in the first-person view. This task requires a system to
spatially and temporally localize the most recent appearance of a given object
query, where query is registered by a single tight visual crop of the object in
a different scene.
Our study is based on the three-stage baseline introduced in the Episodic
Memory benchmark. The baseline solves the problem by detection and tracking:
detect the similar objects in all the frames, then run a tracker from the most
confident detection result. In the VQ2D challenge, we identified two
limitations of the current baseline. (1) The training configuration has
redundant computation. Although the training set has millions of instances,
most of them are repetitive and the number of unique object is only around
14.6k. The repeated gradient computation of the same object lead to an
inefficient training; (2) The false positive rate is high on background frames.
This is due to the distribution gap between training and evaluation. During
training, the model is only able to see the clean, stable, and labeled frames,
but the egocentric videos also have noisy, blurry, or unlabeled background
frames. To this end, we developed a more efficient and effective solution.
Concretely, we bring the training loop from ~15 days to less than 24 hours, and
we achieve 0.17% spatial-temporal AP, which is 31% higher than the baseline.
Our solution got the first ranking on the public leaderboard. Our code is
publicly available at https://github.com/facebookresearch/vq2d_cvpr.Comment: First place winning solution for VQ2D task in CVPR-2022 Ego4D
Challenge. Our code is publicly available at
https://github.com/facebookresearch/vq2d_cvp
Where is my Wallet? Modeling Object Proposal Sets for Egocentric Visual Query Localization
This paper deals with the problem of localizing objects in image and video
datasets from visual exemplars. In particular, we focus on the challenging
problem of egocentric visual query localization. We first identify grave
implicit biases in current query-conditioned model design and visual query
datasets. Then, we directly tackle such biases at both frame and object set
levels. Concretely, our method solves these issues by expanding limited
annotations and dynamically dropping object proposals during training.
Additionally, we propose a novel transformer-based module that allows for
object-proposal set context to be considered while incorporating query
information. We name our module Conditioned Contextual Transformer or
CocoFormer. Our experiments show the proposed adaptations improve egocentric
query detection, leading to a better visual query localization system in both
2D and 3D configurations. Thus, we are able to improve frame-level detection
performance from 26.28% to 31.26 in AP, which correspondingly improves the VQ2D
and VQ3D localization scores by significant margins. Our improved context-aware
query object detector ranked first and second in the VQ2D and VQ3D tasks in the
2nd Ego4D challenge. In addition to this, we showcase the relevance of our
proposed model in the Few-Shot Detection (FSD) task, where we also achieve SOTA
results. Our code is available at
https://github.com/facebookresearch/vq2d_cvpr.Comment: We ranked first and second in the VQ2D and VQ3D tasks in the 2nd
Ego4D challeng
Multi-Modal Few-Shot Temporal Action Detection via Vision-Language Meta-Adaptation
Few-shot (FS) and zero-shot (ZS) learning are two different approaches for
scaling temporal action detection (TAD) to new classes. The former adapts a
pretrained vision model to a new task represented by as few as a single video
per class, whilst the latter requires no training examples by exploiting a
semantic description of the new class. In this work, we introduce a new
multi-modality few-shot (MMFS) TAD problem, which can be considered as a
marriage of FS-TAD and ZS-TAD by leveraging few-shot support videos and new
class names jointly. To tackle this problem, we further introduce a novel
MUlti-modality PromPt mETa-learning (MUPPET) method. This is enabled by
efficiently bridging pretrained vision and language models whilst maximally
reusing already learned capacity. Concretely, we construct multi-modal prompts
by mapping support videos into the textual token space of a vision-language
model using a meta-learned adapter-equipped visual semantics tokenizer. To
tackle large intra-class variation, we further design a query feature
regulation scheme. Extensive experiments on ActivityNetv1.3 and THUMOS14
demonstrate that our MUPPET outperforms state-of-the-art alternative methods,
often by a large margin. We also show that our MUPPET can be easily extended to
tackle the few-shot object detection problem and again achieves the
state-of-the-art performance on MS-COCO dataset. The code will be available in
https://github.com/sauradip/MUPPETComment: Technical Repor
Skin infiltrating NK cells in cutaneous T-cell lymphoma are increased in number and display phenotypic alterations partially driven by the tumor
Cutaneous T-cell lymphomas (CTCL) are characterized by focal infiltration of malignant T cell clones in solitary skin lesions. Many CTCL patients experience an indolent disease, but some progress to advanced disease with high fatality. We hypothesized that natural killer (NK) cells participate in local control of tumor growth in CTCL skin. Immunohistochemistry and flow cytometry analysis of the density, localization, phenotype and function of NK cells in twenty-nine fresh or formalin-fixed skin biopsies from twenty-four CTCL patients and twenty-three biopsies from twenty healthy controls highlighted higher numbers of CD56+CD3- NK cells in CTCL skin. A reduced fraction of CTCL skin NK cells expressed the maturation marker CD57, the cytotoxic protein granzyme B and the activation marker CD69, indicating reduced tumor-killing abilities of the NK cells. Retained expression of immune checkpoint proteins or inhibitory proteins including PD1, TIM3, LAG3, CD73 and NKG2A and the activating receptors CD16 and NKp46 indicated maintained effector functions. Indeed, the capacity of NK cells to produce anti-tumor acting IFNγ upon PMA+ionomycin stimulation was similar in cells from CTCL and healthy skin. Co-cultures of primary human NK cells or the NK cell line NKL with CTCL cells resulted in reduced levels of granzyme B and CD69, indicating that close cellular interactions with CTCL cells induced the impaired functional NK cell phenotype. In conclusion, increased numbers of NK cells in CTCL skin exhibit a partially impaired phenotype in terms of activity. Enhancing NK cell activity with NK cell activating cytokines such as IL-15 or immune checkpoint blockade therefore represents a potential immunotherapeutic approach in CTCL.publishedVersio
- …