176 research outputs found
Scalable Estimation of Dirichlet Process Mixture Models on Distributed Data
We consider the estimation of Dirichlet Process Mixture Models (DPMMs) in
distributed environments, where data are distributed across multiple computing
nodes. A key advantage of Bayesian nonparametric models such as DPMMs is that
they allow new components to be introduced on the fly as needed. This, however,
posts an important challenge to distributed estimation -- how to handle new
components efficiently and consistently. To tackle this problem, we propose a
new estimation method, which allows new components to be created locally in
individual computing nodes. Components corresponding to the same cluster will
be identified and merged via a probabilistic consolidation scheme. In this way,
we can maintain the consistency of estimation with very low communication cost.
Experiments on large real-world data sets show that the proposed method can
achieve high scalability in distributed and asynchronous environments without
compromising the mixing performance.Comment: This paper is published on IJCAI 2017.
https://www.ijcai.org/proceedings/2017/64
UntrimmedNets for Weakly Supervised Action Recognition and Detection
Current action recognition methods heavily rely on trimmed videos for model
training. However, it is expensive and time-consuming to acquire a large-scale
trimmed video dataset. This paper presents a new weakly supervised
architecture, called UntrimmedNet, which is able to directly learn action
recognition models from untrimmed videos without the requirement of temporal
annotations of action instances. Our UntrimmedNet couples two important
components, the classification module and the selection module, to learn the
action models and reason about the temporal duration of action instances,
respectively. These two components are implemented with feed-forward networks,
and UntrimmedNet is therefore an end-to-end trainable architecture. We exploit
the learned models for action recognition (WSR) and detection (WSD) on the
untrimmed video datasets of THUMOS14 and ActivityNet. Although our UntrimmedNet
only employs weak supervision, our method achieves performance superior or
comparable to that of those strongly supervised approaches on these two
datasets.Comment: camera-ready version to appear in CVPR201
DORT: Modeling Dynamic Objects in Recurrent for Multi-Camera 3D Object Detection and Tracking
Recent multi-camera 3D object detectors usually leverage temporal information
to construct multi-view stereo that alleviates the ill-posed depth estimation.
However, they typically assume all the objects are static and directly
aggregate features across frames. This work begins with a theoretical and
empirical analysis to reveal that ignoring the motion of moving objects can
result in serious localization bias. Therefore, we propose to model Dynamic
Objects in RecurrenT (DORT) to tackle this problem. In contrast to previous
global Bird-Eye-View (BEV) methods, DORT extracts object-wise local volumes for
motion estimation that also alleviates the heavy computational burden. By
iteratively refining the estimated object motion and location, the preceding
features can be precisely aggregated to the current frame to mitigate the
aforementioned adverse effects. The simple framework has two significant
appealing properties. It is flexible and practical that can be plugged into
most camera-based 3D object detectors. As there are predictions of object
motion in the loop, it can easily track objects across frames according to
their nearest center distances. Without bells and whistles, DORT outperforms
all the previous methods on the nuScenes detection and tracking benchmarks with
62.5\% NDS and 57.6\% AMOTA, respectively. The source code will be released
PointLLM: Empowering Large Language Models to Understand Point Clouds
The unprecedented advancements in Large Language Models (LLMs) have created a
profound impact on natural language processing but are yet to fully embrace the
realm of 3D understanding. This paper introduces PointLLM, a preliminary effort
to fill this gap, thereby enabling LLMs to understand point clouds and offering
a new avenue beyond 2D visual data. PointLLM processes colored object point
clouds with human instructions and generates contextually appropriate
responses, illustrating its grasp of point clouds and common sense.
Specifically, it leverages a point cloud encoder with a powerful LLM to
effectively fuse geometric, appearance, and linguistic information. We collect
a novel dataset comprising 660K simple and 70K complex point-text instruction
pairs to enable a two-stage training strategy: initially aligning latent spaces
and subsequently instruction-tuning the unified model. To rigorously evaluate
our model's perceptual abilities and its generalization capabilities, we
establish two benchmarks: Generative 3D Object Classification and 3D Object
Captioning, assessed through three different methods, including human
evaluation, GPT-4/ChatGPT evaluation, and traditional metrics. Experiment
results show that PointLLM demonstrates superior performance over existing 2D
baselines. Remarkably, in human-evaluated object captioning tasks, PointLLM
outperforms human annotators in over 50% of the samples. Codes, datasets, and
benchmarks are available at https://github.com/OpenRobotLab/PointLLM .Comment: 19 pages. Empowering large language models with 3D point cloud
understanding, accompanied by a novel dataset and carefully designed
benchmarks. Project page: https://runsenxu.com/projects/PointLL
- …