63 research outputs found
PaletteNeRF: Palette-based Color Editing for NeRFs
Neural Radiance Field (NeRF) is a powerful tool to faithfully generate novel
views for scenes with only sparse captured images. Despite its strong
capability for representing 3D scenes and their appearance, its editing ability
is very limited. In this paper, we propose a simple but effective extension of
vanilla NeRF, named PaletteNeRF, to enable efficient color editing on
NeRF-represented scenes. Motivated by recent palette-based image decomposition
works, we approximate each pixel color as a sum of palette colors modulated by
additive weights. Instead of predicting pixel colors as in vanilla NeRFs, our
method predicts additive weights. The underlying NeRF backbone could also be
replaced with more recent NeRF models such as KiloNeRF to achieve real-time
editing. Experimental results demonstrate that our method achieves efficient,
view-consistent, and artifact-free color editing on a wide range of
NeRF-represented scenes.Comment: 12 pages, 10 figure
OpenAGI: When LLM Meets Domain Experts
Human intelligence has the remarkable ability to assemble basic skills into
complex ones so as to solve complex tasks. This ability is equally important
for Artificial Intelligence (AI), and thus, we assert that in addition to the
development of large, comprehensive intelligent models, it is equally crucial
to equip such models with the capability to harness various domain-specific
expert models for complex task-solving in the pursuit of Artificial General
Intelligence (AGI). Recent developments in Large Language Models (LLMs) have
demonstrated remarkable learning and reasoning abilities, making them promising
as a controller to select, synthesize, and execute external models to solve
complex tasks. In this project, we develop OpenAGI, an open-source AGI research
platform, specifically designed to offer complex, multi-step tasks and
accompanied by task-specific datasets, evaluation metrics, and a diverse range
of extensible models. OpenAGI formulates complex tasks as natural language
queries, serving as input to the LLM. The LLM subsequently selects,
synthesizes, and executes models provided by OpenAGI to address the task.
Furthermore, we propose a Reinforcement Learning from Task Feedback (RLTF)
mechanism, which uses the task-solving result as feedback to improve the LLM's
task-solving ability. Thus, the LLM is responsible for synthesizing various
external models for solving complex tasks, while RLTF provides feedback to
improve its task-solving ability, enabling a feedback loop for self-improving
AI. We believe that the paradigm of LLMs operating various expert models for
complex task-solving is a promising approach towards AGI. To facilitate the
community's long-term improvement and evaluation of AGI's ability, we
open-source the code, benchmark, and evaluation methods of the OpenAGI project
at https://github.com/agiresearch/OpenAGI.Comment: 18 pages, 6 figures, 7 table
ASP: Automatic Selection of Proxy dataset for efficient AutoML
Deep neural networks have gained great success due to the increasing amounts
of data, and diverse effective neural network designs. However, it also brings
a heavy computing burden as the amount of training data is proportional to the
training time. In addition, a well-behaved model requires repeated trials of
different structure designs and hyper-parameters, which may take a large amount
of time even with state-of-the-art (SOTA) hyper-parameter optimization (HPO)
algorithms and neural architecture search (NAS) algorithms. In this paper, we
propose an Automatic Selection of Proxy dataset framework (ASP) aimed to
dynamically find the informative proxy subsets of training data at each epoch,
reducing the training data size as well as saving the AutoML processing time.
We verify the effectiveness and generalization of ASP on CIFAR10, CIFAR100,
ImageNet16-120, and ImageNet-1k, across various public model benchmarks. The
experiment results show that ASP can obtain better results than other data
selection methods at all selection ratios. ASP can also enable much more
efficient AutoML processing with a speedup of 2x-20x while obtaining better
architectures and better hyper-parameters compared to utilizing the entire
dataset.Comment: This paper was actually finished in 202
User-Controllable Recommendation via Counterfactual Retrospective and Prospective Explanations
Modern recommender systems utilize users' historical behaviors to generate
personalized recommendations. However, these systems often lack user
controllability, leading to diminished user satisfaction and trust in the
systems. Acknowledging the recent advancements in explainable recommender
systems that enhance users' understanding of recommendation mechanisms, we
propose leveraging these advancements to improve user controllability. In this
paper, we present a user-controllable recommender system that seamlessly
integrates explainability and controllability within a unified framework. By
providing both retrospective and prospective explanations through
counterfactual reasoning, users can customize their control over the system by
interacting with these explanations.
Furthermore, we introduce and assess two attributes of controllability in
recommendation systems: the complexity of controllability and the accuracy of
controllability. Experimental evaluations on MovieLens and Yelp datasets
substantiate the effectiveness of our proposed framework. Additionally, our
experiments demonstrate that offering users control options can potentially
enhance recommendation accuracy in the future. Source code and data are
available at \url{https://github.com/chrisjtan/ucr}.Comment: Accepted for presentation at 26th European Conference on Artificial
Intelligence (ECAI2023
- …