101,429 research outputs found
Recommended from our members
Reinforcement Learning for Generative Art
Reinforcement learning (RL) is an efficient class of sequential decision-making algorithms that have achieved remarkable success in a broad range of applications, such as robotic manipulations, strategic games, or autonomous driving. The most well-known example of reinforcement learning is AlphaGo, a computer program that plays the board game Go and outperforms top human Go players. Unlike other two major machine learning categories, supervised learning and unsupervised learning, in which media artists are actively engaged, reinforcement learning has yet to result in many creative applications. Generative art is usually driven, in whole or in part, by autonomous systems that are derived from a set of rules. Interestingly, an RL policy can be seen as an autonomous system where the rules are learned by interacting with its environment. Regardless of its initial purpose, reinforcement learning has the potential to expand the boundary of generative art. However, a formal process of applying reinforcement learning to generative art does not yet exist and the current RL tools require an in-depth understanding of RL concepts. To bridge the gap, the first part of the dissertation introduces a conceptual framework to adapt reinforcement learning for generative art. The framework proposes a term RL-based generative art to denote a novel form of generative art of which the use of RL agents is the key element. The creative process of RL-based generative art and possible emergent behaviors are discussed in the framework. This leads to a discussion of several author's related practices on generative art, deep-learning art, and reinforcement learning. Those practices are critical for understanding the conceptual and technical details of each component in order to construct the framework. The second part introduces RL5, a JavaScript library for rapidly prototyping RL environments and training RL policies in web browsers. The library combines RL algorithms and RL environments into one framework and is fully compatible with p5.js. RL5 is developed with a particular focus on simplicity to favor (re)usability of RL algorithms and development of RL environments. Specifically, the library implemented three RL algorithms, Tabular Q-learning, REINFORCE, and DDPG, to cover all the three families of model-free RL, and nine RL environments that six of them address autonomous agents in steering behaviors, which can be used as building blocks for complex systems. Finally, the author demonstrates four different use cases of how to apply RL5 for pedagogical and creative applications
MEGA: Multilingual Evaluation of Generative AI
Generative AI models have shown impressive performance on many Natural
Language Processing tasks such as language understanding, reasoning, and
language generation. An important question being asked by the AI community
today is about the capabilities and limits of these models, and it is clear
that evaluating generative AI is very challenging. Most studies on generative
LLMs have been restricted to English and it is unclear how capable these models
are at understanding and generating text in other languages. We present the
first comprehensive benchmarking of generative LLMs - MEGA, which evaluates
models on standard NLP benchmarks, covering 16 NLP datasets across 70
typologically diverse languages. We compare the performance of generative LLMs
including Chat-GPT and GPT-4 to State of the Art (SOTA) non-autoregressive
models on these tasks to determine how well generative models perform compared
to the previous generation of LLMs. We present a thorough analysis of the
performance of models across languages and tasks and discuss challenges in
improving the performance of generative LLMs on low-resource languages. We
create a framework for evaluating generative LLMs in the multilingual setting
and provide directions for future progress in the field.Comment: EMNLP 202
The Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype Classification
We present the Bayesian Case Model (BCM), a general framework for Bayesian case-based reasoning (CBR) and prototype classification and clustering. BCM brings the intuitive power of CBR to a Bayesian generative framework. The BCM learns prototypes, the âquintessentialâ observations that best represent clusters in a dataset, by performing joint inference on cluster labels, prototypes and important features. Simultaneously, BCM pursues sparsity by learning subspaces, the sets of features that play important roles in the characterization of the prototypes. The prototype and subspace representation provides quantitative benefits in interpretability while preserving classification accuracy. Human subject experiments verify statistically significant improvements to participantsâ understanding when using explanations produced by BCM, compared to those given by prior art
Patch-Wise Point Cloud Generation: A Divide-and-Conquer Approach
A generative model for high-fidelity point clouds is of great importance in
synthesizing 3d environments for applications such as autonomous driving and
robotics. Despite the recent success of deep generative models for 2d images,
it is non-trivial to generate 3d point clouds without a comprehensive
understanding of both local and global geometric structures. In this paper, we
devise a new 3d point cloud generation framework using a divide-and-conquer
approach, where the whole generation process can be divided into a set of
patch-wise generation tasks. Specifically, all patch generators are based on
learnable priors, which aim to capture the information of geometry primitives.
We introduce point- and patch-wise transformers to enable the interactions
between points and patches. Therefore, the proposed divide-and-conquer approach
contributes to a new understanding of point cloud generation from the geometry
constitution of 3d shapes. Experimental results on a variety of object
categories from the most popular point cloud dataset, ShapeNet, show the
effectiveness of the proposed patch-wise point cloud generation, where it
clearly outperforms recent state-of-the-art methods for high-fidelity point
cloud generation
Visuality and the haptic qualities of the line in generative art
The line has an important and particular relationship with the generative artwork distinct from other elements such as the âpixelâ, âvoxelâ or the âpointsâ that make up point clouds. The line has a dual nature as both continuous and discrete which makes it perhaps uniquely placed to straddle the analog and digital worlds. It has a haptic or felt quality as well as an inherent ambiguity that promotes a relatively active interpretive role for the audience.
There is an extensive history of the line in generative systems and artworks, taking both analog and digital forms. That it continues to play an important role, alongside other more photographically inspired âperceptual schemasâ, may be a testament to its enduring usefulness and unique character.
This paper considers the particular affordances and the âvisualityâ of the line in relation to generative artworks. This includes asking how we might account for the felt quality of lines and the socially and culturally constructed aspects that shape our relationship with them. It asks whether, in what has been described as a âpost digitalâ or even âpost post digitalâ world, the line may offer a way to re-emphasise a more human scale and a materiality that can push back, gently, against other more dominant perceptual schemas. It also asks what generative art can learn from drawing theory, many of the concerns of which parallel and intersect with those of generative art
- âŠ