Promptable Game Models: Text-Guided Game Simulation via Masked Diffusion Models

Neural video game simulators emerged as powerful tools to generate and edit videos. Their idea is to represent games as the evolution of an environment's state driven by the actions of its agents. While such a paradigm enables users to play a game action-by-action, its rigidity precludes more semantic forms of control. To overcome this limitation, we augment game models with prompts specified as a set of natural language actions and desired states. The result-a Promptable Game Model (PGM)-makes it possible for a user to play the game by prompting it with high- and low-level action sequences. Most captivatingly, our PGM unlocks the director's mode, where the game is played by specifying goals for the agents in the form of a prompt. This requires learning "game AI", encapsulated by our animation model, to navigate the scene using high-level constraints, play against an adversary, and devise a strategy to win a point. To render the resulting state, we use a compositional NeRF representation encapsulated in our synthesis model. To foster future research, we present newly collected, annotated and calibrated Tennis and Minecraft datasets. Our method significantly outperforms existing neural video game simulators in terms of rendering quality and unlocks applications beyond the capabilities of the current state of the art. Our framework, data, and models are available at https://snap-research.github.io/promptable-game-models/.Comment: ACM Transactions on Graphics \c{opyright} Copyright is held by the owner/author(s) 2023. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in ACM Transactions on Graphics, http://dx.doi.org/10.1145/363570

Vision Sensors and Edge Detection

Vision Sensors and Edge Detection book reflects a selection of recent developments within the area of vision sensors and edge detection. There are two sections in this book. The first section presents vision sensors with applications to panoramic vision sensors, wireless vision sensors, and automated vision sensor inspection, and the second one shows image processing techniques, such as, image measurements, image transformations, filtering, and parallel computing

Analysis domain model for shared virtual environments

The field of shared virtual environments, which also encompasses online games and social 3D environments, has a system landscape consisting of multiple solutions that share great functional overlap. However, there is little system interoperability between the different solutions. A shared virtual environment has an associated problem domain that is highly complex raising difficult challenges to the development process, starting with the architectural design of the underlying system. This paper has two main contributions. The first contribution is a broad domain analysis of shared virtual environments, which enables developers to have a better understanding of the whole rather than the part(s). The second contribution is a reference domain model for discussing and describing solutions - the Analysis Domain Model

Outsourcing Counter-insurgency: State Investment in Pro-government Militias as a Response to Rebel Strength

In India, Nigeria, and Iraq, the government has invested in the capacity of PGMs despite the associated risks. This paper attempts to address under what conditions do governments invest in PGMs during civil wars. I argue that the degree to which a government is willing to invest in a PGM is mediated by the relative threat imposed by the rebel group(s) that the government is facing during the civil war. Governments facing relatively weak PGMs are more likely to invest in PGMs to create effective counter-insurgents. The government is less concerned that a PGM that shirks its duties or defects to the rebel group will be able to shift the war in favor of the rebels. Therefore, I argue that the likelihood of investment in PGMs increases as rebels become weaker relative to the state. Using cross-national data (1989 - 2004), I find partial support for the hypothesis that governments invest in PGMs as rebels become weaker relative to the state.Master of Art

Wired Bodies. New Perspectives on the Machine-Organism Analogy

The machine-organism analogy has played a pivotal role in the history of Western philosophy and science. Notwithstanding its apparent simplicity, it hides complex epistemological issues about the status of both organism and machine and the nature of their interaction. What is the real object of this analogy: organisms as a whole, their parts or, rather, bodily functions? How can the machine serve as a model for interpreting biological phenomena, cognitive processes, or more broadly the social and cultural transformations of the relations between individuals, and between individuals and the environments in which they live? Wired Bodies. New Perspectives on the Machine-Organism Analogy provides the reader with some of the latest perspectives on this vast debate, addressing three major topics:1) the development of a ‘mechanistic’ framework in medicine and biology; 2) the methodological issues underlying the use of ‘simulation’ in cognitive science; 3) the interaction between humans and machines according to 20th-century epistemology

Wired Bodies. New Perspectives on the Machine-Organism Analogy

The machine-organism analogy has played a pivotal role in the history of Western philosophy and science. Notwithstanding its apparent simplicity, it hides complex epistemological issues about the status of both organism and machine and the nature of their interaction. What is the real object of this analogy: organisms as a whole, their parts or, rather, bodily functions? How can the machine serve as a model for interpreting biological phenomena, cognitive processes, or more broadly the social and cultural transformations of the relations between individuals, and between individuals and the environments in which they live. Wired bodies. New perspectives on the machine-organism analogy provides the reader with some of the latest perspectives on this vast debate, addressing three major topics: 1) the development of a ‘mechanistic’ framework in medicine and biology; 2) the methodological issues underlying the use of ‘simulation’ in cognitive science; 3) the interaction between humans and machines according to 20th century epistemology