Implementasi Screenplay dan Audio Foley Effect pada Pembuatan Film Animasi 3D "Si Molek"

Screenplay adalah kisah yang diceritakan dengan gambar, dalam dialog dan deskripsi, dan ditempatkan dalam konteks struktur dramatis, yang menguraikan urut-urutan adegan, tempat, dialog dan keadaan. Foley merupakan jenis suara yang dibuat untuk mengisi noise pemain dalam film pada waktu yang sebenarnya atau nyata. Teknologi animasi sendiri sudah menyebar hingga pada pembuatan cerita daerah/cerita rakyat yang awalnya hanya diceritakan secara lisan/tertulis. Cerita rakyat adalah suatu cerita yang pada dasarnya disampaikan oleh seseorang pada orang lain melalui penuturan lisan. Hampir disetiap daerah di Indonesia memiliki cerita rakyatnya masing-masing. Salah satu cerita rakyat dari provinsi Riau adalah cerita rakyat Si Molek. Pada penelitian ini, akan dibuat sebuah film animasi 3D cerita rakyat Si Molek dengan menggunakan teknik Screenplay dan juga teknik Foley Effect. Teknik Screenplay digunakan sebagai acuan atau pedoman dalam proses pembuatan animasi 3D Si Molek. Sedangkan teknik Foley Effect digunakan karena tanpa adanya Foley Effect sebuah film akan terasa kurang realistis dan natural. Dengan film animasi 3D Si Molek yang menerapkan teknik Screenplay dan Foley Effect ini diharapkan menghasilkan sebuah film animasi 3D yang baik dan juga sebagai sarana memperkenalkan cerita rakyat ini kepada masyarakat. Berdasarkan hasil pengujian yang telah dilakukan teknik Screenplay dan teknik Foley Effect berhasil diimplementasikan dengan benar

Agents meet OKR: An Object and Key Results Driven Agent System with Hierarchical Self-Collaboration and Self-Evaluation

In this study, we introduce the concept of OKR-Agent designed to enhance the capabilities of Large Language Models (LLMs) in task-solving. Our approach utilizes both self-collaboration and self-correction mechanism, facilitated by hierarchical agents, to address the inherent complexities in task-solving. Our key observations are two-fold: first, effective task-solving demands in-depth domain knowledge and intricate reasoning, for which deploying specialized agents for individual sub-tasks can markedly enhance LLM performance. Second, task-solving intrinsically adheres to a hierarchical execution structure, comprising both high-level strategic planning and detailed task execution. Towards this end, our OKR-Agent paradigm aligns closely with this hierarchical structure, promising enhanced efficacy and adaptability across a range of scenarios. Specifically, our framework includes two novel modules: hierarchical Objects and Key Results generation and multi-level evaluation, each contributing to more efficient and robust task-solving. In practical, hierarchical OKR generation decomposes Objects into multiple sub-Objects and assigns new agents based on key results and agent responsibilities. These agents subsequently elaborate on their designated tasks and may further decompose them as necessary. Such generation operates recursively and hierarchically, culminating in a comprehensive set of detailed solutions. The multi-level evaluation module of OKR-Agent refines solution by leveraging feedback from all associated agents, optimizing each step of the process. This ensures solution is accurate, practical, and effectively address intricate task requirements, enhancing the overall reliability and quality of the outcome. Experimental results also show our method outperforms the previous methods on several tasks. Code and demo are available at https://okr-agent.github.io

A Human-Computer Duet System for Music Performance

Virtual musicians have become a remarkable phenomenon in the contemporary multimedia arts. However, most of the virtual musicians nowadays have not been endowed with abilities to create their own behaviors, or to perform music with human musicians. In this paper, we firstly create a virtual violinist, who can collaborate with a human pianist to perform chamber music automatically without any intervention. The system incorporates the techniques from various fields, including real-time music tracking, pose estimation, and body movement generation. In our system, the virtual musician's behavior is generated based on the given music audio alone, and such a system results in a low-cost, efficient and scalable way to produce human and virtual musicians' co-performance. The proposed system has been validated in public concerts. Objective quality assessment approaches and possible ways to systematically improve the system are also discussed

A Review of Text-to-Animation Systems

Text-to-graphics systems encompass three types of tools: text-to-picture, text-to-scene and text-to-animation. They are an artificial intelligence application wherein users can create 2D and 3D scenes or animations and recently immersive environments from natural language. These complex tasks require the collaboration of various fields, such as natural language processing, computational linguistics and computer graphics. Text-to-animation systems have received more interest than their counterparts, and have been developed for various domains, including theatrical pre-production, education or training. In this survey we focus on text-to-animation systems, discussing their requirements, challenges and proposing solutions, and investigate the natural language understanding approaches adopted in previous research works to solve the challenge of animation generation. We review text-to-animation systems developed over the period 2001-2021, and investigate their recent trends in order to paint the current landscape of the field

Generating Animations from Screenplays

Automatically generating animation from natural language text finds application in a number of areas e.g. movie script writing, instructional videos, and public safety. However, translating natural language text into animation is a challenging task. Existing text-to-animation systems can handle only very simple sentences, which limits their applications. In this paper, we develop a text-to-animation system which is capable of handling complex sentences. We achieve this by introducing a text simplification step into the process. Building on an existing animation generation system for screenwriting, we create a robust NLP pipeline to extract information from screenplays and map them to the system’s knowledge base. We develop a set of linguistic transformation rules that simplify complex sentences. Information extracted from the simplified sentences is used to generate a rough storyboard and video depicting the text. Our sentence simplification module outperforms existing systems in terms of BLEU and SARI metrics.We further evaluated our system via a user study: 68% participants believe that our system generates reasonable animation from input screenplays