Viewpoint Recommendation for Aesthetic Photography

abstract: This thesis addresses the problem of recommending a viewpoint for aesthetic photography. Viewpoint recommendation is suggesting the best camera pose to capture a visually pleasing photograph of the subject of interest by using any end-user device such as drone, mobile robot or smartphone. Solving this problem enables to capture visually pleasing photographs autonomously in areal photography, wildlife photography, landscape photography or in personal photography. The viewpoint recommendation problem can be divided into two stages: (a) generating a set of dense novel views based on the basis views captured about the subject. The dense novel views are useful to better understand the scene and to know how the subject looks from different viewpoints and (b) each novel is scored based on how aesthetically good it is. The viewpoint with the greatest aesthetic score is recommended for capturing a visually pleasing photograph.Dissertation/ThesisMasters Thesis Computer Engineering 201

Towards a Smart Drone Cinematographer for Filming Human Motion

Affordable consumer drones have made capturing aerial footage more convenient and accessible. However, shooting cinematic motion videos using a drone is challenging because it requires users to analyze dynamic scenarios while operating the controller. In this thesis, our task is to develop an autonomous drone cinematography system to capture cinematic videos of human motion. We understand the system's filming performance to be influenced by three key components: 1) video quality metric, which measures the aesthetic quality -- the angle, the distance, the image composition -- of the captured video, 2) visual feature, which encapsulates the visual elements that influence the filming style, and 3) camera planning, which is a decision-making model that predicts the next best movement. By analyzing these three components, we designed two autonomous drone cinematography systems using both heuristic-based methods and learning-based methods.For the first system, we designed an Autonomous CinemaTography system -- "ACT" by proposing a viewpoint quality metric focusing on the visibility of the 3D human skeleton of the subject. We expanded the application of human motion analysis and simplified manual control by assisting viewpoint selection using a through-the-lens method. For the second system, we designed an imitation-based system that learns the artistic intention of the cameramen through watching professional aerial videos. We designed a camera planner that analyzes the video contents and previous camera motion to predict future camera motion. Furthermore, we propose a planning framework, which can imitate a filming style by ``seeing" only one single demonstration video of such style. We named it ``one-shot imitation filming." To the best of our knowledge, this is the first work that extends imitation learning to autonomous filming. Experimental results in both simulation and field test exhibit significant improvements over existing techniques and our approach managed to help inexperienced pilots capture cinematic videos

Probabilistic Roadmaps for Virtual Camera Pathing with Cinematographic Principles

As technology use increases in the world and inundates everyday life, the visual aspect of technology or computer graphics becomes increasingly important. This thesis presents a system for the automatic generation of virtual camera paths for fly-throughs of a digital scene. The sample scene used in this work is an underwater setting featuring a shipwreck model with other virtual underwater elements such as rocks, bubbles and caustics. The digital shipwreck model was reconstructed from an actual World War II shipwreck, resting off the coast of Malta. Video and sonar scans from an autonomous underwater vehicle were used in a photogrammetry pipeline to create the model. This thesis presents an algorithm to automatically generate virtual camera paths using a robotics motion planning algorithm, specifically the probabilistic roadmap. This algorithm uses a rapidly-exploring random tree to quickly cover a space and generate small maps with good coverage. For this work, the camera pitch and height along a specified path were automatically generated using cinematographic and geometric principles. These principles were used to evaluate potential viewpoints and influence whether or not a view is used in the final path. A computational evaluation of ‘the rule of thirds’ and evaluation of the model normals relative to the camera viewpoint are used to represent cinematography and geometry principles. In addition to the system that automatically generates virtual camera paths, a user study is presented which evaluates ten different videos produced via camera paths with this system. The videos were created using different viewpoint evaluation methods and different path generation characteristics. The user study indicates that users prefer paths generated by our system over flat and randomly generated paths. Specifically, users prefer paths generated using the computational evaluation of the rule of thirds and paths that show the wreck from a large variety of angles but without too much camera undulation

Autonomous Quadcopter Videographer

In recent years, the interest in quadcopters as a robotics platform for autonomous photography has increased. This is due to their small size and mobility, which allow them to reach places that are difficult or even impossible for humans. This thesis focuses on the design of an autonomous quadcopter videographer, i.e. a quadcopter capable of capturing good footage of a specific subject. In order to obtain this footage, the system needs to choose appropriate vantage points and control the quadcopter. Skilled human videographers can easily spot good filming locations where the subject and its actions can be seen clearly in the resulting video footage, but translating this knowledge to a robot can be complex. We present an autonomous system implemented on a commercially available quadcopter that achieves this using only the monocular information and an accelerometer. Our system has two vantage point selection strategies: 1) a reactive approach, which moves the robot to a fixed location with respect to the human and 2) the combination of the reactive approach and a POMDP planner that considers the target\u27s movement intentions. We compare the behavior of these two approaches under different target movement scenarios. The results show that the POMDP planner obtains more stable footage with less quadcopter motion

The Machine as Art/ The Machine as Artist

The articles collected in this volume from the two companion Arts Special Issues, “The Machine as Art (in the 20th Century)” and “The Machine as Artist (in the 21st Century)”, represent a unique scholarly resource: analyses by artists, scientists, and engineers, as well as art historians, covering not only the current (and astounding) rapprochement between art and technology but also the vital post-World War II period that has led up to it; this collection is also distinguished by several of the contributors being prominent individuals within their own fields, or as artists who have actually participated in the still unfolding events with which it is concerne

Machines That Learn: Aesthetics of Adaptive Behaviors in Agent-based Art

Since the post-war era, artists have been exploring the use of embodied, artificial agents. This artistic activity runs parallel to research in computer science, in domains such as Cybernetics, Artificial Intelligence and Artificial Life. This thesis offers an account of a particular facet of this broader work — namely, a study of the artistic practice of agent-based, adaptive computational artistic installations that make use of Machine Learning methods. Machine Learning is a sub-field of the computer science area of Artificial Intelligence that employs mathematical models to classify and make predictions based on data or experience rather than on logical rules. These artworks that integrate Machine Learning into their structures raise a number of important questions: (1) What new forms of aesthetic experience do Machine Learning methods enable or make possible when utilized outside of their intended context, and are instead carried over into artistic works? (2) What characterizes the practice of using adaptive computational methods in agent-based artworks? And finally, (3) what kind of worldview are these works fostering? To address these questions, I examine the history of Machine Learning in both art and science, illustrating how artists and engineers alike have made use of these methods historically. I also analyze the defining scientific characteristics of Machine Learning through a practitioner’s lens, concretely articulating how properties of Machine Learning interplay in media artworks that behave and evolve in real time. I later develop a framework for understanding machine behaviors based on the morphological aspects of the temporal unfolding of agent behaviors as a tool for comprehending both adaptive and non-adaptive behaviors in works of art. Finally, I expose how adaptive technologies suggest a new worldview for art that accounts for the performative engagement of agents adapting to one another, which implies a certain way of losing control in the face of the indeterminacy and the unintelligibility of alien agencies and their behaviors