Towards a Smart Drone Cinematographer for Filming Human Motion

Affordable consumer drones have made capturing aerial footage more convenient and accessible. However, shooting cinematic motion videos using a drone is challenging because it requires users to analyze dynamic scenarios while operating the controller. In this thesis, our task is to develop an autonomous drone cinematography system to capture cinematic videos of human motion. We understand the system's filming performance to be influenced by three key components: 1) video quality metric, which measures the aesthetic quality -- the angle, the distance, the image composition -- of the captured video, 2) visual feature, which encapsulates the visual elements that influence the filming style, and 3) camera planning, which is a decision-making model that predicts the next best movement. By analyzing these three components, we designed two autonomous drone cinematography systems using both heuristic-based methods and learning-based methods.For the first system, we designed an Autonomous CinemaTography system -- "ACT" by proposing a viewpoint quality metric focusing on the visibility of the 3D human skeleton of the subject. We expanded the application of human motion analysis and simplified manual control by assisting viewpoint selection using a through-the-lens method. For the second system, we designed an imitation-based system that learns the artistic intention of the cameramen through watching professional aerial videos. We designed a camera planner that analyzes the video contents and previous camera motion to predict future camera motion. Furthermore, we propose a planning framework, which can imitate a filming style by ``seeing" only one single demonstration video of such style. We named it ``one-shot imitation filming." To the best of our knowledge, this is the first work that extends imitation learning to autonomous filming. Experimental results in both simulation and field test exhibit significant improvements over existing techniques and our approach managed to help inexperienced pilots capture cinematic videos

Avion 2006-02-28

https://commons.erau.edu/avion/2042/thumbnail.jp

Mirror - Vol. 09, No. 02 - April 11, 1985

The Mirror (sometimes called the Fairfield Mirror) is the official student newspaper of Fairfield University, and is published weekly during the academic year (September - May). It runs from 1977 - the present; current issues are available online.https://digitalcommons.fairfield.edu/archives-mirror/1177/thumbnail.jp

Supplement to Lauri Lahti’s conference article "Educational framework for adoption of vocabulary based on Wikipedia linkage and spaced learning"

A supplement to Lauri Lahti’s conference article in 2012 "Educational framework for adoption of vocabulary based on Wikipedia linkage and spaced learning" so that this supplement was referenced to by the original publication.Not reviewe

Learning by observation using Qualitative Spatial Relations

We present an approach to the problem of learning by observation in spatially-situated tasks, whereby an agent learns to imitate the behaviour of an observed expert, with no direct interaction and limited observations. The form of knowledge representation used for these observations is crucial, and we apply Qualitative Spatial-Relational representations to compress continuous, metric state-spaces into symbolic states to maximise the generalisability of learned models and minimise knowledge engineering. Our system self-configures these representations of the world to discover configurations of features most relevant to the task, and thus build good predictive models. We then show how these models can be employed by situated agents to control their behaviour, closing the loop from observation to practical implementation. We evaluate our approach in the simulated RoboCup Soccer domain and the Real-Time Strategy game Starcraft, and successfully demonstrate how a system using our approach closely mimics the behaviour of both synthetic (AI controlled) players, and also human-controlled players through observation. We further evaluate our work in Reinforcement Learning tasks in these domains, and show that our approach improves the speed at which such models can be learned

Neuro-cognitive and social components of dyadic motor interactions revealed by the kinematics of a joint-grasping task

This thesis describes a PhD project is based on the notion that we live our whole life dipped into an interactive social environment where we observe and act together with others and where our behavior is influenced by first sight impressions, social categorizations and stereotypes which automatically and unavoidably arise during interactions. Nevertheless, the bidirectional impact of interpersonal coding on dyadic motor interactions has never been directly investigated. Moreover, the neurocognitive bases of social interaction are still poorly understood. In particular, in every-day dyadic encounters we usually interact with others in non-imitative fashions (Sebanz et al. 2006), challenging the hypothesis of a direct matching between action observation and action execution within one system (“common coding approach”, Prinz 1997), which is instead supported by neurophysiological data on the so called “mirror neurons”(Rizzolatti and Sinigaglia 2010) which fire both during action execution and observation of similar actions performed by others. Suggestion is made that what characterizes joint action is the presence of a common goal (i.e. the “shared” goal, Butterfill 2012) which organizes individuals’ behaviour and channel simulative processes. During her PhD, Lucia Sacheli developed a novel interactive scenario able to investigate face-to-face dyadic interactions within a naturalistic and yet controlled experimental environment, with the aim to build a more coherent model of the role of simulative mechanisms during social interaction and on the role of socio-emotional variables in modulating these processes. This scenario required pairs of participants to reciprocally coordinate their reach-to-grasp movements and perform on-line mutual adjustments in time and space in order to fulfill a common (motor) goal. So far, she demonstrated by means of kinematic data analysis that simulation of the partner’s movement is task-dependent (Sacheli et al. 2013) and modulated by the interpersonal relationship linking co-agents (Sacheli et al. 2012) and by social stereotypes as ethnic biases (Sacheli et al. under review). Moreover, she used the same scenario to investigate the different contribution of the parietal and frontal nodes of the fronto-parietal “mirror” network during joint-action by means of Transcranial Magnetic Stimulation combined with analysis of kinematics

Neuro-cognitive and social components of dyadic motor interactions revealed by the kinematics of a joint-grasping task

This thesis describes a PhD project is based on the notion that we live our whole life dipped into an interactive social environment where we observe and act together with others and where our behavior is influenced by first sight impressions, social categorizations and stereotypes which automatically and unavoidably arise during interactions. Nevertheless, the bidirectional impact of interpersonal coding on dyadic motor interactions has never been directly investigated. Moreover, the neurocognitive bases of social interaction are still poorly understood. In particular, in every-day dyadic encounters we usually interact with others in non-imitative fashions (Sebanz et al. 2006), challenging the hypothesis of a direct matching between action observation and action execution within one system (“common coding approach”, Prinz 1997), which is instead supported by neurophysiological data on the so called “mirror neurons”(Rizzolatti and Sinigaglia 2010) which fire both during action execution and observation of similar actions performed by others. Suggestion is made that what characterizes joint action is the presence of a common goal (i.e. the “shared” goal, Butterfill 2012) which organizes individuals’ behaviour and channel simulative processes. During her PhD, Lucia Sacheli developed a novel interactive scenario able to investigate face-to-face dyadic interactions within a naturalistic and yet controlled experimental environment, with the aim to build a more coherent model of the role of simulative mechanisms during social interaction and on the role of socio-emotional variables in modulating these processes. This scenario required pairs of participants to reciprocally coordinate their reach-to-grasp movements and perform on-line mutual adjustments in time and space in order to fulfill a common (motor) goal. So far, she demonstrated by means of kinematic data analysis that simulation of the partner’s movement is task-dependent (Sacheli et al. 2013) and modulated by the interpersonal relationship linking co-agents (Sacheli et al. 2012) and by social stereotypes as ethnic biases (Sacheli et al. under review). Moreover, she used the same scenario to investigate the different contribution of the parietal and frontal nodes of the fronto-parietal “mirror” network during joint-action by means of Transcranial Magnetic Stimulation combined with analysis of kinematics

The Tiger Vol. 90 Issue 12 1997-01-24

https://tigerprints.clemson.edu/tiger_newspaper/2913/thumbnail.jp

Täydennysosa väitöskirjaan "Tietokoneavusteinen oppiminen perustuen karttuviin sanastoihin, käsiteverkostoihin ja Wikipedian linkitykseen"

A supplement to Lauri Lahti’s doctoral dissertation in 2015 "Computer-Assisted Learning Based on Cumulative Vocabularies, Conceptual Networks and Wikipedia Linkage" so that this supplement was referenced to by the original publication.Täydennysosa väitöskirjaan "Tietokoneavusteinen oppiminen perustuen karttuviin sanastoihin, käsiteverkostoihin ja Wikipedian linkitykseen"Not reviewe