Deep Architectures for Visual Recognition and Description

In recent times, digital media contents are inherently of multimedia type, consisting of the form text, audio, image and video. Several of the outstanding computer Vision (CV) problems are being successfully solved with the help of modern Machine Learning (ML) techniques. Plenty of research work has already been carried out in the field of Automatic Image Annotation (AIA), Image Captioning and Video Tagging. Video Captioning, i.e., automatic description generation from digital video, however, is a different and complex problem altogether. This study compares various existing video captioning approaches available today and attempts their classification and analysis based on different parameters, viz., type of captioning methods (generation/retrieval), type of learning models employed, the desired output description length generated, etc. This dissertation also attempts to critically analyze the existing benchmark datasets used in various video captioning models and the evaluation metrics for assessing the final quality of the resultant video descriptions generated. A detailed study of important existing models, highlighting their comparative advantages as well as disadvantages are also included. In this study a novel approach for video captioning on the Microsoft Video Description (MSVD) dataset and Microsoft Video-to-Text (MSR-VTT) dataset is proposed using supervised learning techniques to train a deep combinational framework, for achieving better quality video captioning via predicting semantic tags. We develop simple shallow CNN (2D and 3D) as feature extractors, Deep Neural Networks (DNNs and Bidirectional LSTMs (BiLSTMs) as tag prediction models and Recurrent Neural Networks (RNNs) (LSTM) model as the language model. The aim of the work was to provide an alternative narrative to generating captions from videos via semantic tag predictions and deploy simpler shallower deep model architectures with lower memory requirements as solution so that it is not very memory extensive and the developed models prove to be stable and viable options when the scale of the data is increased. This study also successfully employed deep architectures like the Convolutional Neural Network (CNN) for speeding up automation process of hand gesture recognition and classification of the sign languages of the Indian classical dance form, ‘Bharatnatyam’. This hand gesture classification is primarily aimed at 1) building a novel dataset of 2D single hand gestures belonging to 27 classes that were collected from (i) Google search engine (Google images), (ii) YouTube videos (dynamic and with background considered) and (iii) professional artists under staged environment constraints (plain backgrounds). 2) exploring the effectiveness of CNNs for identifying and classifying the single hand gestures by optimizing the hyperparameters, and 3) evaluating the impacts of transfer learning and double transfer learning, which is a novel concept explored for achieving higher classification accuracy

Developing and evaluating a model for human motion to facilitate low degree-of-freedom robot imitation of human movement

Imitation of human motion is a necessary activity for robots to integrate seamlessly into human-facing environments. While perfect replication is not possible, especially for low degree-of-freedom (DOF) robots, this thesis presents a model for human motion that achieves perceptual imitation. Motion capture data of dyadic interactions was first analyzed to quantify a characteristic of human motion observed in the movement. The leaning of the spine, or verticality, was found to correlate with these movement observations. Verticality was then used to inspire a low-DOF model of human motion using motion capture that can be used to command the movement of simulated robots. Experiments were developed to test users’ perception of the imitation by these 3 and 4-DOF simulated robots of human motion. Verticality was preferred in an initial study over artificially generated motion for the higher DOF robot, Broombot, which was preferred over the lower DOF robot, Rollbot. A study was developed to test the preferences of users when the mapping between human and robot motion was changed for variable human motion. Motion capture-based motion was preferred over artificially generated motion, and a sub-group of respondents who preferred verticality and were more engaged in the survey was found. Since the experiments were performed using motion capture data from a trained ballet dancer, a discussion of the differences between two Indian classical dance styles is included that shows that verticality alone is not representative of all motion and prompts a further analysis to develop socially adaptive robot behavior. In-progress and future work include a hardware implementation that will allow real-time motion capture data to drive simulated and/or physical robots. Menagerie is an in-development performance using the tools developed in this thesis that can include a human with simulated and/or physical robots moving together

Holding Stories

The Afterlife Creative Memory Retreat by The Other Way Works (research and development) invited audience participants to an online retreat via Zoom. Inspired by the 1998 film After Life by Kore-eda Hirokazu, the work invites participants to focus on what they value in life through an in-depth creative exploration of their own important memories facilitating hope, togetherness and a deeper connection to one’s sense of self. The scenographer was part of the team of artists who devised exercises that were then given to the online audience to help them revisit some of their memories. A playful exercise featuring a red thread was introduced and the participants were encouraged to use it for connecting with each other’s screen spaces. Through the medium of touch and the playful scenographic illusion of the thread extending to other participants’ rooms, this tool was used as a way to create a sense of togetherness among the group. I will unpack the above scenographic action through the lens of 4Es cognition: enactive, ecological, embodied, embedded ‘and some cases extended and affective’ (Ward and Stapleton, 2012), suggesting that human cognition is an on-going collaboration between brain, body and environment. If places are shaped by significant historic moments (Hannah, 2011: 56), they are also shaped by the memories we choose to attach to those moments and a certain materiality related to those moments. By understanding thinking not as an individualistic activity but one that is happening within socio-cultural and material knowledge and inextricably integrated with perception and action I will argue that in the Afterlife Creative Memory Retreat, the tactile scenographic element of the thread enhanced the sense of memory as storytelling between the screens

3D Information Technologies in Cultural Heritage Preservation and Popularisation

This Special Issue of the journal Applied Sciences presents recent advances and developments in the use of digital 3D technologies to protect and preserve cultural heritage. While most of the articles focus on aspects of 3D scanning, modeling, and presenting in VR of cultural heritage objects from buildings to small artifacts and clothing, part of the issue is devoted to 3D sound utilization in the cultural heritage field

The potential of dance education to promote social cohesion in a post-conflict society: perspectives of South African pre-service student teachers

This study constitutes a theoretical and qualitative investigation into the meanings and locations of social cohesion in dance education. Theoretical connections between culture, dance education and social cohesion are explored. The empirical investigation is designed as a qualitative case study interrogating pre-service student teachers’ experiences and perceptions of a particular dance education course in a culturally and politically diverse university classroom in post-apartheid South Africa. Open-ended questionnaires, reflective journals and focus group interviews were employed to generate data. Findings indicate that involvement in creative movement and ethno-cultural dances raised awareness of the Self and the Other, engendering perspective and personal transformation, important requisites for social transformation and subsequently social cohesion in a formerly divided society, such as South Africa. In addition, these dance education experiences provided participants with unique encounters with the Other’s culture. These occurred through embodied experiences of the culture of the Other, as well as through bodily negotiations with the Other. These findings lead me to argue that dance education, as pertaining to this particular course, can facilitate spaces conducive to cohesion amongst culturally and politically diverse participants in post-apartheid South Africa

Shifting Interfaces: art research at the intersections of live performance and technology

Merged with duplicate record 10026.1/809 on 08.20.2017 by CS (TIS)This collection of published works is an outcome of my practice-led inter-disciplinary collaborative artistic research into deepening understanding of creative process in the field of contemporary dance. It comprises thirty written works published from 1999 to 2007 in various formats and platforms. This collection is framed by a methodological discussion that provides insight into how this research has intersected over time with diverse fields of practice including contemporary dance, digital and new media arts and non-art domains such as cognitive and social science. Fields are understood in the context of this research to be largely constituted out of the expert practices of individual collaborators. This research starts from an interest in the Impact of new media technologies on dance making/ choreography. The collection of works show evidence, established in the first two publications, of an evolving engagement with two concepts related to this interest: (1) the 'algorithm' as a process-level connection or bridge between dance composition and computation; (2) the empirical study of movement embedded as a 'knowledge base' in the practices of both computer animation and dance and thus forming a special correspondence between them. This collection provides evidence of this research through a period of community-building amongst artists using new media technologies in performance, and culminates in the identification of an emerging 'community of practice' coming together around the formation of a unique body of knowledge pertaining to dance. The late 1990s New Media Art movement provided a supportive context for Important peer-to-peer encounters with creators and users of software tools and platforms in the context of inter-disciplinary art-making. A growing interest in software programming as a creative practice opened up fresh perspectives on possible connections with dance making. It became clear that software's utility alone, including artistic uses of software, was a limited conception. This was the background thinking that informed the first major shift in the research towards the design of software that might augment the creative process of expert choreographers and dancers. This shift from software use to its design, framed by a focus on the development of tools to support dance creation, also provided strong rationale to deepen the research into dance making processes. In the second major phase of the research presented here, scientific study is brought collaboratively to bear on questions related to choreographic practice. This lead to a better understanding of ways in which dancers and choreographers, as 'thinking bodies', interact with their design tools and each other in the context of creation work. In addition to this collection, outcomes of this research are traceable to other published papers and art works it has given rise to. Less easily measureable, but just as valuable, are the sustained relations between individuals and groups behind the 'community of practice' now recognised for its development of unique formats for bringing choreographic ideas and processes into contact, now and in the future, with both general audiences and other specialist practices

An Enactivist Model of Improvisational Dance

An Enactivist Model of Improvisational Danc

Interdisciplinary investigation into meditative Flow states and their roles in movement performance

This thesis focuses on the idea of Flow (Csikszentmihalyi, 1990), a psychological phenomenon that involves complete immersion and “optimal” experience. Whereas many existing studies focus on telic Flow – e.g. fixed goals, performance-oriented – the current thesis places focus on paratelic Flow – e.g. open goals, sensation-oriented – by exploring its role in dance and movement performance (Swann et al., 2018). In response to the call to reconceptualise and clarify the concept of Flow with regards to its various manifestations, the thesis draws from other related/similar concepts, such as pre-reflective experiences in dance (Fraleigh, 1987), the idea of no-mind in Zen practice (Yuasa, 1993), and the hypofrontality theory (Dietrich, 2004). Building on these concepts, the thesis examines how dancers might experience paratelic Flow within meditative movement episodes. Another focus of the thesis is to explore how Flow experiences might interact with a dancer’s physical performance. This line of inquiry draws inspiration from both philosophical and neurocognitive accounts, which identify a relationship between Flow and “optimal” movement performance. Given the paucity of cross-disciplinary dialogue, the thesis attempts to investigate the above topics through an interdisciplinary mixed-methods approach. Through an exploratory survey (Chapter 4) followed by a series of event-focused interviews (Chapter 5) – the thesis explores 1.) how dancers and movement practitioners might experience Flow during a single movement session, 2.) how these states might interact with the mover’s physical performance, and 3.) how viewers perceive and describe a mover in Flow. The survey results indicated that movers may experience Flow through various meditative episodes that arise through movement, including improvisational sessions, technique classes, and performative events. The interview study showed that Flow appears in a cyclical process involving five distinct stages – Entering, Opening, Riding, Ebbing, and Resetting – during which the movers’ physical performance show observable changes in quality. Notably, during the “peak” state of Flow (Riding), movements are described by observers as fluid, organic, and well-coordinated, which is consistent with existing literature. Through these findings, the thesis provides clarity to the role of Flow during dance and movement performance and demonstrates possible links between the dancer’s mental state and his/her physical performanc