694 research outputs found
Imagining & Sensing: Understanding and Extending the Vocalist-Voice Relationship Through Biosignal Feedback
The voice is body and instrument. Third-person interpretation of the voice by listeners, vocal teachers, and digital agents is centred largely around audio feedback. For a vocalist, physical feedback from within the body provides an additional interaction. The vocalist’s understanding of their multi-sensory experiences is through tacit knowledge of the body. This knowledge is difficult to articulate, yet awareness and control of the body are innate. In the ever-increasing emergence of technology which quantifies or interprets physiological processes, we must remain conscious also of embodiment and human perception of these processes. Focusing on the vocalist-voice relationship, this thesis expands knowledge of human interaction and how technology influences our perception of our bodies. To unite these different perspectives in the vocal context, I draw on mixed methods from cog- nitive science, psychology, music information retrieval, and interactive system design. Objective methods such as vocal audio analysis provide a third-person observation. Subjective practices such as micro-phenomenology capture the experiential, first-person perspectives of the vocalists them- selves. Quantitative-qualitative blend provides details not only on novel interaction, but also an understanding of how technology influences existing understanding of the body. I worked with vocalists to understand how they use their voice through abstract representations, use mental imagery to adapt to altered auditory feedback, and teach fundamental practice to others. Vocalists use multi-modal imagery, for instance understanding physical sensations through auditory sensations. The understanding of the voice exists in a pre-linguistic representation which draws on embodied knowledge and lived experience from outside contexts. I developed a novel vocal interaction method which uses measurement of laryngeal muscular activations through surface electromyography. Biofeedback was presented to vocalists through soni- fication. Acting as an indicator of vocal activity for both conscious and unconscious gestures, this feedback allowed vocalists to explore their movement through sound. This formed new perceptions but also questioned existing understanding of the body. The thesis also uncovers ways in which vocalists are in control and controlled by, work with and against their bodies, and feel as a single entity at times and totally separate entities at others. I conclude this thesis by demonstrating a nuanced account of human interaction and perception of the body through vocal practice, as an example of how technological intervention enables exploration and influence over embodied understanding. This further highlights the need for understanding of the human experience in embodied interaction, rather than solely on digital interpretation, when introducing technology into these relationships
Transformative interventions. An ecological-enactive approach to art practices
Starting from an ecological-enactive approach to human cognition (Rietveld, Kiverstein 2014) I have articulated a series of transformative interventions whose purpose is to explore how art practices can reorganize our form of life (Noë, 2015; Rietveld, 2019). To do this, I discuss how a plethora of heterogeneous tools traceable in the performing arts, such as masks, puppets, and hybrid costumes, can help us, through what I call monstrous practices, to explore imaginative dimensions that our own bodies "cannot afford." This is the core of the transformative chain that I will define monster-monstrous-Monster: we feed imaginative “monsters” to become “monstrous”– that is, to pool and cross-fertilize our abilities – to confront the "Monsters" in our lives.
My main interest is in analyzing how it is possible to create or collect new affordances so as to transfigure one's repertoire of possibilities and transform a shared practice.
Each transformative intervention is not only defined through written words but is also developed through unorthodox sociomaterial invitations, usually not used in philosophical practice: storyboards, visual ethnographies, performance projects, and installations, which I will define more properly through an enriched notion of real-life thinking model (Rietveld; RAAAF)
Recommended from our members
Revenge of the Nerds: Tech Masculinity and Digital Hegemony
Revenge of the Nerds provides a cultural history of the evolution of white nerd masculinities in American culture through interpretations of a wide variety of texts and representations using the methods of literary studies and American studies. The dissertation is organized around four overlapping stages of nerd masculinity based on changes in technology and their effects on culture, as well as white male nerds’ efforts to remain culturally relevant and gain the benefits of being close to hegemonic masculinity. The four nerd types are the computer nerd, the gamer, the gatekeeper nerd, and the maladaptive nerd which reflect the following movement through chronological development: the introduction of computers into the mainstream of American culture, the early years of video gaming culture, the nerds’ cultural power and influence as early adopters of the Internet and the white male nerds’ participation in a backlash against factors like social media which made gaming and Internet cultures more mainstream and diverse. Throughout these changes, white male nerds chased a promise they felt was implied by the adoption of “revenge of the nerds” as an American pop cultural myth, but the nerd identity was constructed in such a way that it would never be perceived as fully congruent with hegemonic American masculinity. Therefore, the identity was based on an insecurity about masculinity, which nerds tried to assuage by being especially aggressive in their pursuit and enforcement of American norms of masculinity
Unsupervised landmark discovery via self-training correspondence
Object parts, also known as landmarks, convey information about an object’s shape and spatial configuration in 3D space, especially for deformable objects. The goal of landmark detection is to have a model that, for a particular object instance, can estimate the locations of its parts. Research in this field is mainly driven by supervised approaches, where a sufficient amount of human-annotated data is available. As annotating landmarks for all objects is impractical, this thesis focuses on learning landmark detectors without supervision. Despite good performance on limited scenarios (objects showcasing minor rigid deformation), unsupervised landmark discovery mostly remains an open problem. Existing work fails to capture semantic landmarks, i.e. points similar to the ones assigned by human annotators and may not generalise well to highly articulated objects like the human body, complicated backgrounds or large viewpoint variations.
In this thesis, we propose a novel self-training framework for the discovery of unsupervised landmarks. Contrary to existing methods that build on auxiliary tasks such as image generation or equivariance, we depart from generic keypoints and train a landmark detector and descriptor to improve itself, tuning the keypoints into distinctive landmarks. We propose an iterative algorithm that alternates between producing new pseudo-labels through feature clustering and learning distinctive features for each pseudo-class through contrastive learning. Our detector can discover highly semantic landmarks, that are more flexible in terms of capturing large viewpoint changes and out-of-plane rotations (3D rotations). New state-of-the-art performance is achieved in multiple challenging datasets
Generating Holistic 3D Human Motion from Speech
This work addresses the problem of generating 3D holistic body motions from
human speech. Given a speech recording, we synthesize sequences of 3D body
poses, hand gestures, and facial expressions that are realistic and diverse. To
achieve this, we first build a high-quality dataset of 3D holistic body meshes
with synchronous speech. We then define a novel speech-to-motion generation
framework in which the face, body, and hands are modeled separately. The
separated modeling stems from the fact that face articulation strongly
correlates with human speech, while body poses and hand gestures are less
correlated. Specifically, we employ an autoencoder for face motions, and a
compositional vector-quantized variational autoencoder (VQ-VAE) for the body
and hand motions. The compositional VQ-VAE is key to generating diverse
results. Additionally, we propose a cross-conditional autoregressive model that
generates body poses and hand gestures, leading to coherent and realistic
motions. Extensive experiments and user studies demonstrate that our proposed
approach achieves state-of-the-art performance both qualitatively and
quantitatively. Our novel dataset and code will be released for research
purposes at https://talkshow.is.tue.mpg.de.Comment: Project Webpage: https://talkshow.is.tue.mpg.de; CVPR202
- …