A Wireless Future: performance art, interaction and the brain-computer interfaces

Although the use of Brain-Computer Interfaces (BCIs) in the arts originates in the 1960s, there is a limited number of known applications in the context of real-time audio-visual and mixed-media performances and accordingly the knowledge base of this area has not been developed sufficiently. Among the reasons are the difficulties and the unknown parameters involved in the design and implementation of the BCIs. However today, with the dissemination of the new wireless devices, the field is rapidly growing and changing. In this frame, we examine a selection of representative works and artists, in comparison to the current scientific evidence. We identify important performative and neuroscientific aspects, issues and challenges. A model of possible interactions between the performers and the audience is discussed and future trends regarding liveness and interconnectivity are suggested

A Wireless Future: performance art, interaction and the brain-computer interfaces

Although the use of Brain-Computer Interfaces (BCIs) in the arts originates in the 1960s, there is a limited number of known applications in the context of real-time audio-visual and mixed-media performances and accordingly the knowledge base of this area has not been developed sufficiently. Among the reasons are the difficulties and the unknown parameters involved in the design and implementation of the BCIs. However today, with the dissemination of the new wireless devices, the field is rapidly growing and changing. In this frame, we examine a selection of representative works and artists, in comparison to the current scientific evidence. We identify important performative and neuroscientific aspects, issues and challenges. A model of possible interactions between the performers and the audience is discussed and future trends regarding liveness and interconnectivity are suggested

Spectral estimation for random processes with stationary increments

In studying a stationary random process on R, the covariance function is commonlyused to characterize the second-order spatial dependency. Through the inversionof Fourier transformation, its corresponding spectral density has been widely usedto describe the periodical components and frequencies. When the process is with stationarydth increments, that is, when the resulting process after undertaken dth orderof di erences is stationary, the notion of structure function is put forward. Throughthe inversion formula, the spectrum can be represented by the structure function.In this dissertation, we rst investigate the properties of the proposed Method ofMoments structure function estimator, through which we obtain the spectral densityfunction estimation of the underlying process. In particular, when the process is intrinsicallystationary, which is also a process is with stationary increments of order 1,we derive the spectral density functions for commonly used variogram models. Furthermore,our proposed estimation method is applied to estimate the spectral densityof power variogram models. All of the above results are supplemented via simulationsand a real data analysis. Our results show that the proposed estimation method performswell in recovering the true spectral density function on various processes withstationary increments we considered.[This abstract has been edited to remove characters that will not display in this system. Please see the PDF for the full abstract.]]]> 2018 Spectral theory (Mathematics) Estimation theory Stochastic processes Stationary processes English http://libres.uncg.edu/ir/uncg/f/Chen_uncg_0154D_12424.pdf oai:libres.uncg.edu/23131 2019-03-04T14:06:04Z UNCG The effects of instrumental music instruction on the neurophysiological responses and adaptive behaviors of children with autism spectrum disorder Chinn Cannon, Michelle L. NC DOCKS at The University of North Carolina at Greensboro <![CDATA[Autism spectrum disorder, also referred to as autism, is a complex and heterogeneous neurodevelopmental condition characterized by deficits in social communication, delayed or absent language development, and restricted or repetitive behaviors. Finding an appropriate, effective, and affordable intervention that targets these differences may increase access of children with autism to treatment that improves their quality of life, independence, and productivity, while reducing lifetime care costs. The premise of this exploratory study was that music instruction may serve as an appropriate, effective, and affordable intervention for children with autism. Previous researchers noted that children with autism have both an affinity for and ability in music, while neuroscientists demonstrated increased cortical growth and neural network responses among musicians. At the onset of the current study, no published research studies were found that explicitly examined effects of musical training on both neural activity and adaptive behaviors of children with autism. The purpose of this exploratory research study was to investigate the effects of instrumental music instruction on neurophysiological responses and adaptive behaviors of children with autism. Fourteen children with autism participated in the current study. During a 20-week period, a control group (n = 7) received 30 minutes of non-music intervention per week, and an experimental group (n = 7) received 30 minutes of music intervention (i.e., violin instruction) per week. Before and after the intervention period, neurophysiological and adaptive behavioral data were collected from control and experimental groups. The 14 participants of the study were assigned randomly to either the control (i.e., non-music intervention group), or the experimental (i.e., music intervention group). Eleven children completed the behavioral segment of this study, five in the control group and six in the experimental group. As compared to the non-music intervention group, experimental participants displayed significant gains in Expressive Communication (p =.018). Increases in Interpersonal Socialization by the music intervention group also approached significance (p = .057). The researcher found a moderately large effect size for Expressive Communication (r = .694), and for Interpersonal Socialization (r = .589), accounting for approximately 40% and 35% of the variances of the two adaptive behaviors before and after music intervention, respectively. Eight children completed the neurophysiological segment of this study, three in the control group and five in the experimental group. Results revealed several trends in the differences between the control and experimental intervention groups' postintervention neurophysiological responses. While changes were not observed among the non-music group's pre- and post-intervention cortical activity, changes were observed among the experimental group's cortical activation in areas associated with social and language learning. These findings supported the premise that instrumental music study may serve as an appropriate, effective, and affordable intervention, targeting the hallmark behaviors of autism and potentially associated cortical areas

Differences in cortical activation with live music compared to recorded music: an fNIRS study

2020 Spring.Includes bibliographical references.The purpose of this study was to compare and assess the neural activations during live music and recorded music engagement in neurotypical adults aged eighteen to sixty years old within a social context. The research questions sought to answer if cortical activations in areas of the brain involved with social interaction would be different in the live music conditions compared to the recorded music conditions and if blood oxygenation levels across the entire cortical surface would be different in any area across the four conditions. This study was a within-subjects quasi-experimental design where each of the 32 recruited participants were exposed to all four conditions while mirroring the CR (clinician-researcher) in a tapping task. The four conditions were: recorded sung, recorded spoken, live sung, and live spoken. Participants were exposed to the four conditions as well as a rest condition in pseudo-randomized order. Each participant underwent five trials of each condition using a block design. Cortical activation was measured using functional near-infrared spectroscopy (fNIRS). A total of 27 participants were included in the analyses. Imaging results revealed significant differences in inferred cortical activation during live stimuli compared to recorded stimuli, live music compared to recorded music, music stimuli compared to non-music stimuli, live music compared to all other conditions, and live spoken stimuli compared to recorded spoken stimuli in brain regions of interest and globally. Results support the possibility that live music may have a greater effect than recorded music in regions of the brain that process social responses. Future research could better illuminate the comparisons of neural activations between live and recorded auditory stimuli

“A place of images, plus sounds” (Chion, 1994)

The performance is a live experimentation using an interface that facilitates real-time audiovisual collage. Based on an original concept for audio-collage, Unrealtime (MAAP 2017), this expansion into the sphere of live visual editing brings into question many practices for the improviser. Through a series of Max-based audiovisual improvisations, the performance explores how the act of simultaneously improvising over two expressive mediums challenges the perceptual expectations of both performer and audience. The materials used range in abstraction, including both improvised and found audiovisual content. Aiming to probe Michel Chion’s assertion that “there is no image track and no soundtrack in the cinema, but a place of images, plus sounds” (Chion, 1994), the live experimentations will attempt to utilise audiovisual consonance and dissonance in different combinations to produce on-the-fly emergences of abstract or arbitrary narratives. What kinds of conceptual and perceptual hierarchies, such as the perceived dominance of image over sound, emerge or remain suppressed? Can a human computer interface assist in the translation of musical aesthetics, such as non-idiomatic improvisation, into the realm of a visual aesthetic that evolves in real time

Insomnia : the affordance of hybrid media in visualising a sleep disorder

The integration of visual and numerical abstraction in contemporary audio-visual communication has become increasingly prevalent. This increase reflects the evolution of computational machines from simple data processors. Computation and interface have augmented our senses and converged algorithmic logic with cultural techniques to form hybrid channels of communication. These channels are fluid and mutable, allowing creatives to explore and disseminate knowledge through iterative media practice. Insomnia is an auto-ethnographic case study that examines the affordance of merging Brain-Computer Interfaces (BCIs) and node- based programming software (TouchDesigner), as a hybrid media system (McMullan, 2020). As a system, Insomnia compiles my archived brain activity data and processes it through a custom designed generative visualisation interface. Documenting and ‘processing’ a sleep disorder is filtered through key concepts of media archaeology, cultural techniques, and practice-led research allowing Insomnia to inform discussion of the affordance of hybrid media. Insomnia is presented as a virtual exhibition with a supporting exegesis. The methodology and outcomes of the project form a framework that bridges science communication and creative practice and points to continued development for interactive installation design

Exploring inner worlds: where cognitive psychology may take us

Tim Smith researches visual cognition through a variety of techniques that capture what subjects do when they watch naturalistic visual scenes. Although much research into scene perception uses static visual scenes, he is interested in how we process dynamic visual scenes, including feature films. Using techniques such as tracking the eye movements of viewers watching film sequences in combination with behavioural probes during and after viewing, he investigates the cognitive processes occurring during film viewing. His research has contributed to a computational model of fixation durations in scene viewing (Nuthmann, Smith, Engbert, & Henderson, 2010), and he has proposed an Attentional Theory of Cinematic Continuity (Smith, 2012), which has attracted the interest of ‘cognitivist’ film scholars, leading to his involvement in a number of presentations and projects that use empirical testing and modelling to explain what happens when we watch film sequences. I too am interested in the long history of attempts to understand the perceptual and cognitive processes involved in film viewing, which stretch from pioneering work by Munsterberg and Rank in the 1910s, through the renewed interest of Russian psychologists in the 1920s, following Pavlov’s classic studies in conditioning, and later of the Gestalt school, up to present-day work such as that of James Cutting (starting from the ‘psychophysics’ tradition) and Uri Hasson (‘neurocinematics’). In this exchange, written as a dialogue over several months, I invite Tim Smith to summarise some of his own work and to speculate on what value it may have for non-scientists interested in understanding how and why we perceive film as we do – and how this may be changin