Cyberdreams: visualizing music in extended reality

From the visual music films of the twentieth century to the Video Jockey (VJ) performances seen at the latest electronic dance music festivals, there is an extensive body of artistic work that seeks to visualize sound and music. The form that these visualizations take has been shaped significantly by the capabilities of available technologies; thus, we have seen a transition from paint to film; from hand-drawn animations to motion-graphics; and from analog to digital projection systems. In the twenty-first century, visualizations of music are now possible with extended reality (XR) technologies such as virtual reality (VR), augmented/mixed reality (AR/MR), and related forms of multi-projection environment such as fulldome. However, the successful design of visual music and VJ performances using XR technologies requires us to consider the compositional approaches that can be used by artists and designers. To investigate this area, this chapter will begin with an analysis of existing work that visualizes music using XR technologies. This will allow us to consider the spectrum of existing design approaches, and provide a commentary on the possibilities and limitations of the respective technologies. Following this, the chapter will provide an in-depth discussion of Weinel’s practice-led research, which extends from work exhibited at the Carbon Meets Silicon exhibitions held at Wrexham Glyndŵr University (2015, 2017), and includes AR paintings, VJ performances, and a VR application: Cyberdream VR. Through the discussion of these works, the chapter will demonstrate possible compositional principles for visualizing music across media ranging from paint to XR, enabling the realization of work that reinforces the conceptual meanings associated with music

Visualising rave music in Virtual Reality: symbolic and interactive approaches

In 2019 the virtual reality experience Cyberdream VR was presented at various events including the Event Two exhibition of computer art held at the Royal College of Art. This short demo for Oculus Gear VR provided a c.5 minute sonic journey, in which the user moves through a series of symbolic environments based on the futuristic techno-utopian or dystopian imagery of 90s rave fliers. These environments accompanied an original soundtrack of rave music and vaporwave, allowing users to enjoy the music whilst feeling as though they are inside synaesthetic virtual spaces related to the symbolic imagery of rave culture. This paper will discuss the subsequent development of this project, which is now being adapted for the Oculus Quest VR headset. Rajmil Fischman’s concept of ‘Music in the Holodeck’; the fictional ’holophonor’ instrument from the TV series Futurama; or the video game Rez Infinite, are among those that suggest paradigms for performing sound and visuals in VR. Drawing upon these ideas, the current iteration of Cyberdream allows the user to trigger ‘sound instruments’ with the Oculus Quest Touch controllers. This is conceived as a means through which the user can ‘paint with sound’ in 3D space, improvising with the music, whilst also generating synaesthetic imagery. The design of these sound instruments presents interesting challenges, in order for sound to be generated in correspondence with visual imagery, where these materials may complement both the music and spatial environments. Drawing on the author’s VJ work, at the macro level of compositional structure, the latest version of the project also seeks ways to provide a more continuous experience analogous to a DJ or VJ set, through blending of music tracks between scenes. This paper will discuss on-going work on this project, advancing the discourse regarding the visualisation of hardcore rave music in virtual reality

Virtual hallucinations: projects in VJing, virtual reality and cyberculture

This paper discusses a variety of the author’s artistic projects exploring altered states of consciousness and computer art. First, the paper will provide a brief overview of previous creative works, which include compositions of electroacoustic music, interactive visualisations, and visual music films. These previous works use the concept of altered states of consciousness as a compositional principle, as explored in the author’s book Inner Sound: Altered States of Consciousness in Electronic Music and Audio-Visual Media (OUP 2018). Following this, a variety of the author’s recent creative work produced from 2016–2019 will be discussed. These works include: a series of paintings that incorporate computer graphics animations when viewed in augmented reality; VJ performances constructed using direct animation on 8mm film, computer graphics animations generated from code and audio-reactive effects; and Cyberdream VR, a virtual reality experience. These interrelated projects continue to develop the author’s artistic investigations into altered states, while also referencing work such as demo scene videos; cyberdelic imagery of the type seen on fliers from the 1990s rave-era; and the recent Internet-borne subculture vaporwave, which recontextualises the aesthetics of 1980s and 1990s ambient corporate music and utopian computer graphics to construct surrealistic dystopias

States of being: art and identity in digital space and time

This one-day Symposium explored themes of personhood, modernity and digital art, bringing together speakers from a range of disciplines to consider technology, artistic practice and society. It seeks a renewed consideration of the role of art in illuminating human identity in a positive relation with technology, and its transformative effects upon space and time. The concerns for the role of art amidst the forces of a post-modern world are influenced by important legacies of the past, by which ideas about human identity and difference have been made meaningful in the relation of history and technology. In the frequently transient and conflicting forces of humanness and forces of modernity, the digital world of the arts emerges as a means by which new ideas of space and time can be considered, with new perspectives of human identity seen as states of being, towards the possibilities of experience, technology, individuality and society

Epidermal growth factor mediates detachment from and invasion through collagen I and Matrigel in Capan-1 pancreatic cancer cells

BACKGROUND: Pancreatic adenocarcinoma is a highly invasive neoplasm. Epidermal growth factor (EGF) and its receptor are over expressed in pancreatic cancer, and expression correlates with invasion and metastasis. We hypothesized that EGF receptor and integrin signalling pathways interact in mediating cellular adhesion and invasion in pancreatic cancer, and that invasiveness correlates temporally with detachment from extracellular matrix. METHODS: We tested this hypothesis by investigating the role of EGF in mediating adhesion to and invasion through collagen I and Matrigel in the metastatic pancreatic adenocarcinoma cell line Capan-1. Adhesion and invasion were measured using in vitro assays of fluorescently-labeled cells. Adhesion and invasion assays were also performed in the primary pancreatic adenocarcinoma cell line MIA PaCa-2. RESULTS: EGF inhibited adhesion to collagen I and Matrigel in Capan-1 cells. The loss of adhesion was reversed by AG825, an inhibitor of erbB2 receptor signalling and by wortmannin, a PI3K inhibitor, but not by the protein synthesis inhibitor cycloheximide. EGF stimulated invasion through collagen I and Matrigel at concentrations and time courses similar to those mediating detachment from these extracellular matrix components. Adhesion to collagen I was different in MIA PaCa-2 cells, with no significant change elicited following EGF treatment, whereas treatment with the EGF family member heregulin-alpha elicited a marked increase in adhesion. Invasion through Matrigel in response to EGF, however, was similar to that observed in Capan-1 cells. CONCLUSION: An inverse relationship exists between adhesion and invasion capabilities in Capan-1 cells but not in MIA PaCa-2 cells. EGF receptor signalling involving the erbB2 and PI3K pathways plays a role in mediating these events in Capan-1 cells

Communicating Auditory Impairments Using Electroacoustic Composition

Changes in human sensory perception can occur for a variety of reasons. In the case of distortions or transformations in the human auditory system, the aetiology may include factors such as medical conditions affecting cognition or physiology, interaction of the ears with mechanical waves, or stem from chemically induced sources, such the consumption of alcohol. These changes may be permanent, intermittent, or temporary. In order to communicate such effects to an audience in an accessible, and easily understood manner, a series of electroacoustic compositions were produced. This concept follows on from previous work on the theme of representing auditory hallucinations. Specifically, these compositions relate to auditory impairments that humans can experience due to tinnitus or through the consumption of alcohol. In the case of tinnitus, whilst much is known about the causes and symptoms, the experience of what it is like to live with tinnitus is less explored and those who have acquired the condition may often feel frustration when trying to convey the experience of ‘what it is like’ for them. In terms of impairment from alcohol consumption, whilst there is much hearsay, little research exists on the immediate and short-term effects of alcohol consumption on the human auditory system, despite over half of the UK population reported as consuming alcohol in 2017. The methodology employed to design these compositions draws upon scientific research findings, including experimental and explorative studies involving human participants, coupled with electroacoustic composition techniques. The pieces are typically constructed by mixing field recordings with synthesised materials and incorporating a range of temporal and frequency domain manipulations to the elements therein. In this way, the listener is able to experience the phenomenon in a recognisable context, where distortions of reality can be emulated to varying degrees. It is intended that these compositions can serve as easily accessible and understood examples of auditory impairments and that they might find utility in the communication of symptoms to those who have never experienced the underlying causes or conditions. This presents opportunities for pieces like these to be used in scenarios such as education and public health awareness campaigns

Augmenting virtual spaces: affective feedback in computer games

Computer games can be considered a form of art insomuch as they are critiqued, revered and collected for their aesthetics in addition to their ludic qualities. Perhaps most significantly, computer games incite a plethora of emotional responses in their players as a deliberate and defining mechanism. However, unlike other forms of traditional media and art, another key feature of games is their intrinsic interactivity, reliance upon technology and non-linearity. These traits make them particularly noteworthy if one wishes to consider how art forms might respond and adapt to their audience’s emotions. The field of affective computing has been developing for several decades and many of its applications have been in the analysis and modelling of emotional responses to forms of media, such as music and film. In gaming, recent developments have led to an increasing number of consumer-grade biofeedback devices which are available on the market, some of which are explicitly sold as ‘gaming controllers’, giving rise to greater opportunity for affective feedback to be incorporated. In this chapter, a review is provided of the affective gaming field. Specifically, it is proposed that these developments give rise to interesting opportunities whereby virtual environments can be augmented with player affective and contextual information. An overview is provided of affective computing fundamentals and their manifestation in developments relating specifically to games. The chapter considers the impact this biometric information has upon games players, in terms of their experience of the game and the social connections between competitors. A number of associated practical and technological challenges are highlighted along with areas for future research and development activities. It is hoped that by exploring these developments in gaming that the longer established forms of art and media might be inspired to further embrace the possibilities offered by utilising affective feedback

Alterations in integrin expression modulates invasion of pancreatic cancer cells

Background Factors mediating the invasion of pancreatic cancer cells through the extracellular matrix (ECM) are not fully understood. Methods In this study, sub-populations of the human pancreatic cancer cell line, MiaPaCa-2 were established which displayed differences in invasion, adhesion, anoikis, anchorage-independent growth and integrin expression. Results Clone #3 displayed higher invasion with less adhesion, while Clone #8 was less invasive with increased adhesion to ECM proteins compared to MiaPaCa-2. Clone #8 was more sensitive to anoikis than Clone #3 and MiaPaCa-2, and displayed low colony-forming efficiency in an anchorage-independent growth assay. Integrins beta 1, alpha 5 and alpha 6 were over-expressed in Clone #8. Using small interfering RNA (siRNA), integrin β1 knockdown in Clone #8 cells increased invasion through matrigel and fibronectin, increased motility, decreased adhesion and anoikis. Integrin alpha 5 and alpha 6 knockdown also resulted in increased motility, invasion through matrigel and decreased adhesion. Conclusion Our results suggest that altered expression of integrins interacting with different extracellular matrixes may play a significant role in suppressing the aggressive invasive phenotype. Analysis of these clonal populations of MiaPaCa-2 provides a model for investigations into the invasive properties of pancreatic carcinoma

The α2β1 integrin mediates the malignant phenotype on type I collagen in pancreatic cancer cell lines

Pancreatic cancer is characterised by a hallmark desmoplastic response that includes upregulated expression of the extracellular matrix, and type I collagen in particular. Recent studies indicate that pancreatic cancer cells stimulate type I collagen synthesis in adjacent stellate cells, and that this upregulated type I collagen expression promotes the malignant phenotype in tumour cells as defined by increased proliferation, resistance to chemically induced apoptosis, and increased tumorigenesis. The integrin specificity of this interaction between type I collagen and tumour cells was not identified, however. In the present study, we examined eight pancreatic cancer cell lines for adhesion, proliferation, and migration, on types I and IV collagen, fibronectin, laminin, and vitronectin, as well as integrin expression. Our results indicate, for the overwhelming majority of cell lines, that type I collagen promotes the strongest adhesion, proliferation, and migration relative to the other substrates tested. Utilising function-blocking monoclonal antibodies directed against particular integrin subunits in cell adhesion and migration inhibition assays, we demonstrate further that the malignant phenotype on type I collagen is mediated specifically by the α2β1 integrin. These results identify α2β1 integrin-mediated adhesion to type I collagen as a potential therapeutic target in the treatment of pancreatic cancer

T cell adhesion and cytolysis of pancreatic cancer cells: a role for E-cadherin in immunotherapy?

Pancreatic cancer is an aggressive and potent disease, which is largely resistant to conventional forms of treatment. However, the discovery of antigens associated with pancreatic cancer cells has recently suggested the possibility that immunotherapy might become a specific and effective therapeutic option. T cells within many epithelia, including those of the pancreas, are known to express the αEβ7-integrin adhesion molecule, CD103. The only characterised ligand for CD103 is E-cadherin, an epithelial adhesion molecule which exhibits reduced expression in pancreatic cancer. In our study, CD103 was found to be expressed only by activated T cells following exposure to tumour necrosis factor beta 1, a factor produced by many cancer cells. Significantly, the expression of this integrin was restricted mainly to class I major histocompatibility complex-restricted CD8+ T cells. The human pancreatic cancer cell line Panc-1 was transfected with human E-cadherin in order to generate E-cadherin negative (wild type) and positive (transfected) sub-lines. Using a sensitive flow cytometric adhesion assay it was found that the expression of both CD103 (on T cells) and E-cadherin (on cancer cells) was essential for efficient adhesion of activated T cells to pancreatic cancer cells. This adhesion process was inhibited by the addition of antibodies specific for CD103, thereby demonstrating the importance of the CD103→E-cadherin interaction for T-cell adhesion. Using a 51Cr-release cytotoxicity assay it was found that CD103 expressing T cells lysed E-cadherin expressing Panc-1 target cells following T cell receptor stimulation; addition of antibodies specific for CD103 significantly reduced this lysis. Furthermore, absence of either CD103 from the T cells or E-cadherin expression from the cancer cells resulted in a significant reduction in cancer cell lysis. Therefore, potentially antigenic pancreatic cancer cells could evade a local anti-cancer immune response in vivo as a consequence of their loss of E-cadherin expression; this phenotypic change may also favour metastasis by reducing homotypic adhesion between adjacent cancer cells. We conclude that effective immunotherapy is likely to require upregulation of E-cadherin expression by pancreatic cancer cells or the development of cytotoxic immune cells that are less dependent on this adhesion molecule for efficient effecter function