Responsive Systems and Surfaces in Architecture - Design and Challenges

In the last two decades, the impact of digital technologies has dramatically changed the way of thinking in architecture. Following the overall development of digital media and computation, the process of generating forms has become more automated, with the use of algorithms and scripts. Moreover, computational design approach created new groundwork for the new, time-depended systems in architecture - responsive facades and structures. Responsive systems react to the surrounding by changing its surface in order to adapt the architectural form according to predefined influences. An example of such design approach are media facades, artistic based installations etc.Challenges are mainly oriented towards forming a user-architecture connection. Such bonds are formed between sensors and actuators, via programming board and language. Adaptive qualities of such systems are based on sustainability principles as well, utilizing alternative energy sources in situ. In this paper, design and challenges of responsive systems are analyzed, evaluated and applied on a case study in particular.Simultaneous use of above approaches is applied in the design of the structure that is located in Novi Sad's central core. The structure is optimized to use the maximum input of solar energy and adapt to it, while limiting the spatial openness of public squares and increasing human interaction. Resulting design is presented in this paper. Further exploration of responsive generative systems can contribute to financial benefit and better quality of given architectural solutions in the near future

Disturbance and plausibility in a virtual rock concert: a pilot study

We present methods used to produce and study a first version of an attempt to reconstruct a 1983 live rock concert in virtual reality. An approximately 10 minute performance by the rock band Dire Straits was rendered in virtual reality, based on the use of computer vision techniques to extract the appearance and movements of the band, and crowd simulation for the audience. An online pilot study was conducted where participants experienced the scenario and freely wrote about their experience. The documents produced were analyzed using sentiment analysis, and groups of responses with similar sentiment scores were found and compared. The results showed that some participants were disturbed not by the band performance but by the accompanying virtual audience that surrounded them. The results point to a profound level of plausibility of the experience, though not in the way that the authors expected. The findings add to our understanding of plausibility of virtual environments.This work is funded by the European Research Council (ERC) Advanced Grant Moments in Time in Immersive Virtual Environments (MoTIVE) #742989.Peer ReviewedPostprint (author's final draft

Conductor Follower: Controlling sample-based synthesis with expressive gestural input

Vuosien mittaan on kehitetty useita järjestelmiä, jotka seuraavat kapellimestarin liikkeitä, ja soittavat musiikkia keräämänsä tiedon perusteella. Koska orkesterinjohto on pohjimmiltaan kommunikaatiota kapellimestarin ja muusikkojen välillä - sisältäen muutakin kuin eleiden kautta kommunikointia - on selvää, että ongelman täydellinen hallinta on hyvin monimutkaista. Tietokonepeliteollisuuden viimeaikaisen kehityksen myötä kuluttajatason liikesensoreita on helposti saatavilla, ja niille on helppo kehittää ohjelmistoja. Siksi ne ovat hyvin varteenotettava vaihtoehto kapellimestariseuraajan syötteen tuottamiseksi. Tässä tutkimuksessa suunniteltiin ja toteutettiin kyseisenlaista liikesensoria, sekä korkealaatuista sample -pohjaista syntetisaattoria käyttävä kapellimestariseuraaja. Nämä komponentit antavat hyvän perustan luonnolliselle käyttäjäkokemukselle, eikä niitä ole aikaisemmin käytetty vastaavanlaisissa sovelluksissa. Työn tulos on VST-liitännäinen, joka käyttää MIDI-tiedostoja syötteenään. Oikeilla asetuksilla liitännäistä on mahdollista käyttää lähes minkä tahansa samplepohjaisen syntetisaattorin kanssa. Järjestelmän toimintaa arvioitiin ammattikapellimestarin avustuksella, ja vaikka järjestelmän ekspressiiviset ominaisuudet ovat jokseenkin rajoittuneet, sopivilla asetuksilla sen temponseuraamisominaisuuksien todettiin muistuttavan oikeaa orkesteria.Over the years, several systems that follow conductor movement and play back a score accordingly have been implemented. Since conducting is essentially communication between the conductor and the musicians - including more than only gestural communication - it is clear that the problem is very complex to master completely. Recent developments in the computer gaming industry have made consumer grade motion sensing devices commonly available and easy to use in software development, making such a device an intriguing choice for using as the input of a conductor follower system. A system using such a device together with a high quality sample based synthesis engine was designed and implemented. These building blocks provide a solid foundation for a natural user experience, and have not been previously used in a similar way. The result is a VST plugin which uses MIDI scores as input, and may be configured to work with any sample based synthesis engine supporting patch switching via keyswitch events. The system was evaluated by a professional conductor, and while the expressive features of the system are somewhat limited, the tempo following functionality was concluded to resemble a real orchestra when adjusted correctly

AXMEDIS 2007 Conference Proceedings

The AXMEDIS International Conference series has been established since 2005 and is focused on the research, developments and applications in the cross-media domain, exploring innovative technologies to meet the challenges of the sector. AXMEDIS2007 deals with all subjects and topics related to cross-media and digital-media content production, processing, management, standards, representation, sharing, interoperability, protection and rights management. It addresses the latest developments and future trends of the technologies and their applications, their impact and exploitation within academic, business and industrial communities

Music conducting pedagogy and technology : a document analysis on best practices

This document analysis was designed to investigate pedagogical practices of music conducting teachers in conjunction with research of technologists on the use of various technologies as teaching tools. I sought to discern how conducting teachers and pedagogues are applying recent technological advancements into their teaching strategies. I also sought to understand what paths research is taking about the use of software, hardware, and computer systems applied to the teaching of music conducting technique. This dissertation was guided by four main research questions: (1) How has technology been used to aid in the teaching of conducting? (2) What is the role of technology in the context of conducting pedagogy? (3) Given that conducting is a performative act, how can it be developed through technological means? (4) What technological possibilities exist in the teaching of music conducting technique? Data were collected through music conducting syllabi, conducting textbooks, and research articles. Documents were selected through purposive sampling procedures. Analysis of documents through the constant comparative approach identified emerging themes and differences across the three types of documents. Based on a synthesis of information, I discussed implications for conducting pedagogy and made suggestions for conducting educators.Includes bibliographical references

Virtual orchestration: a film composer's creative practice

The advent of digital technologies has led to a major change in the process of film music composition; consequent developments in music technology have forced film composers to adapt to this change. Technological innovations such as digital audio workstations (DAWs) and virtual musical instruments have made possible the creation of virtual orchestras that are technologically capable of simulating the sound and behaviour of a traditional acoustic orchestra. This has had an effect on film music production and on the creative process of the professional film composer in a way that today, creating orchestral simulations or 'mock-ups' that imitate live orchestras (or smaller ensembles) has become a requirement in the film industry and thus an essential part of the film-scoring process. In the context of contemporary film music production, this thesis investigates how orchestral simulations are composed and created using computer music technology and virtual sample-based instruments. In asking 'how', the focus is on the film composer's activities and thought processes during this creative cycle, along with the nature of the interactive relationship between composer and music materials. This study aims to show the complexity of the film composer's creative practice and to advance understanding of how the use of computer music technology and orchestral sample libraries is influencing the compositional process and compositional outcome. To address these questions, a qualitative multiple case study methodology approach was chosen that included examination of the practice of seven professional film composers working in the field of feature film as the primary valid source of data. The exploration involved semi-structured interviews with composers, observations and analysis of their studio practice and inspection of their compositional tools. Taken as a whole, the evidence provided by this study is that the process of creating orchestral simulations is a process of film music composition during which professional film composers are creating orchestral sounds through the use of computers, digital sequencing, samplers and sample-based virtual acoustic instruments for the realisation of musical works. It is a process of using and manipulating recorded samples of real acoustic instruments to generate an expressive and convincing musical performance through sample-based orchestral simulation. A characteristic of this compositional practice is that it is a continuous process that proceeds in stages over time where all procedures can be applied repeatedly between stages. The process of creating orchestral simulations for the purpose of the film score is a multifaceted compositional activity involving a complex set of relationships among different compositional states of mind and compositional activities in which film composers experience music and interact with musical materials and media in various ways. This creative activity is a process involving a single person and a mixture of various compositional tools, the composer's skills and abilities brought into existence through a creative process that requires a thorough blend of art and craft to be demonstrated at all times

Semantics for virtual humans

Population of Virtual Worlds with Virtual Humans is increasing rapidly by people who want to create a virtual life parallel to the real one (i.e. Second Life). The evolution of technology is smoothly providing the necessary elements to increase realism within these virtual worlds by creating believable Virtual Humans. However, creating the amount of resources needed to succeed this believability is a difficult task, mainly because of the complexity of the creation process of Virtual Humans. Even though there are many existing available resources, their reusability is difficult because there is not enough information provided to evaluate if a model contains the desired characteristics to be reused. Additionally, the knowledge involved in the creation of Virtual Humans is not well known, nor well disseminated. There are several different creation techniques, different software components, and several processes to carry out before having a Virtual Human capable of populating a virtual environment. The creation of Virtual Humans involves: a geometrical representation with an internal control structure, the motion synthesis with different animation techniques, higher level controllers and descriptors to simulate human-like behavior such individuality, cognition, interaction capabilities, etc. All these processes require the expertise from different fields of knowledge such as mathematics, artificial intelligence, computer graphics, design, etc. Furthermore, there is neither common framework nor common understanding of how elements involved in the creation, development, and interaction of Virtual Humans features are done. Therefore, there is a need for describing (1) existing resources, (2) Virtual Human's composition and features, (3) a creation pipeline and (4) the different levels/fields of knowledge comprehended. This thesis presents an explicit representation of the Virtual Humans and their features to provide a conceptual framework that will interest to all people involved in the creation and development of these characters. This dissertation focuses in a semantic description of Virtual Humans. The creation of a semantic description involves gathering related knowledge, agreement among experts in the definition of concepts, validation of the ontology design, etc. In this dissertation all these procedures are presented, and an Ontology for Virtual Humans is described in detail together with the validations that conducted to the resulted ontology. The goal of creating such ontology is to promote reusability of existing resources; to create a shared knowledge of the creation and composition of Virtual Humans; and to support new research of the fields involved in the development of believable Virtual Humans and virtual environments. Finally, this thesis presents several developments that aim to demonstrate the ontology usability and reusability. These developments serve particularly to support the research on specialized knowledge of Virtual Humans, the population of virtual environments, and improve the believability of these characters

Conducting a virtual orchestra

The virtual orchestra is a multimodal system architecture based on complex 3D sound environment processing and a compact user interface. The system architecture supports high-performance sound content for VR (virtual reality) applications, and the user controls the performance with a magnetically tracked handheld device. The objective of this work is to give music amateurs the opportunity to conduct a group of musicians and produce a new kind of multimedia experience. To explore this idea, the authors developed a semi-immersive virtual reality system, based on a large projection screen, designed to create an entertaining multimedia experience for nonexpert users. They decided to implement a music performance simulator, a VR system where several virtual characters play different musical instruments and adapt their performance according to the user's instruction