Understanding Optical Music Recognition

For over 50 years, researchers have been trying to teach computers to read music notation, referred to as Optical Music Recognition (OMR). However, this field is still difficult to access for new researchers, especially those without a significant musical background: Few introductory materials are available, and, furthermore, the field has struggled with defining itself and building a shared terminology. In this work, we address these shortcomings by (1) providing a robust definition of OMR and its relationship to related fields, (2) analyzing how OMR inverts the music encoding process to recover the musical notation and the musical semantics from documents, and (3) proposing a taxonomy of OMR, with most notably a novel taxonomy of applications. Additionally, we discuss how deep learning affects modern OMR research, as opposed to the traditional pipeline. Based on this work, the reader should be able to attain a basic understanding of OMR: its objectives, its inherent structure, its relationship to other fields, the state of the art, and the research opportunities it affords

Safe and Sound: Proceedings of the 27th Annual International Conference on Auditory Display

Complete proceedings of the 27th International Conference on Auditory Display (ICAD2022), June 24-27. Online virtual conference

Interactive maps for visually impaired people : design, usability and spatial cognition

Connaître la géographie de son environnement urbain est un enjeu important pour les personnes déficientes visuelles. Des cartes tactiles en relief sont généralement utilisées mais elles présentent des limitations importantes (nombre limité d'informations, recours à une légende braille). Les nouvelles technologies permettent d'envisager des solutions innovantes. Nous avons conçu et développé une carte interactive accessible, en suivant un processus de conception participative. Cette carte est basée sur un dispositif multi-touch, une carte tactile en relief et une sortie sonore. Ce dispositif permet au sujet de recueillir des informations en double-cliquant sur certains objets de la carte. Nous avons démontré expérimentalement que ce prototype était plus efficace et plus satisfaisant pour des utilisateurs déficients visuels qu'une carte tactile simple. Nous avons également exploré et testé différents types d'interactions avancées accessibles pour explorer la carte. Cette thèse démontre l'importance des cartes tactiles interactives pour les déficients visuels et leur cognition spatiale.Knowing the geography of an urban environment is crucial for visually impaired people. Tactile relief maps are generally used, but they retain significant limitations (limited amount of information, use of braille legend, etc.). Recent technological progress allows the development of innovative solutions which overcome these limitations. In this thesis, we present the design of an accessible interactive map through a participatory design process. This map is composed by a multi-touch screen with tactile map overlay and speech output. It provides auditory information when tapping on map elements. We have demonstrated in an experiment that our prototype was more effective and satisfactory for visually impaired users than a simple raised-line map. We also explored and tested different types of advanced non-visual interaction for exploring the map. This thesis demonstrates the importance of interactive tactile maps for visually impaired people and their spatial cognition

A study of the integration of technology in the school arts classroom

This study explores the use of Information and Communications Technology (ICT) and Educational Technology (ET) as an educational resource in the school Arts classroom. Many teachers lack the qualifications to teach the Performing Arts (PA) of the Curriculum and Policy Statement (CAPS). Therefore, teachers rely on curriculum documentation and textbooks to help guide planning of lessons and the execution thereof. Technology as a resource tool can assist teachers as they incorporate an overwhelming amount of content/concepts in lessons. In this way they could improve classroom practice in the PA disciplines. With the emergence of Operation Phakisa: ICT in Education (Department of Education [DoE] 2015), the researcher acknowledges that the integration of ET is unavoidable. Moreover, the research highlights the importance of engaging with the rising hegemony of ICT as the defining characteristic of the information society. Literature supporting the inclusivity of media and media-related resources in education are discussed. In addition, the literature review focuses on a wide variety of ET and concludes with the implementation of ET in a South African context. Teachers from the Foundation Phase (FP), Intermediate Phase (IP) and Senior Phase (SP), namely grade R – 7, were invited to participate in the study where they were observed during contact time drawing on technology to aid teaching pedagogy. The research design involved a qualitative inquiry with aspects of crystallization where the data was collected through observations, interviews and teacher’s self-assessment. The findings suggest that teachers received little or no training in one or more PA disciplines. The integration of technology in the teaching pedagogy led to an enhanced learning environment where the teacher could actively engage learners in meaningful activities. The findings furthermore revealed that the use of technology was not uniform and did not lead to neat conclusion. Instead the researcher found that a great deal of divergence in the use of technology. This divergence was found across art disciplines and the use of technology also differed among teachers. Finally, individual teachers also differed in their use of technology across the art disciplines that they taught. The learners benefited from the use of technology insofar as they displayed a better understanding of the concepts in subsequent lessons. Furthermore, by the end of the study, teachers had begun to implement the new teaching style in learning areas other than the PA

Didactical perspectives of aural training

Thesis (PhD) -- Stellenbosch University, 1993.ENGLISH ABSTRACT: The purpose of this research endeavour is to develop an understanding of the general state of Aural Training as presented in Aural Training literature and at a tertiary level. Based on this understanding, recommendations for a theory towards Aural Training are furthermore made. The investigation consists of three main parts. In the first two parts, an in-depth theoretical study, exploring aspects such as the rationale behind Aural Training, teaching ideologies, contents and target group and methodological approaches found in published and unpublished sources, as well as a practical questionnaire-based survey, investigating selected Aural Training aspects as presented on a tertiary level in the Republic of South Africa (RSA), the Federal Republic of Germany (FRG), and the United States of America (USA) are presented. A model of an integrated Aural Training approach for children is proposed in the third part. Results from both the theoretical and practical surveys indicate that: (a) the rationale behind Aural Training has not been thoroughly thought through; (b) Aural Training lecturers with sound pedagogical credentials are needed; (c) prospective students are not prepared for their tertiary Aural Training courses; (d) the Aural Training curricula at most tertiary music institutions do not meet lecturers' expectations; (e) more instruction time is needed for classes scheduled in both individual and group tuition; (f) Aural Training literature and teaching methods are still influenced by the Behaviouristic school of psychological thinking, with its emphasis on drill and practice; (g) methods such as Sight Singing and Dictation are more frequently applied than others, possibly because they are readily quantifiable; (h) most emphasis is placed on tonal music; (i) research results in which holistic approaches to Aural Training have been recommended since the early 1980s are not reflected in the majority of Aural Training work-/textbooks and teaching methods; (j) commercially available software is often chosen because of its availability and not because of being based on sound pedagogical principles; (k) there is a growing interest amongst lecturers to re-examine the goals of Aural Training and to apply music psychological principles. Because of the already mentioned problem of improperly prepared prospective music students" and the scarcity of Aural Training approaches to develop children's musical consciousnesses, a model for integrating' Aural Training into the instrumental/vocal music lesson through composition is proposed. This model is based entirely on (a) the conviction that most effective learning takes place through active involvement and creation, and (b) music psychological principles such as Gestalt perception, the developmental theories of Piaget, Gardner and Swanwick-Tillman, and the musical thinking process theories of Prince and Webster. Through the application of this model, Aural Training can be approached by dealing with wholes within a musical context. Structural and perceptual thinking, as well as the mental principles of exploring, applying, problem-solving and critical reasoning can also be developed. It offers a comprehensive approach to learning written theory skills, and the opportunity to apply all other Aural Training methods. Examples of children's compositions are presented to support this model. It is finally recommended that: (a) Aural Training lecturers should constantly re-examine their objectives, contents, teaching philosophies and methods applied; (b) advanced courses in Aural Training should be offered at all tertiary music institutions in order to meet the need for adequately trained lecturers; (c) a compulsory course on the didactics of Aural Training should be offered for all music students; (d) more instruction time should be made available for Aural Training; (e) Aural Training should be treated in its own right in grading policies; (f) holistic approaches to both classroom-based and programmed Aural Training should replace drill and practice; (g) music from all style periods should be included; (h) a wide spectrum of Aural Training methods should be incorporated and not mainly Sight Singing and Dictation; (i) lecturers should inform themselves about recent research results and try to incorporate these into their Aural Training curricula and syllabi; g) Work-/textbooks and computer software based on holistic and comprehensive approaches to Aural Training should be released; (k) Aural Training should be incorporated from the very first instrumental/vocal lesson by applying an integrated, holistic approach through composition.AFRIKAANSE OPSOMMING: Die doel van hierdie navorsing is om 'n ondersoek te loods na die algemene stand van gehooropleiding soos daargestel in musiekliteratuur, en soos aangebied aan tersiere musiekinstansies: 'n Model vir gelntegreerde gehooropleiding word ten slotte voorgestel aan hand van hierdie inligting. Die inhoud van die ondersoek is in drie hoofdele georganiseer. In die eerste deel word In teoretiese ondersoek geloods na die redes vir, en doelstellings van gehooropleiding, onderrigfilosojiee, inhoud en teikengroep, en metodiese benaderingswyses. Die mate waarin hierdie aspekte, soos gevind in gepubliseerde en ongepubliseerde bronne, neerslag gevind het in tersiere gehooropleidingsprogramme in die Republiek van Suid-Afrika, die Federale Republiek van Duitsland en die Verenigde State van Amerika, vorm die kern van die vraelys-gebaseerde tweede deel. 'n Model vir gelntegreerde gehooropleiding met kinders word in die derde deel voorgestel. Resultate van beide die teoretiese en praktiese ondersoeke toon dat: (a) die redes vir, en doelstellings van gehooropleiding nog nie goed deurdink is nie; (b) spesiaal geskoolde gehooropleidingsdosente benodig word; (c) voornemende musiekstudente nie voorbereid vir tersiere gehooropleidingskursusse is nie; (d) die gehooropleidingskurrikula nie aandie vereistes van die meeste gehooropleidingsdosente voldoen nie; (e) meer onderrigtyd benodig word vir gehooropleiding, wat in beide individuele en groepsonderrig plaas behoort te vind; (f) gehooropleidingsmetodes steeds sterk belnvloed word deur die Behavioristiese skool van musiekpsigologiese denke waarin dril en herhaling beklemtoon word; (g) bladsang en diktee voorkeur geniet bo ander metodes, waarskynlik as gevolg van die feit dat studente se antwoorde makliker gekwantifiseer kan word as in ander metodes; (h) tonale musiek meestal beklemtoon word; (i) navorsingresultate waarin holistiese benaderingswyses tot gehooropleiding sedert die begin van die 1980's aanbeveel word, min tot geen invloed op die meerderheid gehooropleidingsteksboeke en -metodes gehad het; g) kommersieel beskikbare rekenaarprogramme dikwels gekies word op grond van beskikbaarheid en nie op grond van pedagogiese doelstellings nie; (k) daar 'n toenemende geinteresseerdheid onder dosente is om die doelstelling van gehooropleiding te re-evaIueer, en om musiekpsigologiese beginsels in gehooropleiding te inkorporeer. Vanwee die reeds genoemde probleemvan onvoorbereide voornemende musiekstudente wat gehooropleiding betref en die skaarsheid aan gehooropleidingsprogramme vir kinders, is 'n model vir die integrering van gehooropleiding deur middel van komposisie in die instrumentale en/of sangles voorgestel. Hierdie model is gebaseer op (a) die oortuiging dat die mees effektiewe vorm van leer plaasvind d.m. v. aktiewe betrokkenheid en skepping; en (b) die musiekpsigologiese beginsels van Gestalt-waameming, die ontwikkelingsteoriee van Piaget, Gardner en Swanwick- Tillman, en musikale denkprosesteoriee van Prince en Webster. Deur middel van hierdie model kan gehooropleiding benader word vanuit 'n holistiese oogpunt deurdat gehele binne 'n musikale konteks behandel word. Daardeur word nie net strukturele en persepsuele denke ontwikkel nie, maar ook verstandelike denkprosesse soos ontdekking, aanwending, probleemoplossing en kritiese denke. Dit bied ook die geleentheid vir 'n geheelbenadering tot die aanleer van musiekteoretiese kennis en skryfvaardighede, en die inkorporering van aIle ander gehooropleidingsmetodes. Voorbeelde van kinders se komposisies is ingesluit as bewys daarvoor dat dit weI in die praktyk toepasbaar is. Ten slotte word aanbeveel dat: (a) gehooropleidingsdosente hulle doelstellings, onderrigfilosofiee en -metodes voortdurend krities moet betrag en ondersoek; (b) gevorderde kursusse in gehooropleiding aan alle tersiere musiekinstansies aangebied behoort te word om sodoende te voorsien in die nood van onvoldoende geskoolde,dosente; (c) alle studente verplig moet word om 'n kursus in gehooropleidingsdidaktiek te volg; (d) meer onderrrigtyd ingeruim behoort te word vir gehooropleiding; (e) gehooropleiding as 'n vak in eie reg behandel moet word by puntetoekennings; (t) metodes van 001 en herhaling met holistiese benaderingswyses tot gehooropleiding in beide klaskamergebaseerde en geprogrammeerde onderrig vervang moet word; (g) musiek van alle stylperiodes ingesluit behoort te word; (h) 'n wye spektrum van gehooropleidingsmetodes aangewend moet word en me slegs bladsang en diktee me; (i) gehooropleidingsdosente hulself voortdurend op hoogte van die nuutste navorsingsresul-tate behoort'te hou en moet poog om hierdie resultate in hulle gehooropleidingsprogramme te integreer; g) gehooropleidingsboeke en rekenaarprogramme wat op holistiese en kontekstuele benaderings tot gehooropleiding gebaseer is, gepubliseer moet word; (k) gehooropleiding vanaf die eerste instrumentale en/of sangles gei'ntegreer behoort te word deurdat die voorgestelde model van 'n holistiese benaderingswyse d.m. v. komposisie aangewend word

Proceedings of the 7th international conference on disability, virtual reality and associated technologies, with ArtAbilitation (ICDVRAT 2008)

The proceedings of the conferenc

Proceedings of the 6th international conference on disability, virtual reality and associated technologies (ICDVRAT 2006)

The proceedings of the conferenc

Proceedings of the 4th international conference on disability, virtual reality and associated technologies (ICDVRAT 2002)

The proceedings of the conferenc

Visual-auditory visualisation of dynamic multi-scale heterogeneous objects.

The multi-scale phenomena analysis is an area of active research that is connecting simulations with experiments to get a correct insight into the compound dynamic structure. Visualisation is a challenging task due to a large amount of data and a wide range of complex data representations. The analysis of dynamic multi-scale phenomena requires a combination of geometric modelling and rendering techniques for the analysis of the changes in the internal structure in the case of data coming from different sources of various nature. Moreover, the area often addresses the limitations of solely visual data representation and considers the introduction of other sensory stimuli as a well-known tool to enhance visual analysis. However, there is a lack of software tools allowing perform an advanced real-time analysis of heterogeneous phenomena properties. The hardware-accelerated volume rendering allows getting insight into the internal structure of complex multi-scale phenomena. The technique is convenient for detailed visual analysis and highlights the features of interest in complex structures and is an area of active research. However, the conventional volume visualisation is limited to the use of transfer functions that operate on homogeneous material and, as a result, does not provide flexibility in geometry and material distribution modelling that is crucial for the analysis of heterogeneous objects. Moreover, the extension to visual-auditory analysis emphasises the necessity to review the entire conventional volume visualisation pipeline. The multi-sensory feedback highly depends on the use of modern hardware and software advances for real-time modelling and evaluation. In this work, we explore the aspects of the design of visual-auditory pipelines for the analysis of dynamic multi-scale properties of heterogeneous objects that can allow overcoming well-known problems of complex representations solely visual analysis. We consider the similarities between light and sound propagation as a solution to the problem. The approach benefits from a combination of GPU accelerated ray-casting, geometry, optical and auditory properties modelling. We discuss how the modern GPU techniques application in those areas allows introducing a unified approach to the visual-auditory analysis of dynamic multi-scale heterogeneous objects. Similarly to the conventional volume rendering technique based on light propagation, we model auditory feedback as a result of initial impulse propagation through 3D space and its digital representation as a sampled sound wave obtained with the ray-casting procedure. The auditory stimuli can complement visual ones in the analysis of the dynamic multi-scale heterogeneous object. We propose a framework that facilitates the design of dynamic multi-scale heterogeneous objects visual-auditory pipeline and discuss the framework application for two case studies. The first is a molecular phenomena study that is a result of molecular dynamics simulation and quantum simulation. The second explores microstructures in digital fabrication with an arbitrary irregular lattice structure. For considered case studies, the visual-auditory techniques facilitate the interactive analysis of both spatial structure and internal multi-scale properties of volume nature in complex heterogeneous objects. A GPU-accelerated framework for visual-auditory analysis of heterogeneous objects can be applied and extend beyond this research. Thus, to specify the main direction of such extension from the point of view of the potential users, strengthen the value of this research as well as to evaluate the vision of the application of the techniques described above, we carry out a preliminary evaluation. The user study aims to compare our expectations on the visual-auditory approach with the views of the potential users of this system if it is implemented as a software product. A preliminary evaluation study was carried out with limitations imposed by 2020/2021 restrictions. However, it confirms that the main direction for the visual-auditory analysis of heterogeneous objects has been identified correctly and visual and auditory stimuli can complement each other in the analysis of both volume and spatial distribution properties of heterogeneous phenomena. The user reviews also highlight the necessary enhancements that should be introduced to the approach in terms of the design of more complex user interfaces and consideration of additional application cases. To provide a more detailed picture on evaluation results and recommendations introduced, we also identify the key factors that define the user vision of the approach further enhancement and its possible application areas, such as users experience in the area of complex physical phenomena analysis or multi-sensory area. The discussed in this work aspects of heterogeneous objects analysis task, theoretical and practical solutions allow considering the application, further development and enhancement of the results in multidisciplinary areas of GPU accelerated High-performance visualisation pipelines design and multi-sensory analysis