Study on the Procedural Generation of Visualization from Musical Input using Generative Art Techniques

The purpose of this study was to create a new method for visualizing music. Although many music visualizations already exist, this research was focused on creating high-quality, high-complexity animations that cannot be matched by real-time systems. There should be an obvious similarity between the input music and the final animation, based on the music information which the user decides to extract and visualize. This project includes a pipeline for music data extraction and creation of an editable visualization file. Within the pipeline, a music file is read into a custom analysis tool and time-based data is extracted. This data is output and then read into Autodesk Maya. The user may then manipulate the visualization as they see fit using the tools within Maya and render out a final animation. The default result of this process is a Maya scene file which makes use of the dynamics systems available to warp and contort a jelly-like cube. A variety of other visualizations may be obtained by mapping the data to different object attributes within the Maya interface. When rendered out and overlaid onto the music, there was a recognizable correlation between elements in the music and the animations in the video. This study shows that an accurate musical visualization may be achieved using this pipeline. Also, any number of different music visualizations may be obtained with relative ease when compared to the manual analysis of a music file or the manual animation of Maya objects to match elements in the music

Sound Visualization for Deaf Assistance Using Mobile Computing

This thesis presents a new approach to the visualization of sound for deaf assistance that simultaneously illustrates important dynamic sound properties and the recognized sound icons in an easy readable view. .In order to visualize general sounds efficiently, the MFCC sound features was utilized to represent robust discriminant properties of the sound. The problem of visualizing MFCC vector that has 39 dimension was simplified by visualizing one-dimensional value, which is the result of comparing one reference MFCC vector with the input MFCC vector only. New similarity measure for MFCC feature vectors comparison was proposed that outperforms existing local similarity measures due to their problem of one to one attribute value calculation that leaded to incorrect similarity decisions. Classification of input sound was performed and attached to the visualizing system to make the system more usable for users. Each time frame of sound is put under K-NN classification algorithm to detect short sound events. In addition, every one second the input sound is buffered and forwarded to Dynamic Time Warping (DTW) classification algorithm which is designed for dynamic time series classification. Both classifiers works in the same time and deliver their classification results to the visualization model. The application of the system was implemented using Java programming language to work on smartphones that run Android OS, so many considerations related to the complexity of algorithms is taken into account. The system was implemented to utilize the capabilities of the smartphones GPU to guarantee the smoothness and fastness of the rendering. The system design was built based on interviews with five deaf persons taking into account their preferred visualizing system. In addition to that, the same deaf persons tested the system and the evaluation of the system is carried out based on their interaction with the system. Our approach yields more accessible illustrations of sound and more suitable for casual and little expert users