GUI Tool for Analyzing Color and Material Compositions in VAE Latent Space

Abstract

The Rochester Institute of Technology is home to the Munsell Color Science Lab- oratory and the Program of Color Science, and they are leading the world in the field of color science. In the pursuit of evaluating the effectiveness of color-based learning data augmentations on the semantic segmentation of material classes, they are working on a variational autoencoder VAE model. In an effort to stream- line the process of visualizing the features of this model and others like it, they requested assistance in developing a GUI tool specifically designed to analyze th latent space of variational autoencoders (VAEs), concerning color and material compositions in semantic segmentation applications. This thesis will cover the process of the agile development project created to meetthis request. The resulting lution provides a simple to use method for loading,processing, and visualizing pre-trained VAE models, and erves as a foundation for further development beyond our time with it.This task has given us the opportunity to study and work with machine learning visualization, and to manage and coordinate an international project. It has been a challenging and educational experience, and we would like to thank everyone involved in this process. Thanks to our supervisor at NTNU, Giorgio Trumpy, for continuous feedback and support through this project. Thank you to Mekides Assefa Abebe for presenting us with an interesting task and guidance within a field to which we were new. Thanks to her students, Alireza "Nima" Rabbanifar and Soroush Shahbaznejad, for participating in our user tests and providing feedback. Thank you to Alyssa Brault, Megan Kless, and Tommy Paez at ISS, and Anneli T. Østlien and Mina Stolpe Foss at NTNU, for their assistance in an accelerated study exchange process. And thanks to RIT for providing us with this opportunity. Finally, thank you to everyone at MCSL for a positive working environment