Know What You See - Visual Analytics enabling Machine Learning Performance Evaluation

This work presents a visual analytics-driven workflow for an interpretable and understandable machine learning model. The model is driven by a reverse engineering task in automotive assembly processes. The model aims to predict the assembly parameters leading to the given displacement field on the geometries surface. The derived model can work on both measurement and simulation data. The proposed approach is driven by the scientific goals from visual analytics and interpretable artificial intelligence alike. First, a concept for systematic uncertainty monitoring, an object-oriented, virtual reference scheme (OOVRS), is developed. Afterward, the prediction task is solved via a regressive machine learning model using adversarial neural networks. A profound model parameter study is conducted and assisted with an interactive visual analytics pipeline. Further, the effects of the learned variance in displacement fields are analyzed in detail. Therefore a visual analytics pipeline is developed, resulting in a sensitivity benchmarking tool. This allows the testing of various segmentation approaches to lower the machine learning input dimensions. The effects of the assembly parameters are investigated in domain space to find a suitable segmentation of the training data set’s geometry. Therefore, a sensitivity matrix visualization is developed. Further, it is shown how this concept could directly compare results from various segmentation methods, e.g., topological segmentation, concerning the assembly parameters and their impact on the displacement field variance. The resulting databases are still of substantial size for complex simulations with large and high-dimensional parameter spaces. Finally, the applicability of video compression techniques towards compressing visualization image databases is studied

Know What You See - Visual Analytics enabling Machine Learning Performance Evaluation

This work presents a visual analytics-driven workflow for an interpretable and understandable machine learning model. The model is driven by a reverse engineering task in automotive assembly processes. The model aims to predict the assembly parameters leading to the given displacement field on the geometries surface. The derived model can work on both measurement and simulation data. The proposed approach is driven by the scientific goals from visual analytics and interpretable artificial intelligence alike. First, a concept for systematic uncertainty monitoring, an object-oriented, virtual reference scheme (OOVRS), is developed. Afterward, the prediction task is solved via a regressive machine learning model using adversarial neural networks. A profound model parameter study is conducted and assisted with an interactive visual analytics pipeline. Further, the effects of the learned variance in displacement fields are analyzed in detail. Therefore a visual analytics pipeline is developed, resulting in a sensitivity benchmarking tool. This allows the testing of various segmentation approaches to lower the machine learning input dimensions. The effects of the assembly parameters are investigated in domain space to find a suitable segmentation of the training data set’s geometry. Therefore, a sensitivity matrix visualization is developed. Further, it is shown how this concept could directly compare results from various segmentation methods, e.g., topological segmentation, concerning the assembly parameters and their impact on the displacement field variance. The resulting databases are still of substantial size for complex simulations with large and high-dimensional parameter spaces. Finally, the applicability of video compression techniques towards compressing visualization image databases is studied

Spatial Sound in a 3D Virtual Environment: All Bark and No Bite?

Although the focus of Virtual Reality (VR) lies predominantly on the visual world, acoustic components enhance the functionality of a 3D environment. To study the interaction between visual and auditory modalities in a 3D environment, we investigated the effect of auditory cues on visual searches in 3D virtual environments with both visual and auditory noise. In an experiment, we asked participants to detect visual targets in a 360° video in conditions with and without environmental noise. Auditory cues indicating the target location were either absent or one of simple stereo or binaural audio, both of which assisted sound localization. To investigate the efficacy of these cues in distracting environments, we measured participant performance using a VR headset with an eye tracker. We found that the binaural cue outperformed both stereo and no auditory cues in terms of target detection irrespective of the environmental noise. We used two eye movement measures and two physiological measures to evaluate task dynamics and mental effort. We found that the absence of a cue increased target search duration and target search path, measured as time to fixation and gaze trajectory lengths, respectively. Our physiological measures of blink rate and pupil size showed no difference between the different stadium and cue conditions. Overall, our study provides evidence for the utility of binaural audio in a realistic, noisy and virtual environment for performing a target detection task, which is a crucial part of everyday behaviour—finding someone in a crowd

Spatial Sound in a 3D Virtual Environment: All Bark and No Bite?

Although the focus of Virtual Reality (VR) lies predominantly on the visual world, acoustic components enhance the functionality of a 3D environment. To study the interaction between visual and auditory modalities in a 3D environment, we investigated the effect of auditory cues on visual searches in 3D virtual environments with both visual and auditory noise. In an experiment, we asked participants to detect visual targets in a 360° video in conditions with and without environmental noise. Auditory cues indicating the target location were either absent or one of simple stereo or binaural audio, both of which assisted sound localization. To investigate the efficacy of these cues in distracting environments, we measured participant performance using a VR headset with an eye tracker. We found that the binaural cue outperformed both stereo and no auditory cues in terms of target detection irrespective of the environmental noise. We used two eye movement measures and two physiological measures to evaluate task dynamics and mental effort. We found that the absence of a cue increased target search duration and target search path, measured as time to fixation and gaze trajectory lengths, respectively. Our physiological measures of blink rate and pupil size showed no difference between the different stadium and cue conditions. Overall, our study provides evidence for the utility of binaural audio in a realistic, noisy and virtual environment for performing a target detection task, which is a crucial part of everyday behaviour—finding someone in a crowd