10 research outputs found
Visual Task Classification using Classic Machine Learning and CNNs
Our eyes actively perform tasks including, but not limited to, searching, comparing, and counting. This includes tasks in front of a computer, whether it be trivial activities like reading email, or video gaming, or more serious activities like drone management, or flight simulation. Understanding what type of visual task is being performed is important to develop intelligent user interfaces. In this work, we investigated standard machine and deep learning methods to identify the task type using eye-tracking data-including both raw numerical data and the visual representations of the user gaze scan paths and pupil size. To this end, we experimented with computer vision algorithms such as Convolutional Neural Networks (CNNs) and compared the results to classic machine learning algorithms. We found that Machine learning-based methods performed with high accuracy classifying tasks that involve minimal visual search, while CNNs techniques do better in situations where visual search task is included
Task Classification During Visual Search Using Classic Machine Learning and Deep Learning
In an average human life, the eyes not only passively scan visual scenes, but most times end up actively performing tasks including, but not limited to, searching, comparing, and counting. As a result of the advances in technology, we are observing a boost in the average screen time. Humans are now looking at an increasing number of screens and in turn images and videos. Understanding what scene a user is looking at and what type of visual task is being performed can be useful in developing intelligent user interfaces, and in virtual reality and augmented reality devices. In this research, we run machine learning and deep learning algorithms to identify the task type from eye-tracking data. In addition to looking at raw numerical data, we take a “visual” approach by experimenting on variations of Computer Vision algorithms like Convolutional Neural Networks on the visual representations of the user gaze scan paths. We compare the results of our visual approach to the classic algorithm of random forests
Task Classification during Visual Search with Deep Learning Neural Networks and Machine Learning Methods
Studies have shown the possibility to classify user tasks from eye-movement data. We present a new way to determine the optimal model for different visual attention tasks using data that includes two types of visual search tasks, a visual exploration task, a blank screen task, and a task where a user needs to fixate at the center of any scene. We used deep learning and SVM models on RGB images generated from fixation scan paths from these tasks. We also used AdaBoost on filtered eye movement data as a baseline. Our study shows that deep learning gives the best accuracy for classifying between visual search tasks but misclassified between visual search and visual exploration tasks. Machine learning-based methods performed with high accuracy classifying tasks that involve minimal visual attention. Our study gives insight on the best model to choose by type of visual task using eye movement data
Improving the understanding of web user behaviors through machine learning analysis of eye-tracking data
Eye-tracking techniques are widely used to analyze user behavior. While eye-trackers collect valuable quantitative data, the results are often described in a qualitative manner due to the lack of a model that interprets the gaze trajectories generated by routine tasks, such as reading or comparing two products. The aim of this work is to propose a new quantitative way to analyze gaze trajectories (scanpaths) using machine learning. We conducted a within-subjects study (N = 30) testing six different tasks that simulated specific user behaviors in web sites (attentional, comparing two images, reading in different contexts, and free surfing). We evaluated the scanpath results with three different classifiers (long short-term memory recurrent neural network—LSTM, random forest, and multilayer perceptron neural network—MLP) to discriminate between tasks. The results revealed that it is possible to classify and distinguish between the 6 different web behaviors proposed in this study based on the user’s scanpath. The classifier that achieved the best results was the LSTM, with a 95.7% accuracy. To the best of our knowledge, this is the first study to provide insight about MLP and LSTM classifiers to discriminate between tasks. In the discussion, we propose practical implications of the study results
ICE: An Interactive Configuration Explorer for High Dimensional Categorical Parameter Spaces
There are many applications where users seek to explore the impact of the
settings of several categorical variables with respect to one dependent
numerical variable. For example, a computer systems analyst might want to study
how the type of file system or storage device affects system performance. A
usual choice is the method of Parallel Sets designed to visualize multivariate
categorical variables. However, we found that the magnitude of the parameter
impacts on the numerical variable cannot be easily observed here. We also
attempted a dimension reduction approach based on Multiple Correspondence
Analysis but found that the SVD-generated 2D layout resulted in a loss of
information. We hence propose a novel approach, the Interactive Configuration
Explorer (ICE), which directly addresses the need of analysts to learn how the
dependent numerical variable is affected by the parameter settings given
multiple optimization objectives. No information is lost as ICE shows the
complete distribution and statistics of the dependent variable in context with
each categorical variable. Analysts can interactively filter the variables to
optimize for certain goals such as achieving a system with maximum performance,
low variance, etc. Our system was developed in tight collaboration with a group
of systems performance researchers and its final effectiveness was evaluated
with expert interviews, a comparative user study, and two case studies.Comment: 10 pages, Published by IEEE at VIS 2019 (Vancouver, BC, Canada
Visual Steering for One-Shot Deep Neural Network Synthesis
Recent advancements in the area of deep learning have shown the effectiveness
of very large neural networks in several applications. However, as these deep
neural networks continue to grow in size, it becomes more and more difficult to
configure their many parameters to obtain good results. Presently, analysts
must experiment with many different configurations and parameter settings,
which is labor-intensive and time-consuming. On the other hand, the capacity of
fully automated techniques for neural network architecture search is limited
without the domain knowledge of human experts. To deal with the problem, we
formulate the task of neural network architecture optimization as a graph space
exploration, based on the one-shot architecture search technique. In this
approach, a super-graph of all candidate architectures is trained in one-shot
and the optimal neural network is identified as a sub-graph. In this paper, we
present a framework that allows analysts to effectively build the solution
sub-graph space and guide the network search by injecting their domain
knowledge. Starting with the network architecture space composed of basic
neural network components, analysts are empowered to effectively select the
most promising components via our one-shot search scheme. Applying this
technique in an iterative manner allows analysts to converge to the best
performing neural network architecture for a given application. During the
exploration, analysts can use their domain knowledge aided by cues provided
from a scatterplot visualization of the search space to edit different
components and guide the search for faster convergence. We designed our
interface in collaboration with several deep learning researchers and its final
effectiveness is evaluated with a user study and two case studies.Comment: 9 pages, submitted to IEEE Transactions on Visualization and Computer
Graphics, 202
Task classification model for visual fixation, exploration, and search
Yarbus’ claim to decode the observer’s task from eye movements has received mixed reactions. In this paper, we have supported the hypothesis that it is possible to decode the task. We conducted an exploratory analysis on the dataset by projecting features and data points into a scatter plot to visualize the nuance properties for each task. Following this analysis, we eliminated highly correlated features before training an SVM and Ada Boosting classifier to predict the tasks from this filtered eye movements data. We achieve an accuracy of 95.4% on this task classification problem and hence, support the hypothesis that task classification is possible from a user’s eye movement data
Task Classification Model for Visual Fixation, Exploration, and Search
Yarbus' claim to decode the observer's task from eye movements has received mixed reactions. In this paper, we have supported the hypothesis that it is possible to decode the task. We conducted an exploratory analysis on the dataset by projecting features and data points into a scatter plot to visualize the nuance properties for each task. Following this analysis, we eliminated highly correlated features before training an SVM and Ada Boosting classifier to predict the tasks from this filtered eye movements data. We achieve an accuracy of 95.4% on this task classification problem and hence, support the hypothesis that task classification is possible from a user's eye movement data
Task classification model for visual fixation, exploration, and search
\u3cp\u3eYarbus’ claim to decode the observer’s task from eye movements has received mixed reactions. In this paper, we have supported the hypothesis that it is possible to decode the task. We conducted an exploratory analysis on the dataset by projecting features and data points into a scatter plot to visualize the nuance properties for each task. Following this analysis, we eliminated highly correlated features before training an SVM and Ada Boosting classifier to predict the tasks from this filtered eye movements data. We achieve an accuracy of 95.4% on this task classification problem and hence, support the hypothesis that task classification is possible from a user’s eye movement data.\u3c/p\u3