8 research outputs found
Paths Explored, Paths Omitted, Paths Obscured: Decision Points & Selective Reporting in End-to-End Data Analysis
Drawing reliable inferences from data involves many, sometimes arbitrary,
decisions across phases of data collection, wrangling, and modeling. As
different choices can lead to diverging conclusions, understanding how
researchers make analytic decisions is important for supporting robust and
replicable analysis. In this study, we pore over nine published research
studies and conduct semi-structured interviews with their authors. We observe
that researchers often base their decisions on methodological or theoretical
concerns, but subject to constraints arising from the data, expertise, or
perceived interpretability. We confirm that researchers may experiment with
choices in search of desirable results, but also identify other reasons why
researchers explore alternatives yet omit findings. In concert with our
interviews, we also contribute visualizations for communicating decision
processes throughout an analysis. Based on our results, we identify design
opportunities for strengthening end-to-end analysis, for instance via tracking
and meta-analysis of multiple decision paths
Visual Steering for One-Shot Deep Neural Network Synthesis
Recent advancements in the area of deep learning have shown the effectiveness
of very large neural networks in several applications. However, as these deep
neural networks continue to grow in size, it becomes more and more difficult to
configure their many parameters to obtain good results. Presently, analysts
must experiment with many different configurations and parameter settings,
which is labor-intensive and time-consuming. On the other hand, the capacity of
fully automated techniques for neural network architecture search is limited
without the domain knowledge of human experts. To deal with the problem, we
formulate the task of neural network architecture optimization as a graph space
exploration, based on the one-shot architecture search technique. In this
approach, a super-graph of all candidate architectures is trained in one-shot
and the optimal neural network is identified as a sub-graph. In this paper, we
present a framework that allows analysts to effectively build the solution
sub-graph space and guide the network search by injecting their domain
knowledge. Starting with the network architecture space composed of basic
neural network components, analysts are empowered to effectively select the
most promising components via our one-shot search scheme. Applying this
technique in an iterative manner allows analysts to converge to the best
performing neural network architecture for a given application. During the
exploration, analysts can use their domain knowledge aided by cues provided
from a scatterplot visualization of the search space to edit different
components and guide the search for faster convergence. We designed our
interface in collaboration with several deep learning researchers and its final
effectiveness is evaluated with a user study and two case studies.Comment: 9 pages, submitted to IEEE Transactions on Visualization and Computer
Graphics, 202