19,166 research outputs found
Dynamic Control Flow in Large-Scale Machine Learning
Many recent machine learning models rely on fine-grained dynamic control flow
for training and inference. In particular, models based on recurrent neural
networks and on reinforcement learning depend on recurrence relations,
data-dependent conditional execution, and other features that call for dynamic
control flow. These applications benefit from the ability to make rapid
control-flow decisions across a set of computing devices in a distributed
system. For performance, scalability, and expressiveness, a machine learning
system must support dynamic control flow in distributed and heterogeneous
environments.
This paper presents a programming model for distributed machine learning that
supports dynamic control flow. We describe the design of the programming model,
and its implementation in TensorFlow, a distributed machine learning system.
Our approach extends the use of dataflow graphs to represent machine learning
models, offering several distinctive features. First, the branches of
conditionals and bodies of loops can be partitioned across many machines to run
on a set of heterogeneous devices, including CPUs, GPUs, and custom ASICs.
Second, programs written in our model support automatic differentiation and
distributed gradient computations, which are necessary for training machine
learning models that use control flow. Third, our choice of non-strict
semantics enables multiple loop iterations to execute in parallel across
machines, and to overlap compute and I/O operations.
We have done our work in the context of TensorFlow, and it has been used
extensively in research and production. We evaluate it using several real-world
applications, and demonstrate its performance and scalability.Comment: Appeared in EuroSys 2018. 14 pages, 16 figure
Ariadne: Analysis for Machine Learning Program
Machine learning has transformed domains like vision and translation, and is
now increasingly used in science, where the correctness of such code is vital.
Python is popular for machine learning, in part because of its wealth of
machine learning libraries, and is felt to make development faster; however,
this dynamic language has less support for error detection at code creation
time than tools like Eclipse. This is especially problematic for machine
learning: given its statistical nature, code with subtle errors may run and
produce results that look plausible but are meaningless. This can vitiate
scientific results. We report on Ariadne: applying a static framework, WALA, to
machine learning code that uses TensorFlow. We have created static analysis for
Python, a type system for tracking tensors---Tensorflow's core data
structures---and a data flow analysis to track their usage. We report on how it
was built and present some early results
Doctor of Philosophy
dissertationA broad range of applications capture dynamic data at an unprecedented scale. Independent of the application area, finding intuitive ways to understand the dynamic aspects of these increasingly large data sets remains an interesting and, to some extent, unsolved research problem. Generically, dynamic data sets can be described by some, often hierarchical, notion of feature of interest that exists at each moment in time, and those features evolve across time. Consequently, exploring the evolution of these features is considered to be one natural way of studying these data sets. Usually, this process entails the ability to: 1) define and extract features from each time step in the data set; 2) find their correspondences over time; and 3) analyze their evolution across time. However, due to the large data sizes, visualizing the evolution of features in a comprehensible manner and performing interactive changes are challenging. Furthermore, feature evolution details are often unmanageably large and complex, making it difficult to identify the temporal trends in the underlying data. Additionally, many existing approaches develop these components in a specialized and standalone manner, thus failing to address the general task of understanding feature evolution across time. This dissertation demonstrates that interactive exploration of feature evolution can be achieved in a non-domain-specific manner so that it can be applied across a wide variety of application domains. In particular, a novel generic visualization and analysis environment that couples a multiresolution unified spatiotemporal representation of features with progressive layout and visualization strategies for studying the feature evolution across time is introduced. This flexible framework enables on-the-fly changes to feature definitions, their correspondences, and other arbitrary attributes while providing an interactive view of the resulting feature evolution details. Furthermore, to reduce the visual complexity within the feature evolution details, several subselection-based and localized, per-feature parameter value-based strategies are also enabled. The utility and generality of this framework is demonstrated by using several large-scale dynamic data sets
Fundamental structures of dynamic social networks
Social systems are in a constant state of flux with dynamics spanning from
minute-by-minute changes to patterns present on the timescale of years.
Accurate models of social dynamics are important for understanding spreading of
influence or diseases, formation of friendships, and the productivity of teams.
While there has been much progress on understanding complex networks over the
past decade, little is known about the regularities governing the
micro-dynamics of social networks. Here we explore the dynamic social network
of a densely-connected population of approximately 1000 individuals and their
interactions in the network of real-world person-to-person proximity measured
via Bluetooth, as well as their telecommunication networks, online social media
contacts, geo-location, and demographic data. These high-resolution data allow
us to observe social groups directly, rendering community detection
unnecessary. Starting from 5-minute time slices we uncover dynamic social
structures expressed on multiple timescales. On the hourly timescale, we find
that gatherings are fluid, with members coming and going, but organized via a
stable core of individuals. Each core represents a social context. Cores
exhibit a pattern of recurring meetings across weeks and months, each with
varying degrees of regularity. Taken together, these findings provide a
powerful simplification of the social network, where cores represent
fundamental structures expressed with strong temporal and spatial regularity.
Using this framework, we explore the complex interplay between social and
geospatial behavior, documenting how the formation of cores are preceded by
coordination behavior in the communication networks, and demonstrating that
social behavior can be predicted with high precision.Comment: Main Manuscript: 16 pages, 4 figures. Supplementary Information: 39
pages, 34 figure
- …