63 research outputs found
Information Flow in Computational Systems
We develop a theoretical framework for defining and identifying flows of
information in computational systems. Here, a computational system is assumed
to be a directed graph, with "clocked" nodes that send transmissions to each
other along the edges of the graph at discrete points in time. We are
interested in a definition that captures the dynamic flow of information about
a specific message, and which guarantees an unbroken "information path" between
appropriately defined inputs and outputs in the directed graph. Prior measures,
including those based on Granger Causality and Directed Information, fail to
provide clear assumptions and guarantees about when they correctly reflect
information flow about a message. We take a systematic approach---iterating
through candidate definitions and counterexamples---to arrive at a definition
for information flow that is based on conditional mutual information, and which
satisfies desirable properties, including the existence of information paths.
Finally, we describe how information flow might be detected in a noiseless
setting, and provide an algorithm to identify information paths on the
time-unrolled graph of a computational system.Comment: Significantly revised version which was accepted for publication at
the IEEE Transactions on Information Theor
Child labor and household wealth: Theory and empirical evidence of an inverted-U
Some studies on child labor have shown that greater land wealth leads to higher child labor, thereby casting doubt on the hypothesis that child labor is caused by poverty. This paper argues that the missing ingredient is an explicit modeling of the labor market. We develop a simple model which suggests an inverted-U relationship between land holdings and child labor. A unique data set from India that has child labor hours information confirms this hypothesis. It is shown that the turning point beyond which more land leads to a decline in child labor occurs at 3.6 acres of land per household, which is well below the observed maximum value of and-holding.child labor, land-holding, education, labor markets
Child Labor and Household Wealth : Theory and Empirical Evidence of an Inverted-U
Some studies on child labor have shown that, at the level of the household, greater land wealth leads to higher child labor, thereby casting doubt on the hypothesis that child labor is caused by poverty. This paper argues that the missing ingredient may be an explicit modeling of the labor market. We develop a simple model which suggests the possibility of an inverted-U relationship between land holdings and child labor. Using a unique data set that has child labor hours it is found that, controlling for child, household and village characteristics, the turning point beyond which more land leads to a decline in child labor occurs around 4 acres of land per household.child labor ; land-holding ; labor markets
Demystifying Local and Global Fairness Trade-offs in Federated Learning Using Partial Information Decomposition
In this paper, we present an information-theoretic perspective to group
fairness trade-offs in federated learning (FL) with respect to sensitive
attributes, such as gender, race, etc. Existing works mostly focus on either
\emph{global fairness} (overall disparity of the model across all clients) or
\emph{local fairness} (disparity of the model at each individual client),
without always considering their trade-offs. There is a lack of understanding
of the interplay between global and local fairness in FL, and if and when one
implies the other. To address this gap, we leverage a body of work in
information theory called partial information decomposition (PID) which first
identifies three sources of unfairness in FL, namely, \emph{Unique Disparity},
\emph{Redundant Disparity}, and \emph{Masked Disparity}. Using canonical
examples, we demonstrate how these three disparities contribute to global and
local fairness. This decomposition helps us derive fundamental limits and
trade-offs between global or local fairness, particularly under data
heterogeneity, as well as, derive conditions under which one implies the other.
We also present experimental results on benchmark datasets to support our
theoretical findings. This work offers a more nuanced understanding of the
sources of disparity in FL that can inform the use of local disparity
mitigation techniques, and their convergence and effectiveness when deployed in
practice.Comment: Accepted at ICML Workshop on Federated Learning and Analytics in
Practic
A Unified Coded Deep Neural Network Training Strategy Based on Generalized PolyDot Codes for Matrix Multiplication
This paper has two contributions. First, we propose a novel coded matrix
multiplication technique called Generalized PolyDot codes that advances on
existing methods for coded matrix multiplication under storage and
communication constraints. This technique uses "garbage alignment," i.e.,
aligning computations in coded computing that are not a part of the desired
output. Generalized PolyDot codes bridge between Polynomial codes and MatDot
codes, trading off between recovery threshold and communication costs. Second,
we demonstrate that Generalized PolyDot can be used for training large Deep
Neural Networks (DNNs) on unreliable nodes prone to soft-errors. This requires
us to address three additional challenges: (i) prohibitively large overhead of
coding the weight matrices in each layer of the DNN at each iteration; (ii)
nonlinear operations during training, which are incompatible with linear
coding; and (iii) not assuming presence of an error-free master node, requiring
us to architect a fully decentralized implementation without any "single point
of failure." We allow all primary DNN training steps, namely, matrix
multiplication, nonlinear activation, Hadamard product, and update steps as
well as the encoding/decoding to be error-prone. We consider the case of
mini-batch size , as well as , leveraging coded matrix-vector
products, and matrix-matrix products respectively. The problem of DNN training
under soft-errors also motivates an interesting, probabilistic error model
under which a real number MDS code is shown to correct errors
with probability as compared to for the
more conventional, adversarial error model. We also demonstrate that our
proposed strategy can provide unbounded gains in error tolerance over a
competing replication strategy and a preliminary MDS-code-based strategy for
both these error models.Comment: Presented in part at the IEEE International Symposium on Information
Theory 2018 (Submission Date: Jan 12 2018); Currently under review at the
IEEE Transactions on Information Theor
Management Approach for Earth Venture Instrument
The Earth Venture Instrument (EVI) element of the Earth Venture Program calls for developing instruments for participation on a NASA-arranged spaceflight mission of opportunity to conduct innovative, integrated, hypothesis or scientific question-driven approaches to pressing Earth system science issues. This paper discusses the EVI element and the management approach being used to manage both an instrument development activity as well as the host accommodations activity. In particular the focus will be on the approach being used for the first EVI (EVI-1) selected instrument, Tropospheric Emissions: Monitoring of Pollution (TEMPO), which will be hosted on a commercial GEO satellite and some of the challenges encountered to date and corresponding mitigations that are associated with the management structure for the TEMPO Mission and the architecture of EVI
- …