We develop a theoretical framework for defining and identifying flows of
information in computational systems. Here, a computational system is assumed
to be a directed graph, with "clocked" nodes that send transmissions to each
other along the edges of the graph at discrete points in time. We are
interested in a definition that captures the dynamic flow of information about
a specific message, and which guarantees an unbroken "information path" between
appropriately defined inputs and outputs in the directed graph. Prior measures,
including those based on Granger Causality and Directed Information, fail to
provide clear assumptions and guarantees about when they correctly reflect
information flow about a message. We take a systematic approach---iterating
through candidate definitions and counterexamples---to arrive at a definition
for information flow that is based on conditional mutual information, and which
satisfies desirable properties, including the existence of information paths.
Finally, we describe how information flow might be detected in a noiseless
setting, and provide an algorithm to identify information paths on the
time-unrolled graph of a computational system.Comment: Significantly revised version which was accepted for publication at
the IEEE Transactions on Information Theor