Neural scaling laws for an uncertain world

Autonomous neural systems must efficiently process information in a wide range of novel environments, which may have very different statistical properties. We consider the problem of how to optimally distribute receptors along a one-dimensional continuum consistent with the following design principles. First, neural representations of the world should obey a neural uncertainty principle---making as few assumptions as possible about the statistical structure of the world. Second, neural representations should convey, as much as possible, equivalent information about environments with different statistics. The results of these arguments resemble the structure of the visual system and provide a natural explanation of the behavioral Weber-Fechner law, a foundational result in psychology. Because the derivation is extremely general, this suggests that similar scaling relationships should be observed not only in sensory continua, but also in neural representations of ``cognitive' one-dimensional quantities such as time or numerosity

Quantization in acquisition and computation networks

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2013.Cataloged from PDF version of thesis.Includes bibliographical references (p. 151-165).In modern systems, it is often desirable to extract relevant information from large amounts of data collected at different spatial locations. Applications include sensor networks, wearable health-monitoring devices and a variety of other systems for inference. Several existing source coding techniques, such as Slepian-Wolf and Wyner-Ziv coding, achieve asymptotic compression optimality in distributed systems. However, these techniques are rarely used in sensor networks because of decoding complexity and prohibitively long code length. Moreover, the fundamental limits that arise from existing techniques are intractable to describe for a complicated network topology or when the objective of the system is to perform some computation on the data rather than to reproduce the data. This thesis bridges the technological gap between the needs of real-world systems and the optimistic bounds derived from asymptotic analysis. Specifically, we characterize fundamental trade-offs when the desired computation is incorporated into the compression design and the code length is one. To obtain both performance guarantees and achievable schemes, we use high-resolution quantization theory, which is complementary to the Shannon-theoretic analyses previously used to study distributed systems. We account for varied network topologies, such as those where sensors are allowed to collaborate or the communication links are heterogeneous. In these settings, a small amount of intersensor communication can provide a significant improvement in compression performance. As a result, this work suggests new compression principles and network design for modern distributed systems. Although the ideas in the thesis are motivated by current and future sensor network implementations, the framework applies to a wide range of signal processing questions. We draw connections between the fidelity criteria studied in the thesis and distortion measures used in perceptual coding. As a consequence, we determine the optimal quantizer for expected relative error (ERE), a measure that is widely useful but is often neglected in the source coding community. We further demonstrate that applying the ERE criterion to psychophysical models can explain the Weber-Fechner law, a longstanding hypothesis of how humans perceive the external world. Our results are consistent with the hypothesis that human perception is Bayesian optimal for information acquisition conditioned on limited cognitive resources, thereby supporting the notion that the brain is efficient at acquisition and adaptation.by John Z. Sun.Ph.D

Discounting: A Review of the Basic Economics

I review the justifications given for discounting future benefits relative to present, and distinguish between the pure rate of time preference, or utility discount rate, and the consumption discount rate, also sometimes known as the social rate of discount. I discuss when to choose one or the other, and how to choose a discount rate in a real-world project

Reverse first principles: Weber's law and optimality in different senses

The relationship between optimality and evolvability is analyzed through a case study of Weber's law, a common property of many sensory systems across a wide array of species. After demonstrating a variety of senses in which Weber's law is mathematically optimal, we ask whether principled methods exist for evaluating such optimality analyses. We argue that at least one such method exists: examining the evolvability of a trait with respect to each of the different metrics that it happens to optimize. Through evolvability analyses of Weber's law, it is demonstrated that optimality-equivalent measures of phenotypic quality need not be selectively equivalent: a trait that is optimal by two measures may have very different behavior under selection for each. This non-equivalence allows different optimality analyses of the same phenomenon to be assessed by a standard other than intuition, and in a manner requiring fewer degrees of freedom than are needed to model selection from scratch. Two qualitatively different models of selection are explored: phenotypic selection, a basic form in which mutations directly affect the model phenotype, and embryological selection, a more exotic form in which mutations affect the algorithm by which the phenotype is built

Criminal sentencing by preferred numbers

Criminal sentencing is a complex cognitive activity often performed by the unaided mind under suboptimal conditions. As such, sentencers may not behave according to policy, guidelines and training. We analyzed the distribution of sentences meted out in one year in two different jurisdictions (i.e., England and Wales, and New South Wales, Australia). We reveal that sentencers prefer certain numbers when meting out sentence lengths (in custody and community service) and amounts (for fines/compensation). These ‘common doses’ accounted for over 90% of sentences in each jurisdiction. The size of these doses increased as sentences became more severe, and doses followed a logarithmic pattern. These findings are compatible with psychological research on preferred numbers and are reminiscent of Weber’s and Fechner’s laws. Our findings run contrary to arguments against efforts to reduce judicial discretion, and potentially undermine the notion of individualized justice, as well as raise questions about the (cost) effectiveness of sentencing