Consistency of random forests

Random forests are a learning algorithm proposed by Breiman [Mach. Learn. 45 (2001) 5--32] that combines several randomized decision trees and aggregates their predictions by averaging. Despite its wide usage and outstanding practical performance, little is known about the mathematical properties of the procedure. This disparity between theory and practice originates in the difficulty to simultaneously analyze both the randomization process and the highly data-dependent tree structure. In the present paper, we take a step forward in forest exploration by proving a consistency result for Breiman's [Mach. Learn. 45 (2001) 5--32] original algorithm in the context of additive regression models. Our analysis also sheds an interesting light on how random forests can nicely adapt to sparsity. 1. Introduction. Random forests are an ensemble learning method for classification and regression that constructs a number of randomized decision trees during the training phase and predicts by averaging the results. Since its publication in the seminal paper of Breiman (2001), the procedure has become a major data analysis tool, that performs well in practice in comparison with many standard methods. What has greatly contributed to the popularity of forests is the fact that they can be applied to a wide range of prediction problems and have few parameters to tune. Aside from being simple to use, the method is generally recognized for its accuracy and its ability to deal with small sample sizes, high-dimensional feature spaces and complex data structures. The random forest methodology has been successfully involved in many practical problems, including air quality prediction (winning code of the EMC data science global hackathon in 2012, see http://www.kaggle.com/c/dsg-hackathon), chemoinformatics [Svetnik et al. (2003)], ecology [Prasad, Iverson and Liaw (2006), Cutler et al. (2007)], 3

A characterization of sequential rationalizability

A choice function is sequentially rationalizable if there is an ordered collection of asymmetric binary relations that identifies the selected alternative in every choice problem. We propose a property, F-consistency, and show that it characterizes the notion of sequential rationalizability. F-consistency is a testable property that highlights the behavioral aspects implicit in sequentially rationalizable choice. Further, our characterization result provides a novel tool with which to study how other behavioral concepts are related to sequential rationalizability, and establish a priori unexpected implications. In particular, we show that the concept of rationalizability by game trees, which, in principle, had little to do with sequential rationalizability, is a refinement of the latter. Every choice function that is rationalizable by a game tree is also sequentially rationalizable. Finally, we show that some prominent voting mechanisms are also sequentially rationalizable.Individual rationality, Rationalizability, Consistency, Bounded rationality, Behavioral economics, Voting

Skilling: More Blind Monks Examining the Elephant

Most academics and practitioners with whom the author has discussed the result in Skilling v. United States believe that it is a sensible decision. That is, the Supreme Court did the best it could to limit the reach of 18 U.S.C. § 1346, which all nine justices apparently believed—correctly—was, on its face, unconstitutionally vague. Congress responded quickly and with little consideration with the supremely under-defined § 1346. In the over twenty years since the statute\u27s enactment, the Courts of Appeals have been unable to come up with any unified limiting principles to contain its reach. The Skilling Court, evidently reluctant to again throw the matter back to Congress given that institution\u27s previous default, and not satisfied with the Courts of Appeals\u27 efforts, was determined to come up with its own narrowing interpretation. Thus, the majority deemed it appropriate to rewrite the statute to cover what it concluded was the core of the criminality the prosecutors had addressed in bringing § 1346 cases-bribery and kickbacks. The Court comes up with narrowing constructions to avoid constitutional difficulties in many statutory interpretation cases, the argument goes, and this construction is one that many in the academic and practice communities believe is reasonable. The author’s quibble with this consensus lies in her conviction that what the Court did in Skilling is as patently unconstitutional as § 1346—and that its foray into legislation is not of only academic concern. It clearly accepted Congress\u27 delegation of law-making authority and essentially promulgated a new statute out of the dog\u27s breakfast that was pre-Skilling § 1346. Some would argue that this is a good thing from a practical, if not an orthodox separation-of-powers, point of view. The author focuses on Professor Dan M. Kahan\u27s long-standing arguments in this regard. Kahan favors administrative specification of the content of arguably vague criminal prohibitions, but he believes that if one has to choose between judicial gap-filling and congressional action, the former is preferable to the latter. Kahan has argued that the Court ought to come clean and simply acknowledge that it has long been engaged in interstitial lawmaking because Congress has declined to legislate with any specificity and [a] criminal code at least partially specified by courts is both less costly and more effective than is a code fully specified by Congress. The author disagrees with Kahan’s conclusion about the viability and attractiveness of this delegation of authority to federal courts to fill in the blanks in otherwise underspecified statutory schemes. The honest-services fraud theory, which culminated in Skilling, presents a wonderful example of how criminal law ought not be made, whether viewed from an institutional, societal, or individual standpoint