Reinforcement Learning: A Survey

This paper surveys the field of reinforcement learning from a computer-science perspective. It is written to be accessible to researchers familiar with machine learning. Both the historical basis of the field and a broad selection of current work are summarized. Reinforcement learning is the problem faced by an agent that learns behavior through trial-and-error interactions with a dynamic environment. The work described here has a resemblance to work in psychology, but differs considerably in the details and in the use of the word ``reinforcement.'' The paper discusses central issues of reinforcement learning, including trading off exploration and exploitation, establishing the foundations of the field via Markov decision theory, learning from delayed reinforcement, constructing empirical models to accelerate learning, making use of generalization and hierarchy, and coping with hidden state. It concludes with a survey of some implemented systems and an assessment of the practical utility of current methods for reinforcement learning.Comment: See http://www.jair.org/ for any accompanying file

Making financial markets safer for consumers: lessons from consumer goods markets and beyond

In the wake of the mortgage meltdown, policymakers are discussing how best to protect consumers in financial product markets.Consumer protection ; Financial markets

Predicting Sense Of Community in a Historic Latino/Latina Neighborhood Undergoing Gentrification

Neighborhoods with generational Mexican American populations may have high levels of block Social Cohesion and neighborhood Sense of Community. Streetcar-focused development via federal and local investment often spurs gentrification in neighborhoods with ethnic concentrations, which shifts neighborhood demographics towards more White and higher income households. The new residential and business investment in the neighborhood often has an impact on existing neighborhood social dynamics. This study includes mixed methods resident survey data of long term and newer residents. The qualitative data analysis informs quantitative data analysis in order to better understand resident descriptions of the impact of neighborhood streetcar focused gentrification on social factors in a generational Latino/Latina neighborhood at one point in time just before the streetcar opening. Specifically the study seeks to: (a) provide a description of generational and new resident experiences with block Social Cohesion and neighborhood Sense of Community; (b) determine differences (between Latino/Latina households and those with children present and other study participants) in block Social Cohesion, neighborhood Sense of Community, and Involvement in Neighborhood and Voluntary Associations; and (c) determine what factors predict neighborhood Sense of Community. The study highlights the Latino/Latina residents’ maintenance of a strong ethnic identity, generational neighborhood based social ties, and ongoing involvement in neighborhood schools and religious traditions that contribute to a strong neighborhood Sense of Community. Newer residents report being drawn to and supporting the maintenance of the neighborhood Sense of Community

Regenerative Architecture: A Pathway Beyond Sustainability

The current paradigm in the field of architecture today is one of degeneration and obsolete building technologies. Regenerative architecture is the practice of engaging the natural world as the medium for, and generator of the architecture. It responds to and utilizes the living and natural systems that exist on a site that become the “building blocks” of the architecture. Regenerative architecture has two focuses; it is an architecture that focuses on conservation and performance through a focused reduction on the environmental impacts of a building. This paper introduces regenerative architecture as a means for architectural design. I present the Nine Principles of Regenerative Architecture and Place Analysis Criteria, which I developed in order to provide a logical and succinct means for creating regenerative architecture. These are employed and embedded in the creation of the R_Urban Intervention Dwelling model and tested on the Coop House design project. The result was an architectural design in which the Nine Principles of Regenerative Architecture are embodied through the application of the Place Analysis Criteria process. Though the process underwent many mutations through its infancy, the final product has proven to work in producing successful and potentially regenerative architecture as described in part 1 of this paper

The T cell antigen receptor complex expressed on normal peripheral blood CD4-, CD8- T lymphocytes. A CD3-associated disulfide-linked gamma chain heterodimer.

IL-2-dependent cell lines were established from normal peripheral blood T lymphocytes that express neither CD4 nor CD8 differentiation antigens. CD3+,4-,8- cell lines from 15 different donors failed to react with WT31, an mAb directed against the T cell antigen receptor alpha/beta heterodimer. Anti-Leu-4 mAb was used to isolate the CD3/T cell antigen receptor complex from 125I-labeled CD3+,4-,8- (WT31-) T cells. Using detergent conditions that preserved the CD3/T cell antigen receptor complex, an approximately 90 kD disulfide-linked heterodimer, composed of approximately 45- and approximately 40- (or approximately 37-) kD subunits, was coimmunoprecipitated with the invariant 20-29-kD CD3 complex. Analysis of these components by nonequilibrium pH gradient electrophoresis indicated that the approximately 40-kD and approximately 37-kD subunits were similar, and quite distinct from the more basic approximately 45-kD subunit. None of these three subunits reacted with an antibody directed against a beta chain framework epitope. Heteroantiserum against a T cell receptor gamma chain peptide specifically reacted with both the approximately 37- and approximately 40-kD CD3-associated proteins, but not with the approximately 45-kD subunit. CD3+,4-,8- cells failed to transcribe substantial amounts of functional 1.3-kb beta or 1.6-kb alpha mRNA, but produced abundant 1.6-kb gamma mRNA. Southern blot analysis revealed that these CD3+,4-,8- cell lines rearranged both gamma and beta genes, and indicated that the populations were polyclonal. The expression of a CD3-associated disulfide-linked heterodimer on CD3+,4-,8- T cell lines established from normal, adult peripheral blood contrasts with prior reports describing a CD3-associated non-disulfide-linked heterodimer on CD3+/WT31- cell lines established from thymus and peripheral blood obtained from patients with immunodeficiency diseases. We propose that this discrepancy may be explained by preferential usage of the two C gamma genes in T lymphocytes

Hall-Effect for Neutral Atoms

It is shown that polarizable neutral systems can drift in crossed magnetic and electric fileds. The drift velocity is perpendicular to both fields, but contrary to the drif t velocity of a charged particle, it exists only, if fields vary in space or in time. We develop an adiabatic theory of this phenomenon and analyze conditions of its experimental observation. The most proper objects for the observation of this effect are Rydberg atoms. It can be applied for the separation of excited atoms.Comment: RevTex, 4 pages; to be published in Pis'ma v ZhET

Places to Swim: Perspectives Report

People love to recreate around, on and in the water. As part of the Department of Planning and Open Spaces Program, Places to Swim provides an opportunity to improve access to natural waterways for recreation across NSW. This report investigates the issues, barriers and benefits associated with opening waterways for recreation. NSW is enriched with a range of beautiful and healthy waterways providing opportunities for people to swim and recreate safely, create places that people can visit, and help build better communities. A key attribute for all swimming sites is ensuring they are safe to use. Recreation involving waterways inherently involves risks, including exposure to waterborne contaminants and the risk of injury and drowning. As new swimming sites are opened the risks need to be identified, monitored, and managed

Associations of herbal and specialty supplements with lung and colorectal cancer risk in the VITamins and Lifestyle study.

Millions of Americans use dietary supplements with little knowledge about their benefits or risks. We examined associations of various herbal/specialty supplements with lung and colorectal cancer risk. Men and women, 50 to 76 years, in the VITamins And Lifestyle cohort completed a 24-page baseline questionnaire that captured duration (years) and frequency (days per week) of use of commonly used herbal/specialty supplements. Dose was not assessed due to the lack of accurate potency information. Supplement exposure was categorized as "no use" or "any use" over the previous 10 years. Hazard ratios (HR) were estimated by multivariate Cox regression models. Incident lung (n = 665) and colorectal cancers (n = 428) were obtained from the Surveillance, Epidemiology, and End Results cancer registry. Any use of glucosamine and chondroitin, which have anti-inflammatory properties, over the previous 10 years, was associated with significantly lower lung cancer risk: HR 0.74 [95% confidence interval (95% CI), 0.58-0.94] and HR 0.72 (95% CI, 0.54-0.96) and colorectal cancer risk: HR 0.73 (95% CI, 0.54-0.98) and HR 0.65 (95% CI, 0.45-0.93), respectively. There were also statistically significantly inverse associations of fish oil: HR 0.65 (95% CI, 0.42-0.99), methylsulfonylmethane: HR 0.46 (95% CI, 0.23-0.93), and St. John's wort: HR 0.35 (95% CI, 0.14-0.85) with colorectal cancer risk. In contrast, garlic pills were associated with a statistically significant 35% elevated colorectal cancer risk. These results suggest that some herbal/specialty supplements may be associated with lung and colorectal cancer risk; however, these products should be used with caution. Additional studies examining the effects of herbal/specialty supplements on risk for cancer and other diseases are needed

Reinforced Axial Refinement Network for Monocular 3D Object Detection

Monocular 3D object detection aims to extract the 3D position and properties of objects from a 2D input image. This is an ill-posed problem with a major difficulty lying in the information loss by depth-agnostic cameras. Conventional approaches sample 3D bounding boxes from the space and infer the relationship between the target object and each of them, however, the probability of effective samples is relatively small in the 3D space. To improve the efficiency of sampling, we propose to start with an initial prediction and refine it gradually towards the ground truth, with only one 3d parameter changed in each step. This requires designing a policy which gets a reward after several steps, and thus we adopt reinforcement learning to optimize it. The proposed framework, Reinforced Axial Refinement Network (RAR-Net), serves as a post-processing stage which can be freely integrated into existing monocular 3D detection methods, and improve the performance on the KITTI dataset with small extra computational costs.Comment: Accepted by ECCV 202