Institutional Expenditures and State Economic Factors Influencing 2012-2014 Public University Graduation Rates

American higher education has seen public postsecondary funding sharply decline over the past couple of decades and has now fallen behind other countries in being the world leader in college degree production. Many U.S. states have begun to place more accountability on their public institutions to prove they are using appropriations as effectively and efficiently as possible. State financial support is increasingly being appropriated on the basis of performance – i.e. student outcomes, primarily measured by student graduation rates. The better an institution can use its financial resources to increase its graduation rates, the more state financial support it will likely receive. Yet, even as tracking graduation rates has grown in importance, linkages between graduation rates and institutional spending has not been extensively researched in public higher education. The purpose of this study was to analyze the relationship between institutional expenditures and graduation rates in public higher education institutions – when accounting for both institutional and state level differences. Results of this research may inform methods for adjusting institutional expenses to optimally affect undergraduate student graduation rates. The study examined institutional and state economic characteristics during the first academic years of the 2012, 2013, and 2014 six-year graduating cohorts. Thus, 2006-2008 data were obtained from IPEDS for 560 public institutions, including the input variables of institutional expenditures, student enrollment demographics, ACT scores, and Carnegie classifications, as well as the study’s dependent variable: 2012-2014 six-year graduation rates. State economic indicators of average household income and unemployment rates for the 2006-2008 time frame were obtained from U.S. Bureau of Labor Statistics. Multilevel modeling regression statistics were used to find any significant effects on graduation rates from these institutional and state-level data. The study revealed that instructional expenditures per student FTE had a significant effect on graduation rates, when controlling for other institutional and state level factors. Institutional characteristics, such as enrollment intensity, proportion of minority students, and ACT scores, had significant associations with graduation rates. To a lesser extent, state level economic factors were also found to have associations, particularly average household income, and, interestingly, higher education spending per capita

Learning Adaptive Display Exposure for Real-Time Advertising

In E-commerce advertising, where product recommendations and product ads are presented to users simultaneously, the traditional setting is to display ads at fixed positions. However, under such a setting, the advertising system loses the flexibility to control the number and positions of ads, resulting in sub-optimal platform revenue and user experience. Consequently, major e-commerce platforms (e.g., Taobao.com) have begun to consider more flexible ways to display ads. In this paper, we investigate the problem of advertising with adaptive exposure: can we dynamically determine the number and positions of ads for each user visit under certain business constraints so that the platform revenue can be increased? More specifically, we consider two types of constraints: request-level constraint ensures user experience for each user visit, and platform-level constraint controls the overall platform monetization rate. We model this problem as a Constrained Markov Decision Process with per-state constraint (psCMDP) and propose a constrained two-level reinforcement learning approach to decompose the original problem into two relatively independent sub-problems. To accelerate policy learning, we also devise a constrained hindsight experience replay mechanism. Experimental evaluations on industry-scale real-world datasets demonstrate the merits of our approach in both obtaining higher revenue under the constraints and the effectiveness of the constrained hindsight experience replay mechanism.Comment: accepted by CIKM201

Deep Q-learning from Demonstrations

Deep reinforcement learning (RL) has achieved several high profile successes in difficult decision-making problems. However, these algorithms typically require a huge amount of data before they reach reasonable performance. In fact, their performance during learning can be extremely poor. This may be acceptable for a simulator, but it severely limits the applicability of deep RL to many real-world tasks, where the agent must learn in the real environment. In this paper we study a setting where the agent may access data from previous control of the system. We present an algorithm, Deep Q-learning from Demonstrations (DQfD), that leverages small sets of demonstration data to massively accelerate the learning process even from relatively small amounts of demonstration data and is able to automatically assess the necessary ratio of demonstration data while learning thanks to a prioritized replay mechanism. DQfD works by combining temporal difference updates with supervised classification of the demonstrator's actions. We show that DQfD has better initial performance than Prioritized Dueling Double Deep Q-Networks (PDD DQN) as it starts with better scores on the first million steps on 41 of 42 games and on average it takes PDD DQN 83 million steps to catch up to DQfD's performance. DQfD learns to out-perform the best demonstration given in 14 of 42 games. In addition, DQfD leverages human demonstrations to achieve state-of-the-art results for 11 games. Finally, we show that DQfD performs better than three related algorithms for incorporating demonstration data into DQN.Comment: Published at AAAI 2018. Previously on arxiv as "Learning from Demonstrations for Real World Reinforcement Learning