Hierarchical Policy Search via Return-Weighted Density Estimation

Learning an optimal policy from a multi-modal reward function is a challenging problem in reinforcement learning (RL). Hierarchical RL (HRL) tackles this problem by learning a hierarchical policy, where multiple option policies are in charge of different strategies corresponding to modes of a reward function and a gating policy selects the best option for a given context. Although HRL has been demonstrated to be promising, current state-of-the-art methods cannot still perform well in complex real-world problems due to the difficulty of identifying modes of the reward function. In this paper, we propose a novel method called hierarchical policy search via return-weighted density estimation (HPSDE), which can efficiently identify the modes through density estimation with return-weighted importance sampling. Our proposed method finds option policies corresponding to the modes of the return function and automatically determines the number and the location of option policies, which significantly reduces the burden of hyper-parameters tuning. Through experiments, we demonstrate that the proposed HPSDE successfully learns option policies corresponding to modes of the return function and that it can be successfully applied to a challenging motion planning problem of a redundant robotic manipulator.Comment: The 32nd AAAI Conference on Artificial Intelligence (AAAI 2018), 9 page

Common Value Auctions with Return Policies

This paper examines the role of return policies in common value auctions. We first characterize the unique symmetric equilibrium in first-price and second-price auctions with continuous signals and discrete common values when certain return policies are provided. We then examine how the return policies affect a seller's revenue. When the lowest common value is zero, a more generous return policy generates a higher seller's revenue; the full refund policy extracts all the surplus and therefore implements the optimal selling mechanism; given any return policy, a second-price auction generates a higher revenue than a first-price auction. In a second-price auction where the lowest common value is not zero but still smaller than the seller's reservation value, then a more generous return policy also generates a higher revenue; otherwise, the optimal return policy could be a full refund, no refund or partial refund policy.auctions, return policies, refund

Warranty return policies for products with unknown claim causes and their optimisation

In practical warranty services management, faults may not always be found in claimed items by warranty service agents, which is the well-known no-fault found phenomenon (for example, caused by a loose connection between parts, or simply human error). This phenomenon can contribute more than 40% of reported service faults in electronic products and it can be due to faults of manufacturers or product users. Little research, however, considers this phenomenon in warranty management since faults are normally assumed to be found in the claimed items. On the basis of different levels of testing, this paper proposes three warranty return policies, which decide whether new items should be sent to warranty claimants or not. It then derives and compares the expected costs of the policies, and obtains the optimal warranty periods under supply chain environments. The paper illustrates the results with artificially generated data

Maximizing Quality and Value in Medicaid: Using Return on Investment Forecasting to Support Effective Policymaking

Outlines how forecasting the return on investment of quality measures enhances the efficient allocation of resources, adoption of evidence-based policies, and payment reform. Presents examples from states in an ROI Forecasting Calculator pilot program

Enhancing the Public Provision of Education: The Economics of Education Reform in Developing Countries

The paper argues that a comprehensive evaluation of education reform in particular in developing countries needs considering the triangle’ quality-quantity-equity of educational policies in the short, medium and long term in a broader context than the education system itself. There is no simple “recipe” for improving quality and internal and external efficiency in the public education system but some general results are found. Firstly, that the elasticity of the return of the reform is decreasing with the size of increased budget, making anti-economical the reliance on a reform consisting in more resources only to significantly improve the poor performance of the system. Indeed, very modest target set to improve the system performance, would require -without more sophisticated policies- huge increments in budget with a poor return. In this sense the paper investigate the capacity of focused policies to improve the productivity of the education expenditure, in particular toward basic education or the disadvantaged students. Secondly, the timing of the reform matters: most policies with very different return in the long term are almost undistinguishable by their short run merits, and policies that are more productive in the short term may be less convenient than competing alternatives in the longer term, so the actual policy may be influenced by the time horizon chosen by the policy makers. Thirdly, effects of the reform are accumulative, and to evaluate the reform by modest, in general, short run merits is myopic and may put the reform at risk of reversion or to deter future investment in the sector.public education, developing countries, development of human resources

International Research Project on Job Retention and Return to Work Strategies for Disabled Workers: Key Issues

[From Preface] The International Research Project on Job Retention and Return to Work Strategies for Disabled Workers breaks new ground by examining the inter-relationships of public and enterprise policies and practices as they affect the retention and return to work of disabled workers. The enquiry encompasses public policies to promote employment of disabled people; benefit and compensation programmes; employment support and rehabilitation services; provision to adapt work and workplace; and measures developed and implemented by the enterprise. The Project aims not only to identify successful policies and practices which are transferable from one country to another but also to inform the development of effective, efficient and equitable job retention and return to work strategies for disabled workers. The ultimate objective is to develop strategies which can be put into effect in the workplace

Discrimination, labour markets and the Labour Market Prospects of Older Workers: What Can a Legal Case Teach us?

As governments become increasingly concerned about the fiscal implications of the ageing population, labour market policies have sought to encourage mature workers to remain in the labour force. The ‘human capital’ discourses motivating these policies rest on the assumption that older workers armed with motivation and vocational skills will be able to return to fulfilling work. This paper uses the post-redundancy recruitment experiences of former Ansett Airlines flight attendants to develop a critique of these expectations. It suggests that policies to increase older workers’ labour market participation will not succeed while persistent socially constructed age- and gender- typing shape labour demand. The conclusion argues for policies sensitive to the institutional structures that shape employer preferences, the competitive rationality of discriminatory practices, and the irresolvable tension between workers’ human rights and employers’ property rights

Maternal Labor Market Return, Parental Leave Policies, and Gender Inequality in Housework

This study investigates how the duration of the work interruption and the labor market status of mothers upon their return affect the division of housework in couples after a birth. By observing several parental leave policy reforms in Britain and West-Germany, this research also explores how extended leave entitlements for mothers influence the division of housework. The analysis uses multilevel multiprocess models for 1220 birth events of British couples and 1785 births to German couples based on data from the British Household Panel Survey (1991-2008) and the German Socio-Economic Panel (1985-2009). The results suggest that mothers increase their housework hours with every additional month of employment interruption. Mothers' full-time return seems more effective than a short labor market time-out in altering men's housework contributions and reducing the trend towards a more traditional division of housework. Parental leave policy extensions for mothers were associated with the division of housework only indirectly through their impact on the length of women's work interruptions.Parenthood, parental leave policy, maternal employment, housework, gender division of labor, Britain, Germany

Common value auctions with return policies

This paper examines the role of return policies in common value auctions. We first characterize the unique symmetric equilibrium in first-price and second-price auctions with continuous signals and discrete common values when certain return policies are provided. We then examine how the return policies affect a seller's revenue. When the lowest common value is zero, a more generous return policy generates a higher seller's revenue; the full refund policy extracts all the surplus and therefore implements the optimal selling mechanism; given any return policy, a second-price auction generates a higher revenue than a first-price auction. In a second-price auction where the lowest common value is not zero but still smaller than the seller's reservation value, then a more generous return policy also generates a higher revenue; otherwise, the optimal return policy could be a full refund, no refund or partial refund policy