A Policy Search Method For Temporal Logic Specified Reinforcement Learning Tasks

Reward engineering is an important aspect of reinforcement learning. Whether or not the user's intentions can be correctly encapsulated in the reward function can significantly impact the learning outcome. Current methods rely on manually crafted reward functions that often require parameter tuning to obtain the desired behavior. This operation can be expensive when exploration requires systems to interact with the physical world. In this paper, we explore the use of temporal logic (TL) to specify tasks in reinforcement learning. TL formula can be translated to a real-valued function that measures its level of satisfaction against a trajectory. We take advantage of this function and propose temporal logic policy search (TLPS), a model-free learning technique that finds a policy that satisfies the TL specification. A set of simulated experiments are conducted to evaluate the proposed approach

Gated networks: an inventory

Gated networks are networks that contain gating connections, in which the outputs of at least two neurons are multiplied. Initially, gated networks were used to learn relationships between two input sources, such as pixels from two images. More recently, they have been applied to learning activity recognition or multi-modal representations. The aims of this paper are threefold: 1) to explain the basic computations in gated networks to the non-expert, while adopting a standpoint that insists on their symmetric nature. 2) to serve as a quick reference guide to the recent literature, by providing an inventory of applications of these networks, as well as recent extensions to the basic architecture. 3) to suggest future research directions and applications.Comment: Unpublished manuscript, 17 page

Twisted algebra R-matrices and S-matrices for bn(1)b_n^{(1)} affine Toda solitons and their bound states

We construct new $U_q(a^{(2)}_{2n-1})$ and $U_q(e^{(2)}_6)$ invariant $R$-matrices and comment on the general construction of $R$-matrices for twisted algebras. We use the former to construct $S$-matrices for $b^{(1)}_n$ affine Toda solitons and their bound states, identifying the lowest breathers with the $b^{(1)}_n$ particles.Comment: Latex, 24 pages. Various misprints corrected. New section added clarifying relationship between R-matrices and S-matrice

The Reproductive Ecology of Industrial Societies, Part I Why Measuring Fertility Matters

Is fertility relevant to evolutionary analyses conducted in modern industrial societies? This question has been the subject of a highly contentious debate, beginning in the late 1980s and continuing to this day. Researchers in both evolutionary and social sciences have argued that the measurement of fitness-related traits (e.g., fertility) offers little insight into evolutionary processes, on the grounds that modern industrial environments differ so greatly from those of our ancestral past that our behavior can no longer be expected to be adaptive. In contrast, we argue that fertility measurements in industrial society are essential for a complete evolutionary analysis: in particular, such data can provide evidence for any putative adaptive mismatch between ancestral environments and those of the present day, and they can provide insight into the selection pressures currently operating on contemporary populations. Having made this positive case, we then go on to discuss some challenges of fertility-related analyses among industrialized populations, particularly those that involve large-scale databases. These include “researcher degrees of freedom” (i.e., the choices made about which variables to analyze and how) and the different biases that may exist in such data. Despite these concerns, large datasets from multiple populations represent an excellent opportunity to test evolutionary hypotheses in great detail, enriching the evolutionary understanding of human behavior

Análise da convergência da produtividade da mão-de-obra agropecuária entre os estados brasileiros: aplicação de matrizes de Markov, 1990-2000

The study analyzes the convergence of the labor productivity in the Brazilian agricultural sector in the nineties, comparing it with its productivity in the other sectors and projecting its future behavior through Markov matrices. The results indicate that the Brazilian states are diverging in the labor productivity in the agricultural sector. Some states will move to a high level and others to a low level of labor productivity. In five of the other sectors of the Brazilian economy there will be a convergence of the states with regard to the labor productivity. In three of these five sectors the states will converge to a low level of labor productivity; in one sector they will converge to two classes of low productivity and in another sector the states will move to a class of high productivity. The time required for convergence of the Brazilian states is very long; indicating that the economic factors prevailing during the period on which the analysis is based did not impact very strongly on this process.agricultural, labor productivity, convergence, Markov model, Agribusiness,

High and mighty : height increases authority in professional refereeing

Throughout the animal kingdom, larger males are more likely to attain social dominance. Several lines of evidence suggest that this relationship extends to humans, as height is positively related to dominance, status and authority. We hypothesized that height is also a determinant of authority in professional refereeing. According to the International Football Association Board, FIFA, football ("soccer") referees have full authority to enforce the laws of the game and should use their body language to show authority and to help control the match. We show that height is indeed positively related to authority status: referees were taller than their assistants (who merely have an advisory role) in both a national (French League) and an international (World Cup 2010) tournament. Furthermore, using data from the German League, we found that height was positively associated with authoritative behavior. Taller referees were better able to maintain control of the game by giving fewer fouls, thereby increasing the "flow of the game". Referee height was also positively associated with perceived referee competence, as taller referees were assigned to matches in which the visiting team had a higher ranking. Thus, height appears to be positively related to authority in professional refereeing

Sex Differences in Risk Taking Behavior among Dutch Cyclists

The majority of research examining sex differences in risk-taking behavior focuses on overt physical risk measures in which failed risk attempts may result in serious injury or death. The present research describes sex differences in patterns of risk taking in day-to-day behavior among Dutch cyclists. Through three observational studies we test sex differences in risk taking in situations of financial risk (fines for failing to use bike lights, Study 1), theft risk (bike locking behavior, Study 2) as well as physical risk (risky maneuvers, Study 3). Results corroborate previous findings by showing that across these domains men are more inclined to take risks than women. We discuss how these findings might be used in an applied context

Assortative mating for human height: A meta-analysis

OBJECTIVES: The study of assortative mating for height has a rich history in human biology. Although the positive correlation between the stature of spouses has often been noted in western populations, recent papers suggest that mating patterns for stature are not universal. The objective of this paper was to review the published evidence to examine the strength of and universality in assortative mating for height. METHODS: We conducted an extensive literature review and meta-analysis. We started with published reviews but also searched through secondary databases. Our search led to 154 correlations of height between partners. We classified the populations as western and non-western based on geography. These correlations were then analyzed via meta-analytic techniques. RESULTS: 148 of the correlations for partner heights were positive and the overall analysis indicates moderate positive assortative mating (r = .23). Although assortative mating was slightly stronger in countries that can be described as western compared to non-western, this difference was not statistically significant. We found no evidence for a change in assortative mating for height over time. There was substantial residual heterogeneity in effect sizes and this heterogeneity was most pronounced in western countries. CONCLUSIONS: Positive assortative mating for height exists in human populations, but is modest in magnitude suggesting that height is not a major factor in mate choice. Future research is necessary to understand the underlying causes of the large amount of heterogeneity observed in the degree of assortative mating across human populations, which may stem from a combination of methodological and ecological differences