Reinforcement Learning in Deep Structured Teams: Initial Results with Finite and Infinite Valued Features

In this paper, we consider Markov chain and linear quadratic models for deep structured teams with discounted and time-average cost functions under two non-classical information structures, namely, deep state sharing and no sharing. In deep structured teams, agents are coupled in dynamics and cost functions through deep state, where deep state refers to a set of orthogonal linear regressions of the states. In this article, we consider a homogeneous linear regression for Markov chain models (i.e., empirical distribution of states) and a few orthonormal linear regressions for linear quadratic models (i.e., weighted average of states). Some planning algorithms are developed for the case when the model is known, and some reinforcement learning algorithms are proposed for the case when the model is not known completely. The convergence of two model-free (reinforcement learning) algorithms, one for Markov chain models and one for linear quadratic models, is established. The results are then applied to a smart grid.Comment: This version corrects some typographical error

Strategic I/O Psychology and the Role of Utility Analysis Models

In the 1990’s, the significance of human capital in organizations has been increasing,and measurement issues in human resource management have achieved significant prominence. Yet, I/O psychology research on utility analysis and measurement has actually declined. In this chapter we propose a decision-based framework to review developments in utility analysis research since 1991, and show that through lens of this framework there are many fertile avenues for research. We then show that both I/O psychology and strategic HRM research and practice can be enhanced by greater collaboration and integration, particularly regarding the link between human capital and organizational success. We present an integrative framework as the basis for that integration, and illustrate its implications for future research

Applied Analysis and Synthesis of Complex Systems: Proceedings of the IIASA-Kyoto University Joint Seminar, June 28-29, 2004

This two-day seminar aimed at introducing the new development of the COE by Kyoto University to IIASA and discussing general modeling methodologies for complex systems consisting of many elements, mostly via nonlinear, large-scale interactions. We aimed at clarifying fundamental principles in complex phenomena as well as utilizing and synthesizing the knowledge derived out of them. The 21st Century COE (Center of Excellence) Program is an initiative by the Japanese Ministry of Education, Culture, Science and Technology (MEXT) to support universities establishing discipline-specific international centers for education and research, and to enhance the universities to be the world's apex of excellence with international competitiveness in the specific research areas. Our program of "Research and Education on Complex Functional Mechanical Systems" is successfully selected to be awarded the fund for carrying out new research and education as Centers of Excellence in the field of mechanical engineering in 2003 (five-year project), and is expected to lead Japanese research and education, and endeavor to be the top in the world. The program covers general backgrounds in diverse fields as well as a more in-depth grasp of specific branches such as complex system modeling and analysis of the problems including: nonlinear dynamics, micro-mesoscopic physics, turbulent transport phenomena, atmosphere-ocean systems, robots, human-system interactions, and behaviors of nano-composites and biomaterials. Fundamentals of those complex functional mechanical systems are macroscopic phenomena of complex systems consisting of microscopic elements, mostly via nonlinear, large-scale interactions, which typically present collective behavior such as self-organization, pattern formation, etc. Such phenomena can be observed or created in every aspect of modern technologies. Especially, we are focusing upon; turbulent transport phenomena in climate modeling, dynamical and chaotic behaviors in control systems and human-machine systems, and behaviors of mechanical materials with complex structures. As a partial attainment of this program, IIASA and Kyoto University have exchanged Consortia Agreement at the beginning of the program in 2003, and this seminar was held to introduce the outline of the COE program of Kyoto University to IIASA researchers and to deepen the shared understandings on novel complex system modeling and analysis, including novel climate modeling and carbonic cycle management, through joint academic activities by mechanical engineers and system engineers. In this seminar, we invited a distinguished researcher in Europe as a keynote speaker and our works attained so far in the project were be presented by the core members of the project as well as by the other contributing members who participated in the project. All IIASA research staff and participants of YSSP (Young Scientist Summer Program) were cordially invited to attend this seminar to discuss general modeling methodologies for complex systems