Transferring knowledge as heuristics in reinforcement learning: A case-based approach

The goal of this paper is to propose and analyse a transfer learning meta-algorithm that allows the implementation of distinct methods using heuristics to accelerate a Reinforcement Learning procedure in one domain (the target) that are obtained from another (simpler) domain (the source domain). This meta-algorithm works in three stages: first, it uses a Reinforcement Learning step to learn a task on the source domain, storing the knowledge thus obtained in a case base; second, it does an unsupervised mapping of the source-domain actions to the target-domain actions; and, third, the case base obtained in the first stage is used as heuristics to speed up the learning process in the target domain. A set of empirical evaluations were conducted in two target domains: the 3D mountain car (using a learned case base from a 2D simulation) and stability learning for a humanoid robot in the Robocup 3D Soccer Simulator (that uses knowledge learned from the Acrobot domain). The results attest that our transfer learning algorithm outperforms recent heuristically-accelerated reinforcement learning and transfer learning algorithms. © 2015 Elsevier B.V.Luiz Celiberto Jr. and Reinaldo Bianchi acknowledge the support of FAPESP (grants 2012/14010-5 and 2011/19280-8). Paulo E. Santos acknowledges support from FAPESP (grant 2012/04089-3) and CNPq (grant PQ2 -303331/2011-9).Peer Reviewe

Military Facility Cost Estimation System Using Case-Based Reasoning in Korea

This manuscript was submitted on January 22, 2010; approved on July 30, 2010; published online on August 3, 2010. Discussion period open until October 1, 2011; separate discussions must be submitted for individual papers.Numerous cost estimations are made repetitively in the initial stages of construction projects in response to ongoing scope changes and often need to be recalculated frequently. In practice, the square foot method, considered an effective method for time-saving, is widely used. However, this method requires a great amount of effort to calculate a unit price and does not consider the uniqueness of each case. Thus, the use of the square foot method could bring about unwanted consequences. For example, in the case of military projects in Korea, significant differences have been reported between estimations made using this method and the actual costs. In an effort to deal with this challenging issue, this research develops a military facility cost estimation (MilFaCE) system, based on case-based reasoning (CBR), using case data from 422 construction projects at 16 military facilities. Based on system validation experiments involving 10 military officers (engineers), the effectiveness of the system in terms of estimation accuracy and user-friendliness is confirmed. Consequently, this research can be a CBR application example of construction cost estimation and a basis for further research into the development of cost estimate systems. DOI: 10.1061/(ASCE)CP.1943-5487.0000082. (C) 2011 American Society of Civil Engineers.This research was supported by grants (R&D06CIT-A03 and 05CIT-01) from the Korea Ministry of Land, Transport, and Marine Affairs and the Ministry of Defense.

Analiza i predviđanje toka vremenskih serija pomoću “Case-BasedReasoning” tehnologije.

This thesis describes one promising approach where a problem of time series analysis and prediction was solved by using Case Based Reasoning (CBR) technology. Foundations and main concepts of this technology are described in detail. Furthermore, a detailed study of different approaches in time series analysis is given. System CuBaGe (Curve Base Generator) - A robust and general architecture for curve representation and indexing time series databases, based on Case based reasoning technology, was developed. Also, a corresponding similarity measure was modelled for a given kind of curve representation. The presented architecture may be employed equally well not only in conventional time series (where all values are known), but also in some non-standard time series (sparse, vague, non-equidistant). Dealing with the non-standard time series is the highest advantage of the presented architecture.U ovoj doktorskoj disertaciji prikazan je interesantan i perspektivan pristup rešavanja problema analize i predviđanja vremenskih serija korišćenjem Case Based Reasoning (CBR) tehnologije. Detaljno su opisane osnove i glavni koncepti ove tehnologije. Takođe, data je komparativna analiza različitih pristupa u analizi vremenskih serija sa posebnim osvrtom na predviđanje. Kao najveći doprinos ove disertacije, prikazan je sistem CuBaGe (Curve Base Generator) u kome je realizovan originalni način reprezentacije vremenskih serija zajedno sa, takođe originalnom, odgovarajućom merom sličnosti. Robusnost i generalnost sistema ilustrovana je realnom primenom u domenu finansijskog predviđanja, gde je pokazano da sistem jednako dobro funkcioniše sa standardnim, ali i sa nekim nestandardnim vremenskim serijama (neodređenim, retkim i neekvidistantnim)