A new Potential-Based Reward Shaping for Reinforcement Learning Agent

Potential-based reward shaping (PBRS) is a particular category of machine learning methods which aims to improve the learning speed of a reinforcement learning agent by extracting and utilizing extra knowledge while performing a task. There are two steps in the process of transfer learning: extracting knowledge from previously learned tasks and transferring that knowledge to use it in a target task. The latter step is well discussed in the literature with various methods being proposed for it, while the former has been explored less. With this in mind, the type of knowledge that is transmitted is very important and can lead to considerable improvement. Among the literature of both the transfer learning and the potential-based reward shaping, a subject that has never been addressed is the knowledge gathered during the learning process itself. In this paper, we presented a novel potential-based reward shaping method that attempted to extract knowledge from the learning process. The proposed method extracts knowledge from episodes' cumulative rewards. The proposed method has been evaluated in the Arcade learning environment and the results indicate an improvement in the learning process in both the single-task and the multi-task reinforcement learner agents

Comparing different transit strategies to tackle the last-mile issue in low demand areas Case study: York Region Transit

Providing public transit service in low-density suburban areas is very challenging and inefficient because development patterns and transit demand do not support regular scheduled bus services while flexible and on-demand service is very expensive to provide. A further issue is that effective public transit is essential for providing equal access to opportunities for the residents of these areas. This is a controversial issue in most Canadian cities where they have difficulties in providing sustainable public transit. Building upon the knowledge gained from an overview of the literature, this study aims at contributing to a better understanding of the crucial factors that influence the performance of public transit in low-density areas and develops a framework for evaluating different strategies for providing first/last mile transit service. In order to accomplish this goal the literature of transit system performance measures as well as transit mode choice are reviewed and 7 major criteria are selected: safety & security, cost, time, flexibility, comfort, coverage, and availability of information. Secondly, a systematic literature review is conducted to identify different strategies that can be implemented as a last mile solution in low density areas. Employing the seven criteria in an evaluation framework, these possible strategies are explained and compared. A case study using real data from York Region Transit (YRT) were utilized for comparing the two most common on-demand last mile strategies in the region. Results showed that outsourcing transit rides to instant ride-hailing companies –e.g. Uber- is financially beneficial to YRT and offers more coverage for potential riders, providing that reliability of their service is secured

Word-level Persian Lipreading Dataset

Lip-reading has made impressive progress in recent years, driven by advances in deep learning. Nonetheless, the prerequisite such advances is a suitable dataset. This paper provides a new in-the-wild dataset for Persian word-level lipreading containing 244,000 videos from approximately 1,800 speakers. We evaluated the state-of-the-art method in this field and used a novel approach for word-level lip-reading. In this method, we used the AV-HuBERT model for feature extraction and obtained significantly better performance on our dataset

Subcutaneous Reaction of Rat Tissues to Nanosilver Coated Gutta-Percha

Introduction: Gutta-percha (GP), is a neutral and non-toxic material. The aim of this animal study was to compare the biocompatibility of nanosilver coated GP (NS-GP) with conventional GP in subcutaneous tissues in a rat model. Methods and Materials: Conventional GP and NS-GP were subcutaneously implanted in the backs of 20 male Wistar rats (n=10). A control animal was assigned for each trial period. Ten animals were sacrificed after 7 and 30 days and light microscopic evaluation of tissue reaction to NS-GP (n=20) and conventional GP (n=20) was accomplished. The Mann-Whitney U, Wilcoxon Signed Ranks, Fisher Exact, and McNemar tests were used for statistical analysis of the data. Results: After 7 days, inflammation was moderate and mild for NS-GP and conventional GP, respectively (P<0.001). After 30 days, no inflammation was discernible in conventional GP. However, mild inflammation was reported for NS-GP (P<0.001). Regarding inflammatory cell type, there was a significant difference between two experimental groups at both times (P<0.001). Conclusion: Inflammation decreased over time in both groups. Fibrous connective tissue, a representative of healing and control of inflammatory process, surrounded both test materials. NS-GP was biocompatible and might be a reasonable endodontic obturation material.Keywords: Gutta-Percha; Inflammation; Nanosilver Coated Gutta-Percha; Subcutaneous Connective Tissue

A Design Support System Using Analogy Based Reasoning

Abstract: This paper represents a procedure to support the designer in his/her process of mechanical system design, by inspiring the knowledge acquired from previous projects. To this end, the proposed method represents an appropriate means to capitalize the know-how of the professional experts. Based on this approach, an interactive programme is implemented, which assist designers in the specification of new products. The data structure of the implemented tool is based on the object oriented modelling. This structure allows several classifications of a same design, using different levels of abstraction. This approach enables designer to begin with a more general description of the product, and to refine the description by referring to similar data in the pattern bases

Academic Performance and Friendship Relation in Students: Role of Personality

Abstract: There have been many studies about personality role in students’ studies in recent years. However, these studies do not provide a comprehensive model. In addition, statistical researchers have just attended the relationship between each individual’s personality and his academic performance without considering the impact of people around him on his educationalachievements. In this research, a social simulation has been propounded which shows the impact of an individual’s personality and other people around him in his academic performance using multi-agent systems. Each individual in the simulation has five personality factors which came from the theory of “big five personality”. The simulation is prepared as a tool with a user interface to set parameters and see the result of each simulation. The parameters have been considered by experts to be replaced during the test phase. Data used for simulation was taken with the help of the “big five personality” questionnaire distributed among 35 second-year computer students in Iran University of Science and Technology. Reliability and validity of questions have been previously reviewed by experts. Simulation was executed on the data and the results had been prepared using each student’s personality and adding the friendship links between them. It has been shown that taking into account each student’s friends’ personality can improve the simulation results

Lithium interactions with non-steroidal anti-inflammatory drugs and diuretics – A review

Background: Lithium is often used in bipolar disorder and occasionally in unipolar depression. Non-steroidal anti-inflammatory drugs (NSAIDs) and diuretics are frequently prescribed and their interaction with lithium is based mainly in few small studies. Objectives: Conduct a review, identify different interaction patterns and discuss treatment options. Methods: Three searches were made in PubMed in January 2016: 1) using the keywords “lithium” [and] “non-steroidal anti-inflammatory”; 2) using the keywords “lithium” [and] “diuretics” and the filter “title/abstract”; 3) using the terms “lithium” [and] “toxicity” and the filters “title” [and] “review”. From the 293 remaining articles, 10 were selected. Another search in Scielo.org was made, using the term “lítio” and the filter “Psiquiatria”. Two articles were selected from the initial 53. Six textbooks were added to expand the evidence, achieving a total of 18 references. Results: The majority of NSAIDs and diuretics rises lithium levels, specially thiazides. However, some show great variability or no interaction at all, and others even decrease lithium levels. Discussion: Lower-doses, shorter durations, lithium adjustments and levels' follow-ups are recommended, especially in elderly and multiple co-morbid patients

The emergence of new psychoactive substance (NPS) benzodiazepines: a review

The market for new psychoactive substances has increased markedly in recent years and there is now a steady stream of compounds appearing every year. Benzodiazepines consist of only a fraction of the total number of these compounds but their use and misuse has rapidly increased. Some of these benzodiazepines have only been patented, some of them have not been previously synthesised and the majority have never undergone clinical trials or tests. Despite their structural and chemical similarity, large differences exist between the benzodiazepines in their pharmacokinetic parameters and metabolic pathways and so they are not easily comparable. As benzodiazepines have been clinically used since the 1960s many analytical methods exist to quantify them in a variety of biological matrices and it is expected that these methods would also be suitable for the detection of benzodiazepines that are new psychoactive substances. Illicitly obtained benzodiazepines have been found to contain a wide range of compounds such as opiates which presents a problem since the use of them in conjunction with each other can lead to respiratory depression and death. The aim of this review is to collate the available information on these benzodiazepines and to provide a starting point for the further investigation of their pharmacokinetics which is clearly required