Maximum Entropy Model Correction in Reinforcement Learning

We propose and theoretically analyze an approach for planning with an approximate model in reinforcement learning that can reduce the adverse impact of model error. If the model is accurate enough, it accelerates the convergence to the true value function too. One of its key components is the MaxEnt Model Correction (MoCo) procedure that corrects the model's next-state distributions based on a Maximum Entropy density estimation formulation. Based on MoCo, we introduce the Model Correcting Value Iteration (MoCoVI) algorithm, and its sampled-based variant MoCoDyna. We show that MoCoVI and MoCoDyna's convergence can be much faster than the conventional model-free algorithms. Unlike traditional model-based algorithms, MoCoVI and MoCoDyna effectively utilize an approximate model and still converge to the correct value function

Operator Splitting Value Iteration

We introduce new planning and reinforcement learning algorithms for discounted MDPs that utilize an approximate model of the environment to accelerate the convergence of the value function. Inspired by the splitting approach in numerical linear algebra, we introduce Operator Splitting Value Iteration (OS-VI) for both Policy Evaluation and Control problems. OS-VI achieves a much faster convergence rate when the model is accurate enough. We also introduce a sample-based version of the algorithm called OS-Dyna. Unlike the traditional Dyna architecture, OS-Dyna still converges to the correct value function in presence of model approximation error.Comment: Accepted to NeurIPS202

Effect of excessive Arm Swing on Speed and Cadence of walking

Introduction: One of the changes in the movement patterns that can be seen in upper limb swing is the excessive increase in upper limb movement and swing during walking. As temporal parameters such as cadence and speed in stationary and mobile environments can be equally used to determine early fall potentials, Therefore, this study aims to investigate the effect of excessive arm swing on speed and cadence of walking. Material and Methods: 30 healthy subjects were exposed to Vicon 10 motion capture system analysis and were asked to first walk normally at normal speeds and then move their hands excessively while walking at the same speed. The temporal data were extracted and analyzed by Matlab software. Descriptive (mean, SD) and Shapiro-Wilk test for normality of data distribution, and paired sample t-test were used to compare the patterns. Results: there was a significant difference in cadence and speed variables, between the means of natural arm swing and excessive arm swing modes (p ≤ 0.05). Conclusion: Given these results, it should be considered that the effects of upper limb pattern changes on the lower limbs and gait can compensate for the lack of attention to movement and pattern of upper extremity positioning during walking

Human Gait Database for Normal Walk Collected by Smart Phone Accelerometer

The goal of this study is to introduce a comprehensive gait database of 93 human subjects who walked between two endpoints during two different sessions and record their gait data using two smartphones, one was attached to the right thigh and another one on the left side of the waist. This data is collected with the intention to be utilized by a deep learning-based method which requires enough time points. The metadata including age, gender, smoking, daily exercise time, height, and weight of an individual is recorded. this data set is publicly available

Pan-cancer classifications of tumor histological images using deep learning

Histopathological images are essential for the diagnosis of cancer type and selection of optimal treatment. However, the current clinical process of manual inspection of images is time consuming and prone to intra- and inter-observer variability. Here we show that key aspects of cancer image analysis can be performed by deep convolutional neural networks (CNNs) across a wide spectrum of cancer types. In particular, we implement CNN architectures based on Google Inception v3 transfer learning to analyze 27815 H&E slides from 23 cohorts in The Cancer Genome Atlas in studies of tumor/normal status, cancer subtype, and mutation status. For 19 solid cancer types we are able to classify tumor/normal status of whole slide images with extremely high AUCs (0.995±0.008). We are also able to classify cancer subtypes within 10 tissue types with AUC values well above random expectations (micro-average 0.87±0.1). We then perform a cross-classification analysis of tumor/normal status across tumor types. We find that classifiers trained on one type are often effective in distinguishing tumor from normal in other cancer types, with the relationships among classifiers matching known cancer tissue relationships. For the more challenging problem of mutational status, we are able to classify TP53 mutations in three cancer types with AUCs from 0.65-0.80 using a fully-trained CNN, and with similar cross-classification accuracy across tissues. These studies demonstrate the power of CNNs for not only classifying histopathological images in diverse cancer types, but also for revealing shared biology between tumors. We have made software available at: https://github.com/javadnoorb/HistCNNFirst author draf

A Comparative Study of Abu Yaqube Sejestani and Roman Jakobson’s Communication model and Theory

Every human being needs to send, receive and comprehend messages in a linguistic communication. In other words, Human beings code their massages and decode others’ messages. This communication can occur in a context of six elements, namely, addresser, content, contact, code, message and addressee. Abu Ya'qub Sejestani was an iranian dialectical theologian in the fourth century. Due to his research subject, namely, Ismaili dialectical theology, he studied precisely the mind and language system and the conditions of success or failure in a linguistic communication as an introduction to dialectical discussions. But, unfortunately, his linguistic theories were neglected due to researchers’ dialectical approach to his book Kashf-ul-Mahjoob. This study is going to plan an efficient communication model based on mind and language system and its structure and elements in Sejestani’s viewpoint and compare them with Roman Jakobson’s theory and model of communication to show their common and different aspects and prove that although the discussions about the theory and model of communication became known by  western linguists and, most importantly, by Roman Jacobson, it can be traced in all details and with equivalent terms in Abu Ya'qub Sejestani’s communication theory, which has some advantages over Jakobson’s theory and model

Optimizing title and Meta tags based on distribution of keywords; Lexical and semantic approaches

Problem statement: To increase traffic on websites, Search Engine Optimization (SEO) has provided many costly and time-consuming options. One problem is the inadequate distribution of keywords especially those keywords that users use the title tag and Meta tags. Approach: This study described work on an initial model for handling some of the SEO factors to increase the distribution of keywords. Our purposed model provide users with the words and their values based on the key weights with initiated formula to provide a new title, keywords, or description in order to increase the relativity between content and HTML Meta tags and title tag. Results: The proposed model had been showed evidence of gaining the greater utilization of the distribution of keywords and prevents recognition of search engine spam. Conclusion: The result shows the significant enhancement of the proposed model on Title Weight by 51.69% of original Title Weight defined by user