Reinforcement Learning in Non-Markovian Environments

Following the novel paradigm developed by Van Roy and coauthors for reinforcement learning in arbitrary non-Markovian environments, we propose a related formulation inspired by classical stochastic control that reduces the problem to recursive computation of approximate sufficient statistics.Comment: 15 pages, submitted to Systems and Control Letter

The Effects of Word Of Mouth on Movies & Its Impact on the Audience's Choice

Consumers discuss products and services and their experiences with others and thus become indirect marketers. The only difference is that this marketing may not always be in the favour of the product/service. Thus, marketers need to learn how to use this method to their benefit, and this is only possible through thorough knowledge, practice and patience. Therefore, this study aims to highlight the concepts of word-of-mouth, in the context of services, in order to help marketers better understand the elements and characteristics as well as the influences of word-of-mouth on consumers' decision making process. This is done through the reviewing of literature which not only reveals the theory of word-of-mouth, but also taps onto the reasons as to why people share comments and experiences as well as why people listen to these. It also helped to develop a framework which was used as the base of the study. As a result of the framework, this study explores the impact of the audience's choice on the success of motion pictures/movies as a consequence of word-of-mouth. The research employed shows that viewers search for movie related information before making their purchase and highlights the factors that most influence their choice. It also enlightens on the fact that strong ties are important elements of the social network and their influence on one's choice is powerful. Furthermore, the types of word-of-mouth: negative and positive, affect the decision of the audience and thus have an overall impact on the income for the film in terms of the audience's attendance/box office returns. Overall, it supported the developed framework

Family practices' achievement of diabetes quality of care targets and risk of screen-detected diabetic retinopathy

Background: We aimed to determine whether family practices' achievement of diabetes quality of care targets is associated with diabetic retinal disease in registered patients. Methods: Data for achievement of diabetes quality of care targets, including the proportion of patients with HbA1c≤7.5%, for 144 family practices in London UK, for the years 2004/5 to 2007/8, were linked to data from a population-based diabetes eye screening programme collected from September 2007 to February 2009. Analyses were adjusted for age, sex, duration and type of diabetes, unadjusted diabetes prevalence, ethnicity and deprivation category. Results: Data were analysed for 24,458 participants with one or more eye screening results in the period. There were 9,332 (38%) with any diabetic retinopathy and 2,819 (11.5%) with sight threatening diabetic retinopathy (STDR), including 2,654 (10.9%) with maculopathy. Among participants registered at 13 family practices that were in the highest quartile for achievement of the HbA1c quality of care target for all four years of study, the relative odds of any diabetic retinopathy were 0.78 (0.69 to 0.88) P<0.001. For participants at 12 practices consistently in the lowest quartile of HbA1c achievement, the relative odds of any diabetic retinopathy were 1.16 (1.03 to 1.30), P = 0.015. In the highest achieving practices, the relative odds of maculopathy were 0.74 (0.62 to 0.89), P = 0.001 and STDR 0.77 (0.65 to 0.92), P = 0.004. Conclusions: The risk of diabetic retinopathy might be lower at family practices that consistently achieve highly on diabetes quality of care targets for HbA1c

Revisiting pre-trained remote sensing model benchmarks: resizing and normalization matters

Research in self-supervised learning (SSL) with natural images has progressed rapidly in recent years and is now increasingly being applied to and benchmarked with datasets containing remotely sensed imagery. A common benchmark case is to evaluate SSL pre-trained model embeddings on datasets of remotely sensed imagery with small patch sizes, e.g., 32x32 pixels, whereas standard SSL pre-training takes place with larger patch sizes, e.g., 224x224. Furthermore, pre-training methods tend to use different image normalization preprocessing steps depending on the dataset. In this paper, we show, across seven satellite and aerial imagery datasets of varying resolution, that by simply following the preprocessing steps used in pre-training (precisely, image sizing and normalization methods), one can achieve significant performance improvements when evaluating the extracted features on downstream tasks -- an important detail overlooked in previous work in this space. We show that by following these steps, ImageNet pre-training remains a competitive baseline for satellite imagery based transfer learning tasks -- for example we find that these steps give +32.28 to overall accuracy on the So2Sat random split dataset and +11.16 on the EuroSAT dataset. Finally, we report comprehensive benchmark results with a variety of simple baseline methods for each of the seven datasets, forming an initial benchmark suite for remote sensing imagery

Navigating the Web of Misinformation: A Framework for Misinformation Domain Detection Using Browser Traffic

The proliferation of misinformation and propaganda is a global challenge, with profound effects during major crises such as the COVID-19 pandemic and the Russian invasion of Ukraine. Understanding the spread of misinformation and its social impacts requires identifying the news sources spreading false information. While machine learning (ML) techniques have been proposed to address this issue, ML models have failed to provide an efficient implementation scenario that yields useful results. In prior research, the precision of deployment in real traffic deteriorates significantly, experiencing a decrement up to ten times compared to the results derived from benchmark data sets. Our research addresses this gap by proposing a graph-based approach to capture navigational patterns and generate traffic-based features which are used to train a classification model. These navigational and traffic-based features result in classifiers that present outstanding performance when evaluated against real traffic. Moreover, we also propose graph-based filtering techniques to filter out models to be classified by our framework. These filtering techniques increase the signal-to-noise ratio of the models to be classified, greatly reducing false positives and the computational cost of deploying the model. Our proposed framework for the detection of misinformation domains achieves a precision of 0.78 when evaluated in real traffic. This outcome represents an improvement factor of over ten times over those achieved in previous studies