Reinforcement Learning via AIXI Approximation

This paper introduces a principled approach for the design of a scalable general reinforcement learning agent. This approach is based on a direct approximation of AIXI, a Bayesian optimality notion for general reinforcement learning agents. Previously, it has been unclear whether the theory of AIXI could motivate the design of practical algorithms. We answer this hitherto open question in the affirmative, by providing the first computationally feasible approximation to the AIXI agent. To develop our approximation, we introduce a Monte Carlo Tree Search algorithm along with an agent-specific extension of the Context Tree Weighting algorithm. Empirically, we present a set of encouraging results on a number of stochastic, unknown, and partially observable domains.Comment: 8 LaTeX pages, 1 figur

Probabilities on Sentences in an Expressive Logic

Automated reasoning about uncertain knowledge has many applications. One difficulty when developing such systems is the lack of a completely satisfactory integration of logic and probability. We address this problem directly. Expressive languages like higher-order logic are ideally suited for representing and reasoning about structured knowledge. Uncertain knowledge can be modeled by using graded probabilities rather than binary truth-values. The main technical problem studied in this paper is the following: Given a set of sentences, each having some probability of being true, what probability should be ascribed to other (query) sentences? A natural wish-list, among others, is that the probability distribution (i) is consistent with the knowledge base, (ii) allows for a consistent inference procedure and in particular (iii) reduces to deductive logic in the limit of probabilities being 0 and 1, (iv) allows (Bayesian) inductive reasoning and (v) learning in the limit and in particular (vi) allows confirmation of universally quantified hypotheses/sentences. We translate this wish-list into technical requirements for a prior probability and show that probabilities satisfying all our criteria exist. We also give explicit constructions and several general characterizations of probabilities that satisfy some or all of the criteria and various (counter) examples. We also derive necessary and sufficient conditions for extending beliefs about finitely many sentences to suitable probabilities over all sentences, and in particular least dogmatic or least biased ones. We conclude with a brief outlook on how the developed theory might be used and approximated in autonomous reasoning agents. Our theory is a step towards a globally consistent and empirically satisfactory unification of probability and logic.Comment: 52 LaTeX pages, 64 definiton/theorems/etc, presented at conference Progic 2011 in New Yor

Processing Plant Diseases Using Transformer Model

Agriculture faces challenges in achieving high-yield production while minimizing the use of chemicals. The excessive use of chemicals in agriculture poses many problems. Accurate disease diagnosis is crucial for effective plant disease detection and treatment. Automatic identification of plant diseases using computer vision techniques offers new and efficient approaches compared to traditional methods. Transformers, a type of deep learning model, have shown great promise in computer vision, but as the technology is still new, many vision transformer models struggle to identify diseases by examining the entire leaf. This paper aims to utilize the vision transformer model in analyzing and identifying common diseases that hinder the growth and development of plants through the plant leave images. Besides, it aims to improve the model's stability by focusing more on the entire leaf than individual parts and generalizing better results on leaves not in the image center. Added features such as Shift Patch Tokenization, Locality Self Attention, and Positional Encoding help focus on the whole leaf. The final test accuracy obtained is 89.58%, with relatively slight variances in precision, accuracy, and F1 score across classes, as well as satisfactory model robustness towards changes in leaf orientation and position within the image. The model's effectiveness shows the vision transformer's potential for automated plant disease diagnosis, which can help farmers take timely measures to prevent losses and ensure food security

Context tree switching

This paper describes the Context Tree Switching technique, a modification of Context Tree Weighting for the prediction of binary, stationary, n-Markov sources. By modifying Context Tree Weighting’s recursive weighting scheme, it is possible to mix over a strictly larger class of models without increasing the asymptotic time or space complexity of the original algorithm. We prove that this generalization preserves the desirable theoretical properties of Context Tree Weighting on stationary n-Markov sources, and show empirically that this new technique leads to consistent improvements over Context Tree Weighting as measured on the Calgary Corpus

Reinforcement Learning via AIXI Approximation

This paper introduces a principled approach for the design of a scalable general reinforcement learning agent. This approach is based on a direct approximation of AIXI, a Bayesian optimality notion for general reinforcement learning agents. Previously, it has been unclear whether the theory of AIXI could motivate the design of practical algorithms. We answer this hitherto open question in the affirmative, by providing the first computationally feasible approximation to the AIXI agent. To develop our approximation, we introduce a Monte Carlo Tree Search algorithm along with an agent-specific extension of the Context Tree Weighting algorithm. Empirically, we present a set of encouraging results on a number of stochastic, unknown, and partially observable domains

Treating of Rayon-flocked Fabric by Atmospheric Pressure Plasma

This study investigates hydrophobisation of the surface of rayon-flocked fabric by means of atmospheric pressure plasma (APP) treatment with tetramethylsilane (TMS). Plasma deposition of TMS is regarded as an effective, single-step low pollution method. A detailed study of the process parameters was conducted. A highly hydrophobic surface was successfully fabricated on rayon-flocked fabric and the hydrophobic surface was found to have good stain resistance to coffee and milk tea

Learned Garbage Collection

Several programming languages use garbage collectors (GCs) to automatically manage memory for the programmer. Such collectors must decide when to look for unreachable objects to free, which can have a large performance impact on some applications. In this preliminary work, we propose a design for a learned garbage collector that autonomously learns over time when to perform collections. By using reinforcement learning, our design can incorporate user-defined reward functions, allowing an autonomous garbage collector to learn to optimize the exact metric the user desires (e.g., request latency or queries per second). We conduct an initial experimental study on a prototype, demonstrating that an approach based on tabular Q learning may be promising

A probabilistic chemical programmable computer

The exponential growth of the power of modern digital computers is based upon the miniaturisation of vast nanoscale arrays of electronic switches, but this will be eventually constrained by fabrication limits and power dissipation. Chemical processes have the potential to scale beyond these limits performing computations through chemical reactions, yet the lack of well-defined programmability limits their scalability and performance. We present a hybrid digitally programmable chemical array as a probabilistic computational machine that uses chemical oscillators partitioned in interconnected cells as a computational substrate. This hybrid architecture performs efficient computation by distributing between chemical and digital domains together with error correction. The efficiency is gained by combining digital with probabilistic chemical logic based on nearest neighbour interactions and hysteresis effects. We demonstrated the implementation of one- and two- dimensional Chemical Cellular Automata and solutions to combinatorial optimization problems.Comment: 20 page manuscript, 6 figures, 112 page supplementary volum

CD8α is expressed by human monocytes and enhances FcγR-dependent responses

Abstract Background CD8α enhances the responses of antigen-specific CTL activated through TCR through binding MHC class I, favoring lipid raft partitioning of TCR, and inducing intracellular signaling. CD8α is also found on dendritic cells and rat macrophages, but whether CD8α enhances responses of a partner receptor, like TCR, to activate these cells is not known. TCR and FcR, use analogous or occasionally interchangeable signaling mechanisms suggesting the possibility that CD8α co-activates FcR responses. Interestingly, CD8α+ monocytes are often associated with rat models of disease involving immune-complex deposition and FcR-mediated pathology, such as arthritis, glomerulonephritis, ischaemia, and tumors. While rat macrophages have been shown to express CD8α evidence for CD8α expression by mouse or human monocytes or macrophages was incomplete. Results We detected CD8α, but not CD8β on human monocytes and the monocytic cell line THP-1 by flow cytometry. Reactivity of anti-CD8α mAb with monocytes is at least partly independent of FcR as anti-CD8α mAb detect CD8α by western blot and inhibit binding of MHC class I tetramers. CD8α mRNA is also found in monocytes and THP-1 suggesting CD8α is synthesized by monocytes and not acquired from other CD8α+ cell types. Interestingly, CD8α from monocytes and blood T cells presented distinguishable patterns by 2-D electrophoresis. Anti-CD8α mAb alone did not activate monocyte TNF release. In comparison, TNF release by human monocytes stimulated in a FcR-dependent manner with immune-complexes was enhanced by inclusion of anti-CD8α mAb in immune-complexes. Conclusion Human monocytes express CD8α. Co-engagement of CD8α and FcR enhances monocyte TNF release, suggesting FcR may be a novel partner receptor for CD8α on innate immune cells.</p