Expertness based Cooperative Q-Learning

BIOROBLS

Classification of Devices and Contact Points of Electronic Channels, Regarding the Behavior of Online Retail Customers in the E-Commerce Environment

Purpose: This research aims to enhance understanding of online retailing through electronic channels (such as mobile devices) and touch points of electronic channels (such as mobile shopping software) from the customer's perspective. Methods: This research is applied for the purpose and descriptive in terms of survey and library data collection methods. The statistical population for this research consists of students at Tehran Azad University. The community consists of more than 4000 people, so according to Morgan's table, the number of samples should be 384. Sampling has been conducted using the available method, and data analysis has been performed using the LSD test. The dependent variable in online shopping is purchase intention, while the independent variables include usefulness, ease of use, pleasure, privacy, and satisfaction. Findings: The research findings indicate that customers currently use eight different devices, including laptops/notebooks, personal computers (PCs), smartphones, tablets, internet-equipped TVs, and in-store kiosks. Additionally, the research findings revealed that purchasing devices can be categorized into four electronic channel categories from the perspective of online retail customers. The floors are categorized as A, B, C, and D, and the coordinates of each are listed in the article. The research findings indicate that both the technological quality and the situational benefitsof the context influence consumers' use of electronic channels. Also, customers engage inonline shopping through various electronic channels (sets of Internet-enabled devices, such as mobile devices) and multi-channel touchpoints (digital shopping formats, such as mobile shopping apps). Conclusion: The results indicate that there is no significant difference between the first cluster (A), which involves shopping using personal computers (PCs), laptops, and netbooks, and the second cluster (B), which involves shopping using smartphones and tablets, in terms of usefulness, ease of use, shopping pleasure, privacy, satisfaction, and purchase intention. There is no significant difference between the first cluster (A) and the third cluster (C), i.e., Internet TV (IE TV), in terms of usefulness, ease of use, shopping pleasure, satisfaction, and purchase intention. There is a significant difference between the first cluster (A) and the fourth cluster (D), specifically in-store kiosks, in terms of usefulness, ease of use, shopping pleasure, privacy, satisfaction, and purchase intention. There is a significant difference between the second cluster (B) and the fourth cluster (D) in terms of usefulness, ease of use, shopping enjoyment, privacy, satisfaction, and purchase intention. Also, there is no significant difference between the third cluster (C) and the fourth cluster (D) in terms of usefulness, ease of use, shopping enjoyment, privacy, satisfaction, and purchase intention

Budgeted Knowledge Transfer for State-wise Heterogeneous RL Agents

In this paper we introduce a budgeted knowledge transfer algorithm for non-homogeneous reinforcement learning agents. Here the source and the target agents are completely identical except in their state representations. The algorithm uses functional space (Q-value space) as the transfer-learning media. In this method, the target agent’s functional points (Q-values) are estimated in an automatically selected lower-dimension subspace in order to accelerate knowledge transfer. The target agent searches that subspace using an exploration policy and selects actions accordingly during the period of its knowledge transfer in order to facilitate gaining an appropriate estimate of its Q-table. We show both analytically and empirically that this method decreases the required learning budget for the target agent

Uncovering hidden network architecture from spiking activities using an exact statistical input-output relation of neurons

神経回路網の構造をつきとめる --神経活動と回路構造をつなぐ新しい地図を作成--. 京都大学プレスリリース. 2023-02-16.Charting a course in the brainy frontier. 京都大学プレスリリース. 2023-02-17.Identifying network architecture from observed neural activities is crucial in neuroscience studies. A key requirement is knowledge of the statistical input-output relation of single neurons in vivo. By utilizing an exact analytical solution of the spike-timing for leaky integrate-and-fire neurons under noisy inputs balanced near the threshold, we construct a framework that links synaptic type, strength, and spiking nonlinearity with the statistics of neuronal population activity. The framework explains structured pairwise and higher-order interactions of neurons receiving common inputs under different architectures. We compared the theoretical predictions with the activity of monkey and mouse V1 neurons and found that excitatory inputs given to pairs explained the observed sparse activity characterized by strong negative triple-wise interactions, thereby ruling out the alternative explanation by shared inhibition. Moreover, we showed that the strong interactions are a signature of excitatory rather than inhibitory inputs whenever the spontaneous rate is low. We present a guide map of neural interactions that help researchers to specify the hidden neuronal motifs underlying observed interactions found in empirical data

Context Transfer in Reinforcement Learning Using Action-Value Functions

This paper discusses the notion of context transfer in reinforcement learning tasks. Context transfer, as defined in this paper, implies knowledge transfer between source and target tasks that share the same environment dynamics and reward function but have different states or action spaces. In other words, the agents learn the same task while using different sensors and actuators. This requires the existence of an underlying common Markov decision process (MDP) to which all the agents’ MDPs can be mapped. This is formulated in terms of the notion of MDP homomorphism. The learning framework is Q-learning. To transfer the knowledge between these tasks, the feature space is used as a translator and is expressed as a partial mapping between the state-action spaces of different tasks. The Q-values learned during the learning process of the source tasks are mapped to the sets of Q-values for the target task. These transferred Q-values are merged together and used to initialize the learning process of the target task. An interval-based approach is used to represent and merge the knowledge of the source tasks. Empirical results show that the transferred initialization can be beneficial to the learning process of the target task