Generative Intrinsic Optimization: Intrinsic Control with Model Learning

Future sequence represents the outcome after executing the action into the environment (i.e. the trajectory onwards). When driven by the information-theoretic concept of mutual information, it seeks maximally informative consequences. Explicit outcomes may vary across state, return, or trajectory serving different purposes such as credit assignment or imitation learning. However, the inherent nature of incorporating intrinsic motivation with reward maximization is often neglected. In this work, we propose a policy iteration scheme that seamlessly incorporates the mutual information, ensuring convergence to the optimal policy. Concurrently, a variational approach is introduced, which jointly learns the necessary quantity for estimating the mutual information and the dynamics model, providing a general framework for incorporating different forms of outcomes of interest. While we mainly focus on theoretical analysis, our approach opens the possibilities of leveraging intrinsic control with model learning to enhance sample efficiency and incorporate uncertainty of the environment into decision-making

The Point to Which Soft Actor-Critic Converges

Soft actor-critic is a successful successor over soft Q-learning. While lived under maximum entropy framework, their relationship is still unclear. In this paper, we prove that in the limit they converge to the same solution. This is appealing since it translates the optimization from an arduous to an easier way. The same justification can also be applied to other regularizers such as KL divergence

Facial Action Unit Detection Using Attention and Relation Learning

Attention mechanism has recently attracted increasing attentions in the field of facial action unit (AU) detection. By finding the region of interest of each AU with the attention mechanism, AU-related local features can be captured. Most of the existing attention based AU detection works use prior knowledge to predefine fixed attentions or refine the predefined attentions within a small range, which limits their capacity to model various AUs. In this paper, we propose an end-to-end deep learning based attention and relation learning framework for AU detection with only AU labels, which has not been explored before. In particular, multi-scale features shared by each AU are learned firstly, and then both channel-wise and spatial attentions are adaptively learned to select and extract AU-related local features. Moreover, pixel-level relations for AUs are further captured to refine spatial attentions so as to extract more relevant local features. Without changing the network architecture, our framework can be easily extended for AU intensity estimation. Extensive experiments show that our framework (i) soundly outperforms the state-of-the-art methods for both AU detection and AU intensity estimation on the challenging BP4D, DISFA, FERA 2015 and BP4D+ benchmarks, (ii) can adaptively capture the correlated regions of each AU, and (iii) also works well under severe occlusions and large poses.Comment: This paper is accepted by IEEE Transactions on Affective Computin

Spatio-Temporal Relation and Attention Learning for Facial Action Unit Detection

Spatio-temporal relations among facial action units (AUs) convey significant information for AU detection yet have not been thoroughly exploited. The main reasons are the limited capability of current AU detection works in simultaneously learning spatial and temporal relations, and the lack of precise localization information for AU feature learning. To tackle these limitations, we propose a novel spatio-temporal relation and attention learning framework for AU detection. Specifically, we introduce a spatio-temporal graph convolutional network to capture both spatial and temporal relations from dynamic AUs, in which the AU relations are formulated as a spatio-temporal graph with adaptively learned instead of predefined edge weights. Moreover, the learning of spatio-temporal relations among AUs requires individual AU features. Considering the dynamism and shape irregularity of AUs, we propose an attention regularization method to adaptively learn regional attentions that capture highly relevant regions and suppress irrelevant regions so as to extract a complete feature for each AU. Extensive experiments show that our approach achieves substantial improvements over the state-of-the-art AU detection methods on BP4D and especially DISFA benchmarks

Thrombin is a novel regulator of hexokinase activity in mesangial cells

Thrombin is a novel regulator of hexokinase activity in mesangial cells.BackgroundHexokinase (HK) activity is fundamentally important to cellular glucose uptake and metabolism. Phorbol esters increase both HK activity and glucose utilization in cultured mesangial cells via a protein kinase C (PKC)- and extracellular signal-regulated kinases 1 and 2 (ERK1/2)-dependent mechanism. In adult kidneys, increased HK activity has been reported in both glomerular injury and in diabetes, but the mechanisms responsible for these changes are unknown. Thrombin, a known activator of both PKC and ERK1/2, is increased in the settings of renal injury and diabetes. Thus, thrombin may contribute to the observed changes in HK activity in vivo.MethodsThrombin and thrombin receptor agonists were tested for the ability to increase HK activity and glucose metabolism in murine mesangial (SV40 MES 13) cells. ERK1/2 activation was also evaluated in parallel. Thrombin inhibition (hirudins), PKC depletion, Ser-Thr kinase inhibition (H-7), MEK1/2 inhibition (PD98059), pertussis toxin (PTX), and general inhibitors of transcription or translation were then tested for the ability to attenuate these effects.ResultsThrombin (≥0.01 U/mL) mimicked the effect of phorbol esters, increasing HK activity 50% within 12 to 24 hours (P < 0.05). This effect was inhibited by hirudins, mimicked by thrombin receptor agonists, and accompanied by increased Glc utilization. H-7, PD98059, and general inhibitors of transcription or translation—but not PTX—prevented thrombin-induced HK activity at 24 hours. PKC depletion and PD98059 also blocked the associated phosphorylation and activation of ERK1/2.ConclusionsThrombin increases mesangial cell HK activity via a PTX-insensitive mechanism involving thrombin receptor activation, PKC-dependent activation of ERK1/2, and both ongoing gene transcription and de novo protein synthesis. As such, thrombin is a novel regulator of HK activity in mesangial cells and may play a role in coupling renal injury to metabolism