Deep Reinforcement Learning-based Image Captioning with Embedding Reward

Image captioning is a challenging problem owing to the complexity in understanding the image content and diverse ways of describing it in natural language. Recent advances in deep neural networks have substantially improved the performance of this task. Most state-of-the-art approaches follow an encoder-decoder framework, which generates captions using a sequential recurrent prediction model. However, in this paper, we introduce a novel decision-making framework for image captioning. We utilize a "policy network" and a "value network" to collaboratively generate captions. The policy network serves as a local guidance by providing the confidence of predicting the next word according to the current state. Additionally, the value network serves as a global and lookahead guidance by evaluating all possible extensions of the current state. In essence, it adjusts the goal of predicting the correct words towards the goal of generating captions similar to the ground truth captions. We train both networks using an actor-critic reinforcement learning model, with a novel reward defined by visual-semantic embedding. Extensive experiments and analyses on the Microsoft COCO dataset show that the proposed framework outperforms state-of-the-art approaches across different evaluation metrics

Research on a Model of Extracting Persons\u27 Information Based on Statistic Method and Conceptual Knowledge

PACLIC 21 / Seoul National University, Seoul, Korea / November 1-3, 200

Coulomb Screening of 2D Massive Dirac Fermions

A model of 2D massive Dirac fermions, interacting with a instantaneous $1/r$ Coulomb interaction, is presented to mimic the physics of gapped graphene. The static polarization function is calculated explicitly to analyze screening effect at the finite temperature and density. Results are compared with the massless case . We also show that various other works can be reproduced within our model in a straightforward and unified manner

Hippocampus and two way active avoidance conditioning: contrasting effects of cytotoxic lesion and temporary inactivation

Hippocampal lesions tend to facilitate two way active avoidance (2WAA) conditioning, where rats learn to cross to the opposite side of a conditioning chamber to avoid a tone-signaled footshock. This classical finding has been suggested to reflect that hippocampus-dependent place/context memory inhibits 2WAA (a crossing response to the opposite side is inhibited by the memory that this is the place where a shock was received on the previous trial). However, more recent research suggests other aspects of hippocampal function that may support 2WAA learning. More specifically, the ventral hippocampus has been shown to contribute to behavioral responses to aversive stimuli and to positively modulate the meso-accumbens dopamine system, whose activation has been implicated in 2WAA learning. Permanent hippocampal lesions may not reveal these contributions because, following complete and permanent loss of hippocampal output, other brain regions may mediate these processes or because deficits could be masked by lesion-induced extra-hippocampal changes, including an upregulation of accumbal dopamine transmission. Here, we re-examined the hippocampal role in 2WAA learning in Wistar rats, using permanent NMDA-induced neurotoxic lesions and temporary functional inhibition by muscimol or tetrodotoxin (TTX) infusion. Complete hippocampal lesions tended to facilitate 2WAA learning, whereas ventral or dorsal hippocampal lesions had no effect. In contrast, ventral or dorsal hippocampal muscimol or TTX infusions impaired 2WAA learning. Ventral infusions caused an immediate impairment, whereas after dorsal infusions rats showed intact 2WAA learning for 40-50 min, before a marked deficit emerged. These data show that functional inhibition of ventral hippocampus disrupts 2WAA learning, while the delayed impairment following dorsal infusions may reflect the time required for drug diffusion to ventral hippocampus. Overall, using temporary functional inhibition, our study shows that the ventral hippocampus contributes to 2WAA learning. Permanent lesions may not reveal these contributions due to functional compensation and extra-hippocampal lesion effects