STAIR Captions: Constructing a Large-Scale Japanese Image Caption Dataset

In recent years, automatic generation of image descriptions (captions), that is, image captioning, has attracted a great deal of attention. In this paper, we particularly consider generating Japanese captions for images. Since most available caption datasets have been constructed for English language, there are few datasets for Japanese. To tackle this problem, we construct a large-scale Japanese image caption dataset based on images from MS-COCO, which is called STAIR Captions. STAIR Captions consists of 820,310 Japanese captions for 164,062 images. In the experiment, we show that a neural network trained using STAIR Captions can generate more natural and better Japanese captions, compared to those generated using English-Japanese machine translation after generating English captions.Comment: Accepted as ACL2017 short paper. 5 page

Action Class Relation Detection and Classification Across Multiple Video Datasets

The Meta Video Dataset (MetaVD) provides annotated relations between action classes in major datasets for human action recognition in videos. Although these annotated relations enable dataset augmentation, it is only applicable to those covered by MetaVD. For an external dataset to enjoy the same benefit, the relations between its action classes and those in MetaVD need to be determined. To address this issue, we consider two new machine learning tasks: action class relation detection and classification. We propose a unified model to predict relations between action classes, using language and visual information associated with classes. Experimental results show that (i) pre-trained recent neural network models for texts and videos contribute to high predictive performance, (ii) the relation prediction based on action label texts is more accurate than based on videos, and (iii) a blending approach that combines predictions by both modalities can further improve the predictive performance in some cases.Comment: Accepted to Pattern Recognition Letters. 12 pages, 4 figure

Learning Decorrelated Representations Efficiently Using Fast Fourier Transform

Barlow Twins and VICReg are self-supervised representation learning models that use regularizers to decorrelate features. Although these models are as effective as conventional representation learning models, their training can be computationally demanding if the dimension d of the projected embeddings is high. As the regularizers are defined in terms of individual elements of a cross-correlation or covariance matrix, computing the loss for n samples takes O(n d^2) time. In this paper, we propose a relaxed decorrelating regularizer that can be computed in O(n d log d) time by Fast Fourier Transform. We also propose an inexpensive technique to mitigate undesirable local minima that develop with the relaxation. The proposed regularizer exhibits accuracy comparable to that of existing regularizers in downstream tasks, whereas their training requires less memory and is faster for large d. The source code is available.Comment: Accepted for CVPR 202

Important cardiac transcription factor genes are accompanied by bidirectional long non-coding RNAs

BackgroundHeart development is a relatively fragile process in which many transcription factor genes show dose-sensitive characteristics such as haploinsufficiency and lower penetrance. Despite efforts to unravel the genetic mechanism for overcoming the fragility under normal conditions, our understanding still remains in its infancy. Recent studies on the regulatory mechanisms governing gene expression in mammals have revealed that long non-coding RNAs (lncRNAs) are important modulators at the transcriptional and translational levels. Based on the hypothesis that lncRNAs also play important roles in mouse heart development, we attempted to comprehensively identify lncRNAs by comparing the embryonic and adult mouse heart and brain.ResultsWe have identified spliced lncRNAs that are expressed during development and found that lncRNAs that are expressed in the heart but not in the brain are located close to genes that are important for heart development. Furthermore, we found that many important cardiac transcription factor genes are located in close proximity to lncRNAs. Importantly, many of the lncRNAs are divergently transcribed from the promoter of these genes. Since the lncRNA divergently transcribed from Tbx5 is highly evolutionarily conserved, we focused on and analyzed the transcript. We found that this lncRNA exhibits a different expression pattern than that of Tbx5, and knockdown of this lncRNA leads to embryonic lethality.ConclusionThese results suggest that spliced lncRNAs, particularly bidirectional lncRNAs, are essential regulators of mouse heart development, potentially through the regulation of neighboring transcription factor genes

TLR7-dependent and FcγR-independent production of type I interferon in experimental mouse lupus

Increased type I interferon (IFN-I) production and IFN-stimulated gene (ISG) expression are linked to the pathogenesis of systemic lupus erythematosus (SLE). Although the mechanisms responsible for dysregulated IFN-I production in SLE remain unclear, autoantibody-mediated uptake of endogenous nucleic acids is thought to play a role. 2,6,10,14-tetramethylpentadecane (TMPD; also known as pristane) induces a lupus-like disease in mice characterized by immune complex nephritis with autoantibodies to DNA and ribonucleoproteins. We recently reported that TMPD also causes increased ISG expression and that the development of the lupus is completely dependent on IFN-I signaling (Nacionales, D.C., K.M. Kelly-Scumpia, P.Y. Lee, J.S. Weinstein, R. Lyons, E. Sobel, M. Satoh, and W.H. Reeves. 2007. Arthritis Rheum. 56:3770–3783). We show that TMPD elicits IFN-I production, monocyte recruitment, and autoantibody production exclusively through a Toll-like receptor (TLR) 7– and myeloid differentiation factor 88 (MyD88)–dependent pathway. In vitro studies revealed that TMPD augments the effect of TLR7 ligands but does not directly activate TLR7 itself. The effects of TMPD were amplified by the Y-linked autoimmune acceleration cluster, which carries a duplication of the TLR7 gene. In contrast, deficiency of Fcγ receptors (FcγRs) did not affect the production of IFN-I. Collectively, the data demonstrate that TMPD-stimulated IFN-I production requires TLR7/MyD88 signaling and is independent of autoantibody-mediated uptake of ribonucleoproteins by FcγRs