Cooperative Training of Deep Aggregation Networks for RGB-D Action Recognition

A novel deep neural network training paradigm that exploits the conjoint information in multiple heterogeneous sources is proposed. Specifically, in a RGB-D based action recognition task, it cooperatively trains a single convolutional neural network (named c-ConvNet) on both RGB visual features and depth features, and deeply aggregates the two kinds of features for action recognition. Differently from the conventional ConvNet that learns the deep separable features for homogeneous modality-based classification with only one softmax loss function, the c-ConvNet enhances the discriminative power of the deeply learned features and weakens the undesired modality discrepancy by jointly optimizing a ranking loss and a softmax loss for both homogeneous and heterogeneous modalities. The ranking loss consists of intra-modality and cross-modality triplet losses, and it reduces both the intra-modality and cross-modality feature variations. Furthermore, the correlations between RGB and depth data are embedded in the c-ConvNet, and can be retrieved by either of the modalities and contribute to the recognition in the case even only one of the modalities is available. The proposed method was extensively evaluated on two large RGB-D action recognition datasets, ChaLearn LAP IsoGD and NTU RGB+D datasets, and one small dataset, SYSU 3D HOI, and achieved state-of-the-art results

JAVANESE LANGUAGE MODALITY IN BLENCONG ARTICLES OF SUARA MERDEKA NEWSPAPER

Many languages have modalities which are usually expressed through certain modal verbs. By analyzing the modal verbs, the varying degrees of commitment to or belief in a proposition can be explained. Javanese language also has such modal system. This paper examines the modality in Javanese language used in Blencong articles of Suara Merdeka newspaper. The articles contain opinions towards many issues from the view of Javanese people and Javanese wisdom. This paper is aimed to find out the modalities realized in modal verbs in Blencong articles and to describe the functions of such modal verbs. The data were collected from Blencong articles downloaded from the online version of Suara Merdeka newspaper. They were then analyzed by using translational (identity) method. The result shows that the writers of the articles use epistemic modality to show their certainty and uncertainty about any issue, deontic modality to show their attitude towards obligation, dynamic modality to express one‘s ability, and intentional modality to express the writers‘ wish, willingness, and hope. The modal verbs used are mesthi, pancen, mesthine, bakal, arep, kudu, kudune, dititahke, bisa, bisaa, mbokmenawa, and muga-muga

Which senses dominate at different stages of product experience?

In the area of product design, sensory dominance can be defined as the relative importance of different sensory modalities for product experience. Since product experience is multisensory, it is interesting to know which sensory modality plays a leading role in a particular experience, so that designers could concentrate on the creation of the most relevant product properties. It is often assumed that vision dominates other senses. In the present study, we investigated the importance of different sensory modalities during various episodes of product usage. We asked 120 respondents to describe their experiences with consumer products in the following situations: while buying a product, after the first week, the first month, and the first year of usage. The data suggest that the dominant modality depends on the period of product usage. At the moment of buying, vision is the most important modality, but at later stages other modalities become more important. The dominance of a particular modality may depend on its appropriateness for the particular task. During long-term usage, modality importance depends on product functions and the characteristics of the user-product interaction. We conclude that to create a long-lasting positive product experience, designers need to consider the user-product interaction at different stages of product usage and to determine which sensory modality dominates product experience at each stage. Keywords: Sensory Dominance; User-Product Interaction; Product Design</p