3D Non-Rigid Reconstruction with Prior Shape Constraints

3D non-rigid shape recovery from a single uncalibrated camera is a challenging, under-constrained problem in computer vision. Although tremendous progress has been achieved towards solving the problem, two main limitations still exist in most previous solutions. First, current methods focus on non-incremental solutions, that is, the algorithms require collection of all the measurement data before the reconstruction takes place. This methodology is inherently unsuitable for applications requiring real-time solutions. At the same time, most of the existing approaches assume that 3D shapes can be accurately modelled in a linear subspace. These methods are simple and have been proven effective for reconstructions of objects with relatively small deformations, but have considerable limitations when the deformations are large or complex. The non-linear deformations are often observed in highly flexible objects for which the use of the linear model is impractical. Note that specific types of shape variation might be governed by only a small number of parameters and therefore can be well-represented in a low dimensional manifold. The methods proposed in this thesis aim to estimate the non-rigid shapes and the corresponding camera trajectories, based on both the observations and the prior learned manifold. Firstly, an incremental approach is proposed for estimating the deformable objects. An important advantage of this method is the ability to reconstruct the 3D shape from a newly observed image and update the parameters in 3D shape space. However, this recursive method assumes the deformable shapes only have small variations from a mean shape, thus is still not feasible for objects subject to large scale deformations. To address this problem, a series of approaches are proposed, all based on non-linear manifold learning techniques. Such manifold is used as a shape prior, with the reconstructed shapes constrained to lie within the manifold. Those non-linear manifold based approaches significantly improve the quality of reconstructed results and are well-adapted to different types of shapes undergoing significant and complex deformations. Throughout the thesis, methods are validated quantitatively on 2D points sequences projected from the 3D motion capture data for a ground truth comparison, and are qualitatively demonstrated on real example of 2D video sequences. Comparisons are made for the proposed methods against several state-of-the-art techniques, with results shown for a variety of challenging deformable objects. Extensive experiments also demonstrate the robustness of the proposed algorithms with respect to measurement noise and missing data

Is 2D Unlabeled Data Adequate for Recognizing Facial Expressions?

Automatic facial expression recognition is one of the important challenges for computer vision and machine learning. Despite the fact that many successes have been achieved in the recent years, several important but unresolved problems still remain. This paper describes a facial expression recognition system based on the random forest technique. Contrary to the many previous methods, the proposed system uses only very simple landmark features, with the view of a possible real-time implementation on low-cost portable devices. Both supervised and unsupervised variants of the method are presented. However, the main objective of the paper is to provide some quantitative experimental evidence behind more fundamental questions in facial articulation analysis, namely the relative significance of 3D information as oppose to 2D data only and importance of the labelled training data in the supervised learning as opposed to the unsupervised learning. The comprehensive experiments are performed on the BU-3DFE facial expression database. These experiments not only show the effectiveness of the described methods but also demonstrate that the common assumptions about facial expression recognition are debatable

Is the 2D unlabelled data adequate for facial expressionrecognition?

Automatic facial expression recognition is one of the important challenges for computer vision and machine learning. Despite the fact that many successes have been achieved in the recent years, several important but unresolved problems still remain. This paper describes a facial expression recognition system based on the random forest technique. Contrary to the many previous methods, the proposed system uses only very simple landmark features, with the view of a possible real-time implementation on low-cost portable devices. Both supervised and unsupervised variants of the method are presented. However, the main objective of the paper is toprovide some quantitative experimental evidence behind more fundamental questions in facial articulation analysis, namely the relative significance of 3D information as oppose to 2D data only and importance of the labelled training data in the supervised learning as opposed to the unsupervised learning. The comprehensive experiments are performed on the BU-3DFE facial expression database. These experiments not only show theeffectiveness of the described methods but also demonstrate that the common assumptions about facial expression recognition are debatable

Convolutional Neural Networks over Tree Structures for Programming Language Processing

Programming language processing (similar to natural language processing) is a hot research topic in the field of software engineering; it has also aroused growing interest in the artificial intelligence community. However, different from a natural language sentence, a program contains rich, explicit, and complicated structural information. Hence, traditional NLP models may be inappropriate for programs. In this paper, we propose a novel tree-based convolutional neural network (TBCNN) for programming language processing, in which a convolution kernel is designed over programs' abstract syntax trees to capture structural information. TBCNN is a generic architecture for programming language processing; our experiments show its effectiveness in two different program analysis tasks: classifying programs according to functionality, and detecting code snippets of certain patterns. TBCNN outperforms baseline methods, including several neural models for NLP.Comment: Accepted at AAAI-1

Rationalizing Text Matching: Learning Sparse Alignments via Optimal Transport

Selecting input features of top relevance has become a popular method for building self-explaining models. In this work, we extend this selective rationalization approach to text matching, where the goal is to jointly select and align text pieces, such as tokens or sentences, as a justification for the downstream prediction. Our approach employs optimal transport (OT) to find a minimal cost alignment between the inputs. However, directly applying OT often produces dense and therefore uninterpretable alignments. To overcome this limitation, we introduce novel constrained variants of the OT problem that result in highly sparse alignments with controllable sparsity. Our model is end-to-end differentiable using the Sinkhorn algorithm for OT and can be trained without any alignment annotations. We evaluate our model on the StackExchange, MultiNews, e-SNLI, and MultiRC datasets. Our model achieves very sparse rationale selections with high fidelity while preserving prediction accuracy compared to strong attention baseline models.Comment: To appear at ACL 202

PERBANDINGAN METAFORA WARNA DALAM BAHASA INDONESIA DAN MANDARIN SERTA PEMANFAATANNYA SEBAGAI BUKU PENGAYAAN BIPA BERAKSES DIGITAL

Abstrak Jika orang Indonesia dan orang Tiongkok tidak dapat saling memahami makna metafora warna dalam bahasanya, kesalahpahaman akan mudah muncul dalam komunikasi lintas budaya kedua pihak ini. Misalnya, buku kuning bermakna buku berisikan ajaran Islam dalam bahasa Indonesia, sedangkan bermakna buku porno dalam bahasa Mandarin. Penelitian ini bertujuan untuk membandingkan metafora warna dasar dalam kedua bahasa ini. Metode yang digunakan adalah kualitatif dengan teknik analisis komparatif. Hasil penelitian ini menunjukkan bahwa dalam kedua bahasa ini masing-masing terdapat 6 warna dasar, antara lain hitam/, putih/, merah/, hijau/, kuning/, dan biru/ . Dalam kedua bahasa ini digunakan berbagai bentuk lingual metafora warna, yaitu (1) kata, (2) frase, (3) kata majemuk, (4) klausa, dan (5) kalimat. Kedua bahasa ini cenderung menggunakan kata majemuk untuk mengungkapkan metafora warna. Metafora warna dasar dalam kedua bahasa ini kebanyakan diberikan makna negatif. Warna putih kebanyakan diberikan makna positif dalam bahasa Indonesia, sebaliknya kebanyakan diberikan makna negatif dalam bahasa Mandarin. Warna merah kebanyakan diberikan makna negatif dalam bahasa Indonesia, sebaliknya kebanyakan diberikan makna positif dalam bahasa Mandarin. Dalam bahasa Indonesia warna hijau lebih banyak diberikan makna netral. Dalam bahasa Mandarin warna (kuning) kebanyakan diberikan makna negatif . Makna yang diberikan pada metafora warna biru dalam bahasa Indonesia relatif stabil, sedangkan dalam bahasa Mandarin metafora warna (biru) hanya memiliki makna netral. Enam warna dasar dalam kedua bahasa ini sama-sama paling banyak digunakan untuk memetaforakan benda dan bagian tubuh. Buku pengayaan BIPA yang dirancang mencakup makna metafora 6 kata warna dasar dalam kedua bahasa ini serta contoh kalimatnya. Kata kunci: Metafora, Warna, Indonesia, Tiongkok Abstract If Indonesians and Chinese people cannot understand each other's meaning of color metaphors in their language, misunderstandings will easily appear in cross -cultural communication between these two countries. For example, a yellow book means a book containing Islamic teachings in Indonesian, while pornbooks in Mandarin. This study aims to compare basic color metaphors in Indonesian and Mandarin. The method used is qualitative with comparative analysis techniques. The results of this study indicate that in these two languages there are 6 basic colors, including black/, white/, red/, green/ , yellow/, dan blue/. In these two languages, various forms of lingual metaphors are used, namely (1) words, (2) phrases, (3) compound words, (4) clauses, and (5) sentences. Both of these languages tend to use compound words to express color metaphors. Basic color metaphors in these two languages are mostly given negative meanings. The white color is mostly given a positive meaning in Indonesian, on the contrary mostly given the negative meanings in Mandarin. The red color is mostly given a negative meaning in Indonesian, on the contrary mostly given the positive meanings in Mandarin. In Indonesian the green color is given more neutral meanings. In Mandarin color (yellow) is mostly given the negative meanings. The meanings given to the blue metaphor in Indonesian are relatively stable, whereas in Mandarin the color (blue) only has neutral meanings. Six basic colors in these two languages are both most widely used to metaphor objects and body parts. The BIPA enrichment book includes the meaning of metaphor 6 basic colors in these two languages and examples of sentences. Keywords: Metaphor, Colors, Indonesia, Chin