Semantic Validation in Structure from Motion

The Structure from Motion (SfM) challenge in computer vision is the process of recovering the 3D structure of a scene from a series of projective measurements that are calculated from a collection of 2D images, taken from different perspectives. SfM consists of three main steps; feature detection and matching, camera motion estimation, and recovery of 3D structure from estimated intrinsic and extrinsic parameters and features. A problem encountered in SfM is that scenes lacking texture or with repetitive features can cause erroneous feature matching between frames. Semantic segmentation offers a route to validate and correct SfM models by labelling pixels in the input images with the use of a deep convolutional neural network. The semantic and geometric properties associated with classes in the scene can be taken advantage of to apply prior constraints to each class of object. The SfM pipeline COLMAP and semantic segmentation pipeline DeepLab were used. This, along with planar reconstruction of the dense model, were used to determine erroneous points that may be occluded from the calculated camera position, given the semantic label, and thus prior constraint of the reconstructed plane. Herein, semantic segmentation is integrated into SfM to apply priors on the 3D point cloud, given the object detection in the 2D input images. Additionally, the semantic labels of matched keypoints are compared and inconsistent semantically labelled points discarded. Furthermore, semantic labels on input images are used for the removal of objects associated with motion in the output SfM models. The proposed approach is evaluated on a data-set of 1102 images of a repetitive architecture scene. This project offers a novel method for improved validation of 3D SfM models

MOTION-DIRECTION SERIAL VERB CONSTRUCTIONS IN JAVANESE: A LEXICAL-FUNCTIONAL APPROACH

Motion-direction serialization (MDS) is a type of verb serialization that involves motion and directional verbs. This type of verb serialization commonly occurs in serializing languages including Javanese. This paper aims to discuss the characteristics and syntactic structure of MDS in Javanese. The syntactic structure, which comprises constituent and functional structures, is presented by using the theory of lexical -functional grammar (LFG) By adopting the lexical conceptual structure, the writer presents a model of functional structure to explain the syntactic and semantic relation between the motion verb and the directional verb in MDS. The data used in this study were taken from the novel “Suparto Brata’s Omnibus: Kumpulan Roman” written by Suparto Brata (2007) In Addition, this paper also used spoken data from two Javanese native speakers of the Surakarta dialect. The result shows that MDS in Javanese shares the same SUBJ argument, aspect, and negation. This shows that MDS expresses a single event. The LFG analysis of MDS shows that the directional verb has an X-COMP function, which is semantically represented as DIRECTION in the functional structure. The use of lexical conceptual structure in lexical entry and functional structure can clearly show the semantic and syntactic relation of the verbs involved in MDS

From Physical Motion to ‘Come and Go’: A Spoken Corpus Based Analysis of Kata ‘go’-specific Constructions in Korean

I analyze one of the motion verbs in Korean, kata ‘go,’ and its argument structure constructions. The verb shows an extremely high token frequency and its argument structure constructions have been subject to a great degree of variation in terms of its emergent semantics and syntax. However, there have been recurring issues across the previous studies. First, there is the problem of the so-called “written language bias in linguistics” (Linell, 1982), such that most studies on kata have drawn upon mostly invented sentences or written language data. Secondly, previous studies on kata have focused on the verb itself and have made few efforts on examining the construal of kata as it relates to the argument structure constructions in which the verb appears. Considering what has been pointed out so far, on the basis of contemporary Korean spoken data extracted from Sejong Corpus, the current study aims to establish argument structure constructions focusing on the specification of components, i.e. the subject, the oblique phrase containing the suffix, and kata. Argument structure constructions where kata appears and their components are fully specified are called kata-specific constructions. The objective of this study is to outline the alternations of the argument structure constructions in the physical motion domain, and how and to what extent they are inherited by other semantic domains in accordance with semantic extensions. All the semantic domains are argued to be metaphorically or via constructionalization extended from the physical domain. Further, I aim to examine whether the Principle of Maximized Motivation works or not by virtue of two types of cluster analysis. The first one based on binary coding showed that the metaphorical extension and constructionalization starting from the physical motion domain is not limited to the semantic side, but it also influences how and to what extent the allowed argument structure constructions in the physical motion domain are inherited by other semantic domains. This advocates the Principle of Maximized Motivation. However, the second cluster analysis based on relative frequency showed that abstract motion inherits frequency patterns concerning alternations of argument structure constructions from physical motion to the strongest degree, which weakens the principle

Lifting GIS Maps into Strong Geometric Context for Scene Understanding

Contextual information can have a substantial impact on the performance of visual tasks such as semantic segmentation, object detection, and geometric estimation. Data stored in Geographic Information Systems (GIS) offers a rich source of contextual information that has been largely untapped by computer vision. We propose to leverage such information for scene understanding by combining GIS resources with large sets of unorganized photographs using Structure from Motion (SfM) techniques. We present a pipeline to quickly generate strong 3D geometric priors from 2D GIS data using SfM models aligned with minimal user input. Given an image resectioned against this model, we generate robust predictions of depth, surface normals, and semantic labels. We show that the precision of the predicted geometry is substantially more accurate other single-image depth estimation methods. We then demonstrate the utility of these contextual constraints for re-scoring pedestrian detections, and use these GIS contextual features alongside object detection score maps to improve a CRF-based semantic segmentation framework, boosting accuracy over baseline models

Semantic Categories in the Domain of Motion Verbs by Adult Speakers of Danish, German, and Turkish

Languages differ in the ways they divide the world. This study applies cluster analysis to understand how and why languages differ in the way they express motion events. It further lays out what the parameters of the structure of the semantic space of motion are, based on data collected from participants who were adult speakers of Danish, German, and Turkish. The participants described 37 video clips depicting a large variety of motion events. The results of the study show that the segmentation of the semantic space displays a great deal of variation across all three groups. Turkish differs from German and Danish with respect to the features used to segment the semantic space – namely by using vector orientation. German and Danish differ greatly with respect to (a) how fine-grained the distinctions made are, and (b) how motion verbs with a common Germanic root are distributed across the semantic space. Consequently, this study illustrates that the parameters applied for categorization by speakers are, to some degree, related to typological membership, in relation to Talmy's typological framework for the expression of motion events. Finally, the study shows that the features applied for categorization differ across languages and that typological membership is not necessarily a predictor of elaboration of the motion verb lexicon

Geometry meets semantics for semi-supervised monocular depth estimation

Depth estimation from a single image represents a very exciting challenge in computer vision. While other image-based depth sensing techniques leverage on the geometry between different viewpoints (e.g., stereo or structure from motion), the lack of these cues within a single image renders ill-posed the monocular depth estimation task. For inference, state-of-the-art encoder-decoder architectures for monocular depth estimation rely on effective feature representations learned at training time. For unsupervised training of these models, geometry has been effectively exploited by suitable images warping losses computed from views acquired by a stereo rig or a moving camera. In this paper, we make a further step forward showing that learning semantic information from images enables to improve effectively monocular depth estimation as well. In particular, by leveraging on semantically labeled images together with unsupervised signals gained by geometry through an image warping loss, we propose a deep learning approach aimed at joint semantic segmentation and depth estimation. Our overall learning framework is semi-supervised, as we deploy groundtruth data only in the semantic domain. At training time, our network learns a common feature representation for both tasks and a novel cross-task loss function is proposed. The experimental findings show how, jointly tackling depth prediction and semantic segmentation, allows to improve depth estimation accuracy. In particular, on the KITTI dataset our network outperforms state-of-the-art methods for monocular depth estimation.Comment: 16 pages, Accepted to ACCV 201

Small clause results revisited

The main purpose of this paper is to show that argument structure constructions like complex telic path of motion constructions (John walked to the store) or complex resultative constructions (The dog barked the chickens awake) are not to be regarded as "theoretical entities" (Jackendoff (1997b); Goldberg (1995)). As an alternative to these semanticocentric accounts, I argue that their epiphenomenal status can be shown iff we take into account some important insights from three syntactically-oriented works: (i) Hoekstra's (1988, 1992) analysis of SC R, (ii) Hale & Keyser's (1993f.) configurational theory of argument structure, and (iii) Mateu & Rigau’s (1999; i.p.) syntactic account of Talmy's (1991) typological distinction between 'satellite framed languages' (e.g., English, German, Dutch, etc.) and 'verb-framed languages' (e.g., Catalan, Spanish, French, etc.). In particular, it is argued that the formation of the abovementioned constructions involves a conflation process of two different syntactic argument structures, this process being carried out via a 'generalized transformation'. Accordingly, the so-called 'lexical subordination process' (Levin & Rapoport (1988)) is argued to involve a syntactic operation, rather than a semantic one. Due to our assuming that the parametric variation involved in the constructions under study cannot be explained in purely semantic terms (Mateu & Rigau (1999)), Talmy's (1991) typological distinction is argued to be better stated in lexical syntactic terms

Argument structure, conceptual metaphor and semantic change : how to succeed in Indo-European without really trying

In contrast to grammaticalization studies of lexical verbs changing into auxiliaries, the realm of semantic changes associated with lexical verbs is an understudied area of historical semantics. We concentrate on the emergence of verbs of success from more semantically concrete verbs, uncovering six conceptual metaphors which all co-occur with non-canonical encoding of subjects in Indo-European. Careful scrutiny of the relevant data reveals a semantic development most certainly inherited from Indo-European; hence, we reconstruct a DAT-‘succeeds’ construction at different levels of schematicity for Proto-Indo-European, including a novel reconstruction of a conceptual metaphor, success is motion forward, and the mapping between this metaphor and the verb-class-specific argument structure construction. Hence, this article offers a systematic analysis of regularity in semantic change, highlighting the importance of predicate and argument structure for lexical semantic developments