RelVAE: Generative Pretraining for few-shot Visual Relationship Detection

Visual relations are complex, multimodal concepts that play an important role in the way humans perceive the world. As a result of their complexity, high-quality, diverse and large scale datasets for visual relations are still absent. In an attempt to overcome this data barrier, we choose to focus on the problem of few-shot Visual Relationship Detection (VRD), a setting that has been so far neglected by the community. In this work we present the first pretraining method for few-shot predicate classification that does not require any annotated relations. We achieve this by introducing a generative model that is able to capture the variation of semantic, visual and spatial information of relations inside a latent space and later exploiting its representations in order to achieve efficient few-shot classification. We construct few-shot training splits and show quantitative experiments on VG200 and VRD datasets where our model outperforms the baselines. Lastly we attempt to interpret the decisions of the model by conducting various qualitative experiments

Multimodal Signal Processing and Learning Aspects of Human-Robot Interaction for an Assistive Bathing Robot

We explore new aspects of assistive living on smart human-robot interaction (HRI) that involve automatic recognition and online validation of speech and gestures in a natural interface, providing social features for HRI. We introduce a whole framework and resources of a real-life scenario for elderly subjects supported by an assistive bathing robot, addressing health and hygiene care issues. We contribute a new dataset and a suite of tools used for data acquisition and a state-of-the-art pipeline for multimodal learning within the framework of the I-Support bathing robot, with emphasis on audio and RGB-D visual streams. We consider privacy issues by evaluating the depth visual stream along with the RGB, using Kinect sensors. The audio-gestural recognition task on this new dataset yields up to 84.5%, while the online validation of the I-Support system on elderly users accomplishes up to 84% when the two modalities are fused together. The results are promising enough to support further research in the area of multimodal recognition for assistive social HRI, considering the difficulties of the specific task. Upon acceptance of the paper part of the data will be publicly available

Composite event recognition for maritime monitoring

Τα συστήματα θαλάσσιας επιτήρησης υποστηρίζουν την ασφαλέστερη ναυτιλία, καθώς επιτρέπουν την ανίχνευση σε πραγματικό χρόνο, επικίνδυνες, ύποπτες και παράνομες δραστηριοτήτες σκαφών. Η πρόθεση αυτής της πτυχιακής είναι η ανάπτυξη μίας αρχιτεκτονικής συστημάτων εστιασμένη στην θαλάσσια επιτήρηση, καθώς και ενός συνόλου “μοτίβων”, ικανά να εφράσουν αποτελεσματικά ναυτιλιακές δραστηριότητες και συμβάντα. Σε αυτή την δουλεία χρησιμοποιούμε ως μήχανη αναγνωρίσης γεγονότων τον Λογισμό Γεγονότων Πραγματικού Χρόνου, μία σύγχρονη υλοποιήση σε γλώσσα Λογικού Προγραμματισμού, του Λογισμού Γεγονότων, καθώς επίσης ένα εργαλείο συμπίεσης τροχιών και ένα εργαλείο ευρέσης χωρικών σχέσεων. Για να βελτιώσουμε περαιτέρω την απόδοση της μηχανής αναγνωρίσης γεγονότων, δημιουργήσαμε ένα γενικό μηχανισμό δυναμικής θεμελίωσης ο οποίος φαίνεται να είναι αποτελεσματικός στα ναυτιλιακά δεδομένα. Επιπλεόν, μέσω της συνεργάσιας μας με τους ειδικούς του δημιουργήσαμε ένα σύνολο από μοτιβά ναυτιλιακής δραστηριότητας, τα οποία και χρησιμοποιούμε στην πειραματική ανάλυση του συστήματος. Για την αξιολόγηση της προτεινόμενης αρχιτεκτονικής εστιάζουμε σε απόδοση και σε ακρίβεια, χρησιμοποιώντας δύο μορφές ροών πραγματικών δεδομένων πλοιών.Maritime monitoring systems support safe shipping as they allow for the real-time detection of dangerous, suspicious and illegal vessel activities. The intent of this thesis was the development of a composite event recognition engine for maritime monitoring and the construction of a set of patterns expressing effectively maritime activities in the Event Calculus. In this work, we use the Run-Time Event Calculus, a modern Prolog implementation of the Event Calculus along with tools allowing the compression of data streams, and the spatio-temporal link discovery. Additionally, to further improve the performance of recognition engine we extended the Run-Time Event Calculus with a dynamic grounding mechanism. Moreover, to increase the accuracy of the proposed system, we have been collaborating with domain experts in order to construct effective patterns of maritime activity. We evaluated our system in terms of predictive accuracy and efficiency using real kinematic vessel data

Handling of Past and Future with Phenesthe+

Writing temporal logic formulae for properties that combine instantaneous events with overlapping temporal phenomena of some duration is difficult in classical temporal logics. To address this issue, in previous work we introduced a new temporal logic with intuitive temporal modalities specifically tailored for the representation of both instantaneous and durative phenomena. We also provided an implementation of a complex event processing system, Phenesthe, based on this logic, that has been applied and tested on a real maritime surveillance scenario. In this work, we extend our temporal logic with two extra modalities to increase its expressive power for handling future formulae. We compare the expressive power of different fragments of our logic with Linear Temporal Logic and dyadic first-order logic. Furthermore, we define correctness criteria for stream processors that use our language. Last but not least, we evaluate empirically the performance of Phenesthe+, our extended implementation, and show that the increased expressive power does not affect efficiency significantly

Making Sense of Heterogeneous Maritime Data

While an abundance of real-time maritime information exists and is readily available to monitoring authorities, there are still many instances in which ships are found to be engaged in dangerous or illegal activities. In order to prevent such activities, authorities employ Vessel Traffic Services systems since they promote safety at sea while also assisting in management of ports. In this paper we report on research done in cooperation with Denbridge Marine Ltd., a global provider of maritime solutions, and present an application integrated in a Vessel Tracking Services system that allows the detection of normal vessel activity as well as dangerous or illegal situations in real-time, using information from the Automatic Identification System, a radar sensor and other information. We use a set of phenomena representing maritime activities of interest in the language of Phenesthe, our Complex Event Processing engine, and detect them on real maritime data streams from the area of Liverpool, United Kingdom. We evaluate our application and show that our system is capable of detecting and visualising maritime activities on the map in real time. Finally, we study and demonstrate the significance of using data from the Automatic Identification System along with radar data for maritime monitoring

Logic Rules Meet Deep Learning: A Novel Approach for Ship Type Classification

The shipping industry is an important component of the global trade and economy, however in order to ensure law compliance and safety it needs to be monitored. In this paper, we present a novel Ship Type classification model that combines vessel transmitted data from the Automatic Identification System, with vessel imagery. The main components of our approach are the Faster R-CNN Deep Neural Network and a Neuro-Fuzzy system with IF-THEN rules. We evaluate our model using real world data and showcase the advantages of this combination while also compare it with other methods. Results show that our model can increase prediction scores by up to 15.4\% when compared with the next best model we considered, while also maintaining a level of explainability as opposed to common black box approaches

Poly(urethane-norbornene) Aerogels via Ring Opening Metathesis Polymerization of Dendritic Urethane-Norbornene Monomers: Structure-Property Relationships as a Function of an Aliphatic Versus an Aromatic Core and the Number of Peripheral Norbornene Moieties

We report the synthesis and characterization of synthetic polymer aerogels based on dendritic-type urethane-norbornene monomers. The core of those monomers is based either on an aromatic/rigid (TIPM/Desmodur RE), or an aliphatic/flexible (Desmodur N3300) triisocyanate. The terminal norbornene groups (three at the tip of each of the three branches) were polymerized via ROMP using the inexpensive 1st generation Grubbs catalyst. The polymerization/gelation conditions were optimized by varying the amount of the catalyst. The resulting wet-gels were dried either from pentane under ambient pressure at 50 oC, or from t-butanol via freeze-drying, or by using supercritical fluid (SCF) CO2. Monomers were characterized with high resolution mass spectrometry (HRMS), 1H- and solid-state 13C-NMR. Aerogels were characterized with ATR-FTIR and solid-state 13C-NMR. The porous network was probed with N2-sorption and SEM. The thermal stability of monomers and aerogels was studied with TGA, which also provides evidence for the number of norbornene groups that reacted via ROMP. At low densities (<0.1 g cm-3) all aerogels were highly porous (porosity > 90%), mostly macroporous materials; aerogels based on the aliphatic/flexible core were fragile, whereas aerogels containing the aromatic/rigid core were plastic, and at even lower densities (0.03 g cm-3) foamy. At higher densities (0.2–0.7 g cm-3) all materials were stiff, strong, and hard. At low monomer concentrations all aerogels consisted of discrete primary particles that formed spherical secondary aggregates. At higher monomer concentrations the structure consisted of fused particles with the size of the previous secondary aggregates, due to the low solubility of the developing polymer, which phase-separated and formed a primary particle network. Same-size fused aggregates were observed for both aliphatic and aromatic triisocyanate-derived aerogels, leading to the conclusion that it is not the aliphatic or aromatic core that determines phase separation, but rather the solubility of the polymeric backbone (polynorbornene) that is in both cases the same. The material properties were compared to those of analogous aerogels bearing only one norbornene moiety at the tip of each branch deriving from the same cores