Learning semantic sentence representations from visually grounded language without lexical knowledge

Current approaches to learning semantic representations of sentences often use prior word-level knowledge. The current study aims to leverage visual information in order to capture sentence level semantics without the need for word embeddings. We use a multimodal sentence encoder trained on a corpus of images with matching text captions to produce visually grounded sentence embeddings. Deep Neural Networks are trained to map the two modalities to a common embedding space such that for an image the corresponding caption can be retrieved and vice versa. We show that our model achieves results comparable to the current state-of-the-art on two popular image-caption retrieval benchmark data sets: MSCOCO and Flickr8k. We evaluate the semantic content of the resulting sentence embeddings using the data from the Semantic Textual Similarity benchmark task and show that the multimodal embeddings correlate well with human semantic similarity judgements. The system achieves state-of-the-art results on several of these benchmarks, which shows that a system trained solely on multimodal data, without assuming any word representations, is able to capture sentence level semantics. Importantly, this result shows that we do not need prior knowledge of lexical level semantics in order to model sentence level semantics. These findings demonstrate the importance of visual information in semantics

Principles in Patterns (PiP) : Project Evaluation Synthesis

Evaluation activity found the technology-supported approach to curriculum design and approval developed by PiP to demonstrate high levels of user acceptance, promote improvements to the quality of curriculum designs, render more transparent and efficient aspects of the curriculum approval and quality monitoring process, demonstrate process efficacy and resolve a number of chronic information management difficulties which pervaded the previous state. The creation of a central repository of curriculum designs as the basis for their management as "knowledge assets", thus facilitating re-use and sharing of designs and exposure of tacit curriculum design practice, was also found to be highly advantageous. However, further process improvements remain possible and evidence of system resistance was found in some stakeholder groups. Recommendations arising from the findings and conclusions include the need to improve data collection surrounding the curriculum approval process so that the process and human impact of C-CAP can be monitored and observed. Strategies for improving C-CAP acceptance among the "late majority", the need for C-CAP best practice guidance, and suggested protocols on the knowledge management of curriculum designs are proposed. Opportunities for further process improvements in institutional curriculum approval, including a re-engineering of post-faculty approval processes, are also recommended

MAG: A Multilingual, Knowledge-base Agnostic and Deterministic Entity Linking Approach

Entity linking has recently been the subject of a significant body of research. Currently, the best performing approaches rely on trained mono-lingual models. Porting these approaches to other languages is consequently a difficult endeavor as it requires corresponding training data and retraining of the models. We address this drawback by presenting a novel multilingual, knowledge-based agnostic and deterministic approach to entity linking, dubbed MAG. MAG is based on a combination of context-based retrieval on structured knowledge bases and graph algorithms. We evaluate MAG on 23 data sets and in 7 languages. Our results show that the best approach trained on English datasets (PBOH) achieves a micro F-measure that is up to 4 times worse on datasets in other languages. MAG, on the other hand, achieves state-of-the-art performance on English datasets and reaches a micro F-measure that is up to 0.6 higher than that of PBOH on non-English languages.Comment: Accepted in K-CAP 2017: Knowledge Capture Conferenc

Principles in Patterns (PiP) : User Acceptance Testing of Course and Class Approval Online Pilot (C-CAP)

The PiP Evaluation Plan documents four distinct evaluative strands, the first of which entails an evaluation of the PiP system pilot (WP7:37 – Systems & tool evaluation). Phase 1 of this evaluative strand focused on the heuristic evaluation of the PiP Course and Class Approval Online Pilot system (C-CAP) and was completed in December 2011. Phase 2 of the evaluation is broadly concerned with "user acceptance testing". This entails exploring the extent to which C-CAP functionality meets users' expectations within specific curriculum design tasks, as well as eliciting data on C-CAP's overall usability and its ability to support academics in improving the quality of curricula. The general evaluative approach adopted therefore employs a combination of standard Human-Computer Interaction (HCI) approaches and specially designed data collection instruments, including protocol analysis, stimulated recall and pre- and post-test questionnaire instruments. This brief report summarises the methodology deployed, presents the results of the evaluation and discusses their implications for the further development of C-CAP

Relating satellite imagery with grain protein content

Satellite images, captured during the growing seasons of barley, sorghum and wheat were analysed to establish a relationship between the spectral response and the harvested grain protein content. This study was conducted near Jimbour (approx. 151°10’E and 27°05’S) in southern Queensland. Grain protein contents of the geo-referenced samples, collected manually during the harvest, were determined using a laboratory-based near-infrared spectrophotometer. Grain protein contents in grain varied between 7.4–15.2% in barley, 6.2– 10.6% in sorghum and 13.1–15.6% in wheat. The Landsat images of 18 September 1999 (a week after barley flowering), 5 March 2000 (three weeks before sorghum harvest), and 15 August 2001 (two weeks before wheat flowering) were analysed. Additionally, an ASTER image of 24 September 2001 (three weeks after wheat flowering) was also examined. Digital numbers, extracted from raw image bands and derived indices, were correlated with grain protein contents. The grain protein content in barley was correlated strongly (r>0.80) with bands 2, 4 and 5 of the Landsat scene, first principal component, and the tasselled cap brightness and greenness indices. Similarly, wheat protein content was well correlated (r>0.75) with the near infrared band (band 4) of the Landsat scene, first principal component, and the tasselled cap brightness, greenness and wetness indices. The band 3 (near infrared band) of the ASTER image, captured well after flowering, was moderately correlated (r<0.5) with the protein content of the wheat. However, the grain protein content in sorghum was found poorly correlated (r<0.20) with Landsat image bands and indices. Results indicate that it may be possible to use certain bands and indices of the satellite images, captured around the time of flowering, to predict grain protein content of barley and wheat crops

Knowledge society arguments revisited in the semantic technologies era

In the light of high profile governmental and international efforts to realise the knowledge society, I review the arguments made for and against it from a technology standpoint. I focus on advanced knowledge technologies with applications on a large scale and in open- ended environments like the World Wide Web and its ambitious extension, the Semantic Web. I argue for a greater role of social networks in a knowledge society and I explore the recent developments in mechanised trust, knowledge certification, and speculate on their blending with traditional societal institutions. These form the basis of a sketched roadmap for enabling technologies for a knowledge society