Interactive Search and Exploration in Online Discussion Forums Using Multimodal Embeddings

In this paper we present a novel interactive multimodal learning system, which facilitates search and exploration in large networks of social multimedia users. It allows the analyst to identify and select users of interest, and to find similar users in an interactive learning setting. Our approach is based on novel multimodal representations of users, words and concepts, which we simultaneously learn by deploying a general-purpose neural embedding model. We show these representations to be useful not only for categorizing users, but also for automatically generating user and community profiles. Inspired by traditional summarization approaches, we create the profiles by selecting diverse and representative content from all available modalities, i.e. the text, image and user modality. The usefulness of the approach is evaluated using artificial actors, which simulate user behavior in a relevance feedback scenario. Multiple experiments were conducted in order to evaluate the quality of our multimodal representations, to compare different embedding strategies, and to determine the importance of different modalities. We demonstrate the capabilities of the proposed approach on two different multimedia collections originating from the violent online extremism forum Stormfront and the microblogging platform Twitter, which are particularly interesting due to the high semantic level of the discussions they feature

A personalized and context-aware news offer for mobile devices

For classical domains, such as movies, recommender systems have proven their usefulness. But recommending news is more challenging due to the short life span of news content and the demand for up-to-date recommendations. This paper presents a news recommendation service with a content-based algorithm that uses features of a search engine for content processing and indexing, and a collaborative filtering algorithm for serendipity. The extension towards a context-aware algorithm is made to assess the information value of context in a mobile environment through a user study. Analyzing interaction behavior and feedback of users on three recommendation approaches shows that interaction with the content is crucial input for user modeling. Context-aware recommendations using time and device type as context data outperform traditional recommendations with an accuracy gain dependent on the contextual situation. These findings demonstrate that the user experience of news services can be improved by a personalized context-aware news offer

Interactive searching and browsing of video archives: using text and using image matching

Over the last number of decades much research work has been done in the general area of video and audio analysis. Initially the applications driving this included capturing video in digital form and then being able to store, transmit and render it, which involved a large effort to develop compression and encoding standards. The technology needed to do all this is now easily available and cheap, with applications of digital video processing now commonplace, ranging from CCTV (Closed Circuit TV) for security, to home capture of broadcast TV on home DVRs for personal viewing. One consequence of the development in technology for creating, storing and distributing digital video is that there has been a huge increase in the volume of digital video, and this in turn has created a need for techniques to allow effective management of this video, and by that we mean content management. In the BBC, for example, the archives department receives approximately 500,000 queries per year and has over 350,000 hours of content in its library. Having huge archives of video information is hardly any benefit if we have no effective means of being able to locate video clips which are of relevance to whatever our information needs may be. In this chapter we report our work on developing two specific retrieval and browsing tools for digital video information. Both of these are based on an analysis of the captured video for the purpose of automatically structuring into shots or higher level semantic units like TV news stories. Some also include analysis of the video for the automatic detection of features such as the presence or absence of faces. Both include some elements of searching, where a user specifies a query or information need, and browsing, where a user is allowed to browse through sets of retrieved video shots. We support the presentation of these tools with illustrations of actual video retrieval systems developed and working on hundreds of hours of video content

Named Data Networking in Vehicular Ad hoc Networks: State-of-the-Art and Challenges

International audienceInformation-Centric Networking (ICN) has been proposed as one of the future Internet architectures. It is poised to address the challenges faced by today's Internet that include, but not limited to, scalability, addressing, security, and privacy. Furthermore, it also aims at meeting the requirements for new emerging Internet applications. To realize ICN, Named Data Networking (NDN) is one of the recent implementations of ICN that provides a suitable communication approach due to its clean slate design and simple communication model. There are a plethora of applications realized through ICN in different domains where data is the focal point of communication. One such domain is Intelligent Transportation System (ITS) realized through Vehicular Ad hoc NETwork (VANET) where vehicles exchange information and content with each other and with the infrastructure. To date, excellent research results have been yielded in the VANET domain aiming at safe, reliable, and infotainment-rich driving experience. However, due to the dynamic topologies, host-centric model, and ephemeral nature of vehicular communication, various challenges are faced by VANET that hinder the realization of successful vehicular networks and adversely affect the data dissemination, content delivery, and user experiences. To fill these gaps, NDN has been extensively used as underlying communication paradigm for VANET. Inspired by the extensive research results in NDN-based VANET, in this paper, we provide a detailed and systematic review of NDN-driven VANET. More precisely, we investigate the role of NDN in VANET and discuss the feasibility of NDN architecture in VANET environment. Subsequently, we cover in detail, NDN-based naming, routing and forwarding, caching, mobility, and security mechanism for VANET. Furthermore, we discuss the existing standards, solutions, and simulation tools used in NDN-based VANET. Finally, we also identify open challenges and issues faced by NDN-driven VANET and highlight future research directions that should be addressed by the research community

Calibration of the UMass Advanced Multi-Frequency Radar

The Advanced Multi-Frequency Radar is a three-frequency system designed and built by the University of Massachusetts Microwave Remote Sensing Lab (MIRSL). The radar has three frequencies, Ku-band (13.4 GHz), Ka-band (35.6 GHz), and W-band (94.92GHz). The additional information gained from additional frequencies allows the system to be sensitive to a wide range of atmospheric and precipitation particle sizes, while increasing the ability to derive particle microphysics from radar retrievals. This thesis details the calibration of data from the Canadian CloudSat/CALIPSO Validation Project (C3VP) held during January 2007 in Ontario, Canada. The calibration used internal calibration path data and was confirmed through comparison of precipitation reflectivity with an Environment Canada radar. The calibrated data was then used to estimate the median mass diameter of precipitating snow from a high-priority C3VP data set. This median mass diameter retrieval was compared to the results from a local ground instrument, the Snow Video Imager (SVI), showing good agreement

Visual Search at eBay

In this paper, we propose a novel end-to-end approach for scalable visual search infrastructure. We discuss the challenges we faced for a massive volatile inventory like at eBay and present our solution to overcome those. We harness the availability of large image collection of eBay listings and state-of-the-art deep learning techniques to perform visual search at scale. Supervised approach for optimized search limited to top predicted categories and also for compact binary signature are key to scale up without compromising accuracy and precision. Both use a common deep neural network requiring only a single forward inference. The system architecture is presented with in-depth discussions of its basic components and optimizations for a trade-off between search relevance and latency. This solution is currently deployed in a distributed cloud infrastructure and fuels visual search in eBay ShopBot and Close5. We show benchmark on ImageNet dataset on which our approach is faster and more accurate than several unsupervised baselines. We share our learnings with the hope that visual search becomes a first class citizen for all large scale search engines rather than an afterthought.Comment: To appear in 23rd SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2017. A demonstration video can be found at https://youtu.be/iYtjs32vh4

Management and display of four-dimensional environmental data sets using McIDAS

Over the past four years, great strides have been made in the areas of data management and display of 4-D meteorological data sets. A survey was conducted of available and planned 4-D meteorological data sources. The data types were evaluated for their impact on the data management and display system. The requirements were analyzed for data base management generated by the 4-D data display system. The suitability of the existing data base management procedures and file structure were evaluated in light of the new requirements. Where needed, new data base management tools and file procedures were designed and implemented. The quality of the basic 4-D data sets was assured. The interpolation and extrapolation techniques of the 4-D data were investigated. The 4-D data from various sources were combined to make a uniform and consistent data set for display purposes. Data display software was designed to create abstract line graphic 3-D displays. Realistic shaded 3-D displays were created. Animation routines for these displays were developed in order to produce a dynamic 4-D presentation. A prototype dynamic color stereo workstation was implemented. A computer functional design specification was produced based on interactive studies and user feedback