Rethinking summarization and storytelling for modern social multimedia

Traditional summarization initiatives have been focused on specific types of documents such as articles, reviews, videos, image feeds, or tweets, a practice which may result in pigeonholing the summarization task in the context of modern, content-rich multimedia collections. Consequently, much of the research to date has revolved around mostly toy problems in narrow domains and working on single-source media types. We argue that summarization and story generation systems need to re-focus the problem space in order to meet the information needs in the age of user-generated content in different formats and languages. Here we create a framework for flexible multimedia storytelling. Narratives, stories, and summaries carry a set of challenges in big data and dynamic multi-source media that give rise to new research in spatial-temporal representation, viewpoint generation, and explanatio

A Review on Personalized Tag based Image based Search Engines

The development of social media based on Web 2.0, amounts of images and videos spring up everywhere on the Internet. This phenomenon has brought great challenges to multimedia storage, indexing and retrieval. Generally speaking, tag-based image search is more commonly used in social media than content based image retrieval and content understanding. Thanks to the low relevance and diversity performance of initial retrieval results, the ranking problem in the tag-based image retrieval has gained researchers� wide attention. We will review some of techniques proposed by different authors for image retrieval in this paper

The Faculty Notebook, April 2018

The Faculty Notebook is published periodically by the Office of the Provost at Gettysburg College to bring to the attention of the campus community accomplishments and activities of academic interest. Faculty are encouraged to submit materials for consideration for publication to the Associate Provost for Faculty Development. Copies of this publication are available at the Office of the Provost

CHORUS Deliverable 2.1: State of the Art on Multimedia Search Engines

Based on the information provided by European projects and national initiatives related to multimedia search as well as domains experts that participated in the CHORUS Think-thanks and workshops, this document reports on the state of the art related to multimedia content search from, a technical, and socio-economic perspective. The technical perspective includes an up to date view on content based indexing and retrieval technologies, multimedia search in the context of mobile devices and peer-to-peer networks, and an overview of current evaluation and benchmark inititiatives to measure the performance of multimedia search engines. From a socio-economic perspective we inventorize the impact and legal consequences of these technical advances and point out future directions of research

Beautiful and damned. Combined effect of content quality and social ties on user engagement

User participation in online communities is driven by the intertwinement of the social network structure with the crowd-generated content that flows along its links. These aspects are rarely explored jointly and at scale. By looking at how users generate and access pictures of varying beauty on Flickr, we investigate how the production of quality impacts the dynamics of online social systems. We develop a deep learning computer vision model to score images according to their aesthetic value and we validate its output through crowdsourcing. By applying it to over 15B Flickr photos, we study for the first time how image beauty is distributed over a large-scale social system. Beautiful images are evenly distributed in the network, although only a small core of people get social recognition for them. To study the impact of exposure to quality on user engagement, we set up matching experiments aimed at detecting causality from observational data. Exposure to beauty is double-edged: following people who produce high-quality content increases one's probability of uploading better photos; however, an excessive imbalance between the quality generated by a user and the user's neighbors leads to a decline in engagement. Our analysis has practical implications for improving link recommender systems.Comment: 13 pages, 12 figures, final version published in IEEE Transactions on Knowledge and Data Engineering (Volume: PP, Issue: 99

A Deep Learning-Based Pipeline for the Generation of Synthetic Tabular Data

The recent and rapid progresses in Machine Learning (ML) tools and methodologies paved the way for an accessible market of ML services. In principle, small and medium-sized enterprises, as well as big companies, could act as providers and consumers of services, resulting in an intense exchange of ML services where a consumer may ask many providers for a service preview based on its particular business case, that is, its data. In practice, however, many potential service consumers are reluctant to release their data, when seeking for ML services, because of privacy or intellectual property concerns. As a consequence, the market of ML services is not as fluid as it could be. An alternative to providing real data when looking for an ML service consists in generating and releasing synthetic data. The synthetic data should 1) allow the service provider to preview an ML service whose performance is predictive of the one the same service will achieve on the real data; and 2) prevent the disclosure of the real data. In this paper, we propose a data synthesis technique tailored to a family of very relevant business cases: supervised and unsupervised learning on single-table datasets and relational datasets. Our technique is based on generative deep learning models and we instantiate it in three variants: standard Variational Autoencoders (VAEs), β -VAEs, and Introspective VAEs. We experimentally evaluate the two variants to measure the degree to which they meet the two requirements above, using several performance indexes that capture different aspects of the quality of the generated data. The results suggest that data synthesis is a practical answer to the need of decoupling ML service providers and consumers and, hence, can favor the arising of an active and accessible market of ML services

Connecting the microscopic depolarizing origin of samples with macroscopic measures of the Indices of Polarimetric Purity

In this work we show how a specific set of three depolarizing observables, the Indices of Polarimetric Purity (IPP), P1, P2 and P3, are ideal metrics to study the depolarization characteristic of media. We simulate different depolarizing scenarios, based on different depolarizing origins, and we study the corresponding IPP values. The simulations are based on the incoherent addition of multiple elemental polarizing elements, as ideal polarizers and/or retarders with different specific characteristics (orientation, retardance, transmittance, etc.). Further depolarizing scenarios are also studied by including the effect of ideal depolarizers. We show for the first time how by analyzing depolarizing systems through IPP we unravel two different depolarizing origins: isotropic and anisotropic depolarization, with meaningful physical interpretation. The former, isotropic depolarization is related to pure scattering processes, and mainly connected with P3 observable. The later, anisotropic depolarization is originated by microscopic constituent elements showing polarimetric anisotropy (dichroic and/or birefringent elements with different characteristics) and anisotropic scattering produced by these elements, and mainly described by P1 and P2 observables. Both effects can be simultaneously observed in real samples and give us information of the processes that give rise to depolarization in light-matter interactions. The simulated results are experimentally validated by analyzing the depolarizing behavior, in terms of IPP, of diverse real samples with easy physical interpretation, and direct connection with simulations. The present study could be of interest in multiple scenarios, to further understand the depolarizing response of samples, and it can be of special interest for the study of biological tissues and pathologies, as they present important depolarizing behavior.Monica Canabal-Carbia reports financial support was provided by Spain Ministry of Science and Innovation (PID2021-560 126509OB-C21 and PDC2022-133332-C21). Juan Campos reports financial support was provided by Spain Ministry of Science and Innovation (PID2021-560 126509OB-C21 and PDC2022-133332-C21). Angel Lizana reports financial support was provided by Spain Ministry of Science and Innovation (PID2021-560 126509OB-C21 and PDC2022-133332-C21). Irene Estevez reports financial support was provided by Government of Catalonia (Beatriu de Pinos, 2021-BP-00206). Ignacio Moreno reports financial support was provided by Spain Ministry of Science and Innovation (PID2021-126509OB-C22). Andres Marquez reports financial support was provided by Government of Valencia. Andres Marquez reports financial support was provided by Spain Ministry of Science and Innovation ( PID2021-123124OB-I00). Esther Nabadda reports financial support was provided by Government of Valencia. Mónica Canabal-Carbia, Angel Lizana and Juan Campos reports financial support was provided by the Generalitat de Catalunya (2021SGR00138)