5 research outputs found
Look, Read and Feel: Benchmarking Ads Understanding with Multimodal Multitask Learning
Given the massive market of advertising and the sharply increasing online
multimedia content (such as videos), it is now fashionable to promote
advertisements (ads) together with the multimedia content. It is exhausted to
find relevant ads to match the provided content manually, and hence, some
automatic advertising techniques are developed. Since ads are usually hard to
understand only according to its visual appearance due to the contained visual
metaphor, some other modalities, such as the contained texts, should be
exploited for understanding. To further improve user experience, it is
necessary to understand both the topic and sentiment of the ads. This motivates
us to develop a novel deep multimodal multitask framework to integrate multiple
modalities to achieve effective topic and sentiment prediction simultaneously
for ads understanding. In particular, our model first extracts multimodal
information from ads and learn high-level and comparable representations. The
visual metaphor of the ad is decoded in an unsupervised manner. The obtained
representations are then fed into the proposed hierarchical multimodal
attention modules to learn task-specific representations for final prediction.
A multitask loss function is also designed to train both the topic and
sentiment prediction models jointly in an end-to-end manner. We conduct
extensive experiments on the latest and large advertisement dataset and achieve
state-of-the-art performance for both prediction tasks. The obtained results
could be utilized as a benchmark for ads understanding.Comment: 8 pages, 5 figure
A Survey of Operations Research and Analytics Literature Related to Anti-Human Trafficking
Human trafficking is a compound social, economic, and human rights issue
occurring in all regions of the world. Understanding and addressing such a
complex crime requires effort from multiple domains and perspectives. As of
this writing, no systematic review exists of the Operations Research and
Analytics literature applied to the domain of human trafficking. The purpose of
this work is to fill this gap through a systematic literature review. Studies
matching our search criteria were found ranging from 2010 to March 2021. These
studies were gathered and analyzed to help answer the following three research
questions: (i) What aspects of human trafficking are being studied by
Operations Research and Analytics researchers? (ii) What Operations Research
and Analytics methods are being applied in the anti-human trafficking domain?
and (iii) What are the existing research gaps associated with (i) and (ii)? By
answering these questions, we illuminate the extent to which these topics have
been addressed in the literature, as well as inform future research
opportunities in applying analytical methods to advance the fight against human
trafficking.Comment: 28 pages, 6 Figures, 2 Table
Computer Vision for Multimedia Geolocation in Human Trafficking Investigation: A Systematic Literature Review
The task of multimedia geolocation is becoming an increasingly essential
component of the digital forensics toolkit to effectively combat human
trafficking, child sexual exploitation, and other illegal acts. Typically,
metadata-based geolocation information is stripped when multimedia content is
shared via instant messaging and social media. The intricacy of geolocating,
geotagging, or finding geographical clues in this content is often overly
burdensome for investigators. Recent research has shown that contemporary
advancements in artificial intelligence, specifically computer vision and deep
learning, show significant promise towards expediting the multimedia
geolocation task. This systematic literature review thoroughly examines the
state-of-the-art leveraging computer vision techniques for multimedia
geolocation and assesses their potential to expedite human trafficking
investigation. This includes a comprehensive overview of the application of
computer vision-based approaches to multimedia geolocation, identifies their
applicability in combating human trafficking, and highlights the potential
implications of enhanced multimedia geolocation for prosecuting human
trafficking. 123 articles inform this systematic literature review. The
findings suggest numerous potential paths for future impactful research on the
subject
Large-Scale Multimedia Content Analysis Using Scientific workflows
Analyzing web content, particularly multimedia content, for security applications is of great interest. However, it often requires deep expertise in data analytics that is not always accessible to non-experts. Our approach is to use scientific workflows that capture expert-level methods to examine web content. We use workflows to analyze the image and text components of multimedia web posts separately, as well as by a multimodal fusion of both image and text data. In particular, we re-purpose workflow fragments to do the multimedia analysis and create additional components for the fusion of the image and text modalities. In this paper, we present preliminary work which focuses on a Human Trafficking Detection task to help deter human trafficking of minors by thus fusing image and text content from the web. We also examine how workflow fragments save time and effort in multimedia content analysis while bringing together multiple areas of machine learning and computer vision. We further export these workflow fragments using linked data as web objects
Proyecto Docente e Investigador, Trabajo Original de Investigación y Presentación de la Defensa, preparado por Germán Moltó para concursar a la plaza de Catedrático de Universidad, concurso 082/22, plaza 6708, área de Ciencia de la Computación e Inteligencia Artificial
Este documento contiene el proyecto docente e investigador del candidato Germán Moltó MartÃnez presentado como requisito para el concurso de acceso a plazas de Cuerpos Docentes Universitarios. Concretamente, el documento se centra en el concurso para la plaza 6708 de Catedrático de Universidad en el área de Ciencia de la Computación en el Departamento de Sistemas Informáticos y Computación de la Universitat Politécnica de València. La plaza está adscrita a la Escola Técnica Superior d'Enginyeria Informà tica y tiene como perfil las asignaturas "Infraestructuras de Cloud Público" y "Estructuras de Datos y Algoritmos".También se incluye el Historial Académico, Docente e Investigador, asà como la presentación usada durante la defensa.Germán Moltó MartÃnez (2022). Proyecto Docente e Investigador, Trabajo Original de Investigación y Presentación de la Defensa, preparado por Germán Moltó para concursar a la plaza de Catedrático de Universidad, concurso 082/22, plaza 6708, área de Ciencia de la Computación e Inteligencia Artificial. http://hdl.handle.net/10251/18903