eStorys: A visual storyboard system supporting back-channel communication for emergencies

This is the post-print version of the final paper published in Journal of Visual Languages & Computing. The published article is available from the link below. Changes resulting from the publishing process, such as peer review, editing, corrections, structural formatting, and other quality control mechanisms may not be reflected in this document. Changes may have been made to this work since it was submitted for publication. Copyright @ 2010 Elsevier B.V.In this paper we present a new web mashup system for helping people and professionals to retrieve information about emergencies and disasters. Today, the use of the web during emergencies, is confirmed by the employment of systems like Flickr, Twitter or Facebook as demonstrated in the cases of Hurricane Katrina, the July 7, 2005 London bombings, and the April 16, 2007 shootings at Virginia Polytechnic University. Many pieces of information are currently available on the web that can be useful for emergency purposes and range from messages on forums and blogs to georeferenced photos. We present here a system that, by mixing information available on the web, is able to help both people and emergency professionals in rapidly obtaining data on emergency situations by using multiple web channels. In this paper we introduce a visual system, providing a combination of tools that demonstrated to be effective in such emergency situations, such as spatio/temporal search features, recommendation and filtering tools, and storyboards. We demonstrated the efficacy of our system by means of an analytic evaluation (comparing it with others available on the web), an usability evaluation made by expert users (students adequately trained) and an experimental evaluation with 34 participants.Spanish Ministry of Science and Innovation and Universidad Carlos III de Madrid and Banco Santander

Learning Visual Clothing Style with Heterogeneous Dyadic Co-occurrences

With the rapid proliferation of smart mobile devices, users now take millions of photos every day. These include large numbers of clothing and accessory images. We would like to answer questions like `What outfit goes well with this pair of shoes?' To answer these types of questions, one has to go beyond learning visual similarity and learn a visual notion of compatibility across categories. In this paper, we propose a novel learning framework to help answer these types of questions. The main idea of this framework is to learn a feature transformation from images of items into a latent space that expresses compatibility. For the feature transformation, we use a Siamese Convolutional Neural Network (CNN) architecture, where training examples are pairs of items that are either compatible or incompatible. We model compatibility based on co-occurrence in large-scale user behavior data; in particular co-purchase data from Amazon.com. To learn cross-category fit, we introduce a strategic method to sample training data, where pairs of items are heterogeneous dyads, i.e., the two elements of a pair belong to different high-level categories. While this approach is applicable to a wide variety of settings, we focus on the representative problem of learning compatible clothing style. Our results indicate that the proposed framework is capable of learning semantic information about visual style and is able to generate outfits of clothes, with items from different categories, that go well together.Comment: ICCV 201

Portinari: A Data Exploration Tool to Personalize Cervical Cancer Screening

Socio-technical systems play an important role in public health screening programs to prevent cancer. Cervical cancer incidence has significantly decreased in countries that developed systems for organized screening engaging medical practitioners, laboratories and patients. The system automatically identifies individuals at risk of developing the disease and invites them for a screening exam or a follow-up exam conducted by medical professionals. A triage algorithm in the system aims to reduce unnecessary screening exams for individuals at low-risk while detecting and treating individuals at high-risk. Despite the general success of screening, the triage algorithm is a one-size-fits all approach that is not personalized to a patient. This can easily be observed in historical data from screening exams. Often patients rely on personal factors to determine that they are either at high risk or not at risk at all and take action at their own discretion. Can exploring patient trajectories help hypothesize personal factors leading to their decisions? We present Portinari, a data exploration tool to query and visualize future trajectories of patients who have undergone a specific sequence of screening exams. The web-based tool contains (a) a visual query interface (b) a backend graph database of events in patients' lives (c) trajectory visualization using sankey diagrams. We use Portinari to explore diverse trajectories of patients following the Norwegian triage algorithm. The trajectories demonstrated variable degrees of adherence to the triage algorithm and allowed epidemiologists to hypothesize about the possible causes.Comment: Conference paper published at ICSE 2017 Buenos Aires, at the Software Engineering in Society Track. 10 pages, 5 figure

Algorithms that Remember: Model Inversion Attacks and Data Protection Law

Many individuals are concerned about the governance of machine learning systems and the prevention of algorithmic harms. The EU's recent General Data Protection Regulation (GDPR) has been seen as a core tool for achieving better governance of this area. While the GDPR does apply to the use of models in some limited situations, most of its provisions relate to the governance of personal data, while models have traditionally been seen as intellectual property. We present recent work from the information security literature around `model inversion' and `membership inference' attacks, which indicate that the process of turning training data into machine learned systems is not one-way, and demonstrate how this could lead some models to be legally classified as personal data. Taking this as a probing experiment, we explore the different rights and obligations this would trigger and their utility, and posit future directions for algorithmic governance and regulation.Comment: 15 pages, 1 figur

Development of an Ontology of Tourist Attractions for Recommending Points of Interest in a Group Recommender System for Tourism

In recent years, the tourism industry has witnessed substantial growth, thanks to the pro liferation of digital technology and online platforms. Tourists now have greater access to information and the ability to make informed travel decisions. However, the abundance of available information often leaves tourists overwhelmed when selecting points of inter est (POI) that align with their preferences. Recommender Systems (RS) have emerged as a solution, personalising recommendations based on tourist behaviour, social networks, and contextual factors. To enhance RS efficacy, researchers have begun exploring the integration of psychological factors, such as personality traits. Yet, to meet the demands of modern tourists, a robust knowledge base, such as a tourist attractions ontology, is essential for seamless and rapid matching of tourist characteristics and preferences with available POI. With that in mind, this project aims to enhance a Group Recommender System (GRS) prototype, GrouPlanner, by creating a robust tourist attractions ontology. This ontology will facilitate rapid and accurate matching of points of interest with tourists’ character istics, including personality, preferences, and demographic data, ultimately improving POI recommendations. First, there needs to be an understanding of the personality of tourists and how it influences their choices when it comes to picking the best point of interest based on their personality. With that knowledge acquired, it is time to choose a way to represent this knowledge in the form of an ontology. In this project, the Protégé ontology editor was used to design the ontology and the rela tionships between the tourists’ personality and the points of interest. After designing the ontology, it had to be converted to a database so the Grouplanner system could access it. So, to do that, a solution was designed to integrate the designed ontology in a triple store data base, in this case, Apache Fuseki. With the database implemented, several tests were made to verify if the database would give the recommended points of interests based on the tourists’ preferences. This tests were later analysed.Nos anos mais recentes, a indústria do turismo presenciou um crescimento substancial dev ido à tecnologia digital e plataformas online. Cada vez mais, os turistas têm acesso a uma abundância de informação que influencia a habilidade de tomar decisões sobre viajar. No entanto, esta informação pode complicar a seleção dos pontos de interesse que alinhem com as preferências dos turistas. Para combater isso, sistemas de recomendação (SR) emergi ram como uma solução, personalizando as recomendações com base no comportamento do turista, redes socias e outros fatores. Para aumentar a eficácia destes sistemas, os investi gadores começaram a explorar a possibilidade de integração com fatores psicológicos, como traços de personalidade. Apesar disso, para cumprir as exigências dos turistas modernos, uma base de conhecimento robusta, como uma ontologia de atrações turísticas, é essencial para, de forma eficaz e eficiente, corresponder as características dos turistas com os pontos de interesse disponíveis. Com isso em mente, este projeto tem como objetivo melhorar um protótipo de um sistema de recomendação (GrouPlanner), criando uma ontologia robusta de atrações turísticas. Essa ontologia facilitará a correspondência rápida e precisa de pontos de interesse com as car acterísticas dos turistas, incluindo a sua personalidade e as suas preferências, melhorando assim as recomendações de pontos de interesse. Em primeiro lugar, é necessário compreender a personalidade dos turistas e como ela influ encia as suas escolhas ao selecionar o melhor ponto de interesse com base na sua person alidade. Com esse ponto adquirido, é necessário escolher uma maneira de representar esse conhecimento na forma de uma ontologia. Neste projeto, o editor de ontologias Protégé foi utilizado para projetar a ontologia e as relações entre a personalidade dos turistas e os pontos de interesse. Após a construção da ontologia, foi necessário convertê-la numa base de dados para que o sistema Grouplanner pudesse ter acesso. Para isso, foi desenhada uma solução para integrar a ontologia projetada numa base de dados "triple store", neste caso, o Apache Fuseki. Com a base de dados implementada, foram realizados vários testes para verificar se esta forneceria os pontos de interesse recomendados com base nas preferências dos turistas. Esses testes foram depois analisados