Cognitive Foundations for Visual Analytics

In this report, we provide an overview of scientific/technical literature on information visualization and VA. Topics discussed include an update and overview of the extensive literature search conducted for this study, the nature and purpose of the field, major research thrusts, and scientific foundations. We review methodologies for evaluating and measuring the impact of VA technologies as well as taxonomies that have been proposed for various purposes to support the VA community. A cognitive science perspective underlies each of these discussions

Embedded electronic systems driven by run-time reconfigurable hardware

Abstract This doctoral thesis addresses the design of embedded electronic systems based on run-time reconfigurable hardware technology –available through SRAM-based FPGA/SoC devices– aimed at contributing to enhance the life quality of the human beings. This work does research on the conception of the system architecture and the reconfiguration engine that provides to the FPGA the capability of dynamic partial reconfiguration in order to synthesize, by means of hardware/software co-design, a given application partitioned in processing tasks which are multiplexed in time and space, optimizing thus its physical implementation –silicon area, processing time, complexity, flexibility, functional density, cost and power consumption– in comparison with other alternatives based on static hardware (MCU, DSP, GPU, ASSP, ASIC, etc.). The design flow of such technology is evaluated through the prototyping of several engineering applications (control systems, mathematical coprocessors, complex image processors, etc.), showing a high enough level of maturity for its exploitation in the industry.Resumen Esta tesis doctoral abarca el diseño de sistemas electrónicos embebidos basados en tecnología hardware dinámicamente reconfigurable –disponible a través de dispositivos lógicos programables SRAM FPGA/SoC– que contribuyan a la mejora de la calidad de vida de la sociedad. Se investiga la arquitectura del sistema y del motor de reconfiguración que proporcione a la FPGA la capacidad de reconfiguración dinámica parcial de sus recursos programables, con objeto de sintetizar, mediante codiseño hardware/software, una determinada aplicación particionada en tareas multiplexadas en tiempo y en espacio, optimizando así su implementación física –área de silicio, tiempo de procesado, complejidad, flexibilidad, densidad funcional, coste y potencia disipada– comparada con otras alternativas basadas en hardware estático (MCU, DSP, GPU, ASSP, ASIC, etc.). Se evalúa el flujo de diseño de dicha tecnología a través del prototipado de varias aplicaciones de ingeniería (sistemas de control, coprocesadores aritméticos, procesadores de imagen, etc.), evidenciando un nivel de madurez viable ya para su explotación en la industria.Resum Aquesta tesi doctoral està orientada al disseny de sistemes electrònics empotrats basats en tecnologia hardware dinàmicament reconfigurable –disponible mitjançant dispositius lògics programables SRAM FPGA/SoC– que contribueixin a la millora de la qualitat de vida de la societat. S’investiga l’arquitectura del sistema i del motor de reconfiguració que proporcioni a la FPGA la capacitat de reconfiguració dinàmica parcial dels seus recursos programables, amb l’objectiu de sintetitzar, mitjançant codisseny hardware/software, una determinada aplicació particionada en tasques multiplexades en temps i en espai, optimizant així la seva implementació física –àrea de silici, temps de processat, complexitat, flexibilitat, densitat funcional, cost i potència dissipada– comparada amb altres alternatives basades en hardware estàtic (MCU, DSP, GPU, ASSP, ASIC, etc.). S’evalúa el fluxe de disseny d’aquesta tecnologia a través del prototipat de varies aplicacions d’enginyeria (sistemes de control, coprocessadors aritmètics, processadors d’imatge, etc.), demostrant un nivell de maduresa viable ja per a la seva explotació a la indústria

Contextual Factors Affecting Information Sharing Patterns in Technology Mediated Communication

In this thesis, we investigate how and what contextual factors affect user’s information sharing. We build our work on six individual research projects which cover a variety of systems (search engines, social network sites, teleconferencing systems, monitoring technology, and general purpose conversational agents) in a variety of communication scenarios with diverse relationships and dispositions of users. Alongside detailed findings for particular systems and communication scenarios from each individual project, we provide a consolidated analysis of these results across systems and scenarios, which allows us to identify patterns specific for different system types and aspects shared between systems. In particular, we show that depending on the system’s position between a user and an intended information receiving agent – whether communication happens through, around, or directly with the system – the system should have different patterns of operational adaptation to communication context. Specifically, when communication happens through the system, the system needs to gather communication context unavailable to the user and integrate it into information communication; when communication happens around the system, the system should adapt its operations to provide information in the most contextually suitable format; finally, when a user communicates with the system, the role of the system is to “match” this context in communication with the user. We then argue that despite the differences between system types in patterns of required context-based adaptation, there are contextual factors affecting user’s information sharing intent that should be acknowledged across systems. Grounded in our cumulative findings and analysis of related literature, we identify four such high-level contextual factors. We then present these four factors synthesized into an early design framework, which we call SART according to the included factors of space, addressee, reason, and time. Each factor in SART is presented as a continuum defined through a descriptive dichotomy: perceived breadth of communication space (public to private); perceived specificity of an information addressee (defined to undefined); intended reason for information sharing (instrumental to objective); and perceived time of information relevance and life-span (immediate to indefinite)

Structure-aware narrative summarization from multiple views

Narratives, such as movies and TV shows, provide a testbed for addressing a variety of challenges in the field of artificial intelligence. They are examples of complex stories where characters and events interact in many ways. Inferring what is happening in a narrative requires modeling long-range dependencies between events, understanding commonsense knowledge and accounting for non-linearities in the presentation of the story. Moreover, narratives are usually long (i.e., there are hundreds of pages in a screenplay and thousands of frames in a video) and cannot be easily processed by standard neural architectures. Movies and TV episodes also include information from multiple sources (i.e., video, audio, text) that are complementary to inferring high-level events and their interactions. Finally, creating large-scale multimodal datasets with narratives containing long videos and aligned textual data is challenging, resulting in small datasets that require data efficient approaches. Most prior work that analyzes narratives does not consider the above challenges all at once. In most cases, text-only approaches focus on full-length narratives with complex semantics and address tasks such as question-answering and summarization, or multimodal approaches are limited to short videos with simpler semantics (e.g., isolated actions and local interactions). In this thesis, we combine these two different directions in addressing narrative summarization. We use all input modalities (i.e., video, audio, text), consider full-length narratives and perform the task of narrative summarization both in a video-to-video setting (i.e., video summarization, trailer generation) and a video-to-text setting (i.e., multimodal abstractive summarization). We hypothesize that information about the narrative structure of movies and TVepisodes can facilitate summarizing them. We introduce the task of Turning Point identification and provide a corresponding dataset called TRIPOD as a means of analyzing the narrative structure of movies. According to screenwriting theory, turning points (e.g., change of plans, major setback, climax) are crucial narrative moments within a movie or TV episode: they define the plot structure and determine its progression and thematic units. We validate that narrative structure contributes to extractive screenplay summarization by testing our hypothesis on a dataset containing TV episodes and summary-specific labels. We further hypothesize that movies should not be viewed as a sequence of scenes from a screenplay or shots from a video and instead be modelled as sparse graphs, where nodes are scenes or shots and edges denote strong semantic relationships between them. We utilize multimodal information for creating movie graphs in the latent space, and find that both graph-related and multimodal information help contextualization and boost performance on extractive summarization. Moving one step further, we also address the task of trailer moment identification, which can be viewed as a specific instiatiation of narrative summarization. We decompose this task, which is challenging and subjective, into two simpler ones: narrativestructure identification, defined again by turning points, and sentiment prediction. We propose a graph-based unsupervised algorithm that uses interpretable criteria for retrieving trailer shots and convert it into an interactive tool with a human in the loop for trailer creation. Semi-automatic trailer shot selection exhibits comparable performance to fully manual selection according to human judges, while minimizing processing time. After identifying salient content in narratives, we next attempt to produce abstractive textual summaries (i.e., video-to-text). We hypothesize that multimodal information is directly important for generating textual summaries, apart from contributing to content selection. For that, we propose a parameter efficient way for incorporating multimodal information into a pre-trained textual summarizer, while training only 3.8% of model parameters, and demonstrate the importance of multimodal information for generating high-quality and factual summaries. The findings of this thesis underline the need to focus on realistic and multimodal settings when addressing narrative analysis and generation tasks

Anales del XIII Congreso Argentino de Ciencias de la Computación (CACIC)

Contenido: Arquitecturas de computadoras Sistemas embebidos Arquitecturas orientadas a servicios (SOA) Redes de comunicaciones Redes heterogéneas Redes de Avanzada Redes inalámbricas Redes móviles Redes activas Administración y monitoreo de redes y servicios Calidad de Servicio (QoS, SLAs) Seguridad informática y autenticación, privacidad Infraestructura para firma digital y certificados digitales Análisis y detección de vulnerabilidades Sistemas operativos Sistemas P2P Middleware Infraestructura para grid Servicios de integración (Web Services o .Net)Red de Universidades con Carreras en Informática (RedUNCI

Anales del XIII Congreso Argentino de Ciencias de la Computación (CACIC)

Contenido: Arquitecturas de computadoras Sistemas embebidos Arquitecturas orientadas a servicios (SOA) Redes de comunicaciones Redes heterogéneas Redes de Avanzada Redes inalámbricas Redes móviles Redes activas Administración y monitoreo de redes y servicios Calidad de Servicio (QoS, SLAs) Seguridad informática y autenticación, privacidad Infraestructura para firma digital y certificados digitales Análisis y detección de vulnerabilidades Sistemas operativos Sistemas P2P Middleware Infraestructura para grid Servicios de integración (Web Services o .Net)Red de Universidades con Carreras en Informática (RedUNCI

An infrastructure for context-dependent RDF data replication on mobile devices

Der im Rahmen dieser Arbeit vorgestellte Ansatz beschreibt die Erstellung einer technischen Infrastruktur, die selektiv RDF-Daten in Abhängigkeit der Informationsbedürfnisse und den unterschiedlichen Kontexten mobiler Nutzer auf ein mobiles Endgerät repliziert und diese somit in intelligenter Art und Weise unterstützt. Eine Zusammenführung kontextspezifischer Konzepte und semantischer Technologien stellt einen wesentlichen Bestandteil zur Verbesserung der mobilen Informationssuche dar und erhöht gleichzeitig die Präzision mobiler Informationsgewinnungsprozesse. Trotz des vorhandenen Potentials einer proaktiven, kontextabhängigen Replizierung von RDF-Daten, gestaltet sich die Verarbeitung auf mobilen Endgeräten schwierig. Die Gründe dafür liegen in den technischen und netzwerkspezifischen Beschränkungen, in der fehlenden Verarbeitungs- und Verwaltungsfunktionalität von ontologiebasierten Beschreibungsverfahren sowie in der Unzulänglichkeit bestehender Replikationsansätze, sich an verändernde Informationsbedürfnisse sowie an unterschiedliche technische, umgebungsspezifische und infrastrukturbezogene Eigenheiten anzupassen. Verstärkt wird diese Problematik durch das Fehlen ausdrucksstarker Beschreibungsverfahren zur Repräsentation kontextspezifischer Daten. Existierende Ansätze leiden dementsprechend unter der Verwendung proprietärer Datenformate, dem Einsatz serverabhängiger Applikationsinfrastrukturen sowie dem Unvermögen, kontextspezifische Daten auszutauschen. Dies äußert sich in Studien, welche die Berücksichtigung der Informationsbedürfnisse mobiler Nutzer als unzureichend einstuft und einen Großteil der benötigten Informationen als kontextrelevant auszeichnet. Obgleich Fortschritte bei der Adaption von semantischen Technologien und Beschreibungsverfahren zur kontextabhängigen Verarbeitung zu erkennen sind, bleibt eine auf semantische Technologien basierende, proaktive Replizierung von RDF-Daten auf mobile Endgeräte ein offenes Forschungsfeld. Die vorliegende Arbeit diskutiert Möglichkeiten zur Erweiterung der mobilen, kontextspezifischen Datenverarbeitung durch semantische Technologien und beinhaltet eine vergleichende Studie zur Leistungsfähigkeit aktueller mobiler RDF-Frameworks. Kernpunkt ist die formale Beschreibung eines abstrakten Modells zur effizienten Akquise, Repräsentation, Verwaltung und Verarbeitung von Kontextinformationen unter Berücksichtigung der technischen Gegebenheiten mobiler Informationssysteme. Ergänzt wird es durch die formale Spezifikation eines nebenläufigen, transaktionsbasierten Verarbeitungsmodells, welches Vollständigkeits- und Konsistenzbedingungen auf Daten- und Prozessebene berücksichtigt. Der praktische Nutzen des vorliegenden Ansatzes wird anhand typischer Informationsbedürfnisse eines Wissensarbeiters demonstriert. Der Ansatz reduziert Abhängigkeiten zu externen Systemen und ermöglicht Nutzern, unabhängig von zeitlichen, örtlichen und netzwerkspezifischen Gegebenheiten, auf die für sie relevanten Daten zuzugreifen und diese zu verarbeiten. Durch die lokale Verarbeitung kontextbezogener Daten wird sowohl die Privatssphäre des Nutzers gewahrt als auch sicherheitsrelevanten Aspekten Rechnung getragen.This work describes an infrastructure for the selective RDF data replication to mobile devices while considering current and future information needs of mobile users and the different contexts they are operating in. It presents a novel approach in synthesizing context-aware computing concepts with semantic technologies and distributed transaction management concepts for intelligently assisting mobile users while enhancing mobile information seeking behavior and increasing the precision of mobile information retrieval processes. Despite the huge potential of a proactive, context-dependent replication of RDF data, such data can not be efficiently processed on mobile devices due to (i) technical limitations and network-related constraints, (ii) missing processing and management capabilities of ontology-based description frameworks, (iii) the inability of traditional data replication strategies to adapt to changing user information needs and to consider technical, environmental, and infrastructural restrictions of mobile operating systems, and (iv) the dynamic and emergent nature of context, which requires flexible and extensible description frameworks that allow for elaborating on the semantics of contextual constellations as well as on the relationships that exist between them. As a consequence, existing approaches suffer from the deployment of proprietary data formats, server-dependent application infrastructures, and the inability to share and exchange contextual information across system borders. Moreover, results of recently conducted studies reveal that mobile users find their information needs inadequately addressed, where a large share can be attributed as context or context-relevant. Although progress has been made in applying semantic technologies, concepts, and languages to the domain of context-aware computing, a synthesis of those fields for the proactive provision of RDF data replicas on mobile devices remains an open research issue. This work discusses possible fields where context-aware computing can be enhanced using technologies, languages, and concepts from the Semantic Web and contains a comparative study about the performance of current mobile RDF frameworks in replication-specific tasks. The main contribution of this thesis is a formal description of an abstract model that allows for an efficient acquisition, representation, management, and processing of contextual information while taking into account the peculiarities and operating environments of mobile information systems. It is complemented by a formal specification of a concurrently operating transaction-based processing model that considers completeness and consistency requirements on data and process level. We demonstrate the practicability of the presented approach trough a prototypical implementation of context and data providers that satisfy typical information needs of a mobile knowledge worker. As a consequence, dependencies to external systems are reduced and users are equipped with relevant information that adheres to their information needs anywhere and at any time, independent of any network-related constraints. Since context-relevant data are processed directly on a mobile device, security and privacy issues are preserved