Combining Visual and Textual Features for Semantic Segmentation of Historical Newspapers

The massive amounts of digitized historical documents acquired over the last decades naturally lend themselves to automatic processing and exploration. Research work seeking to automatically process facsimiles and extract information thereby are multiplying with, as a first essential step, document layout analysis. If the identification and categorization of segments of interest in document images have seen significant progress over the last years thanks to deep learning techniques, many challenges remain with, among others, the use of finer-grained segmentation typologies and the consideration of complex, heterogeneous documents such as historical newspapers. Besides, most approaches consider visual features only, ignoring textual signal. In this context, we introduce a multimodal approach for the semantic segmentation of historical newspapers that combines visual and textual features. Based on a series of experiments on diachronic Swiss and Luxembourgish newspapers, we investigate, among others, the predictive power of visual and textual features and their capacity to generalize across time and sources. Results show consistent improvement of multimodal models in comparison to a strong visual baseline, as well as better robustness to high material variance

Research knows best, but how to communicate distraction measures practically in an industrial context

Selection and comparison of human-factors related measures for evaluations of in-vehicle devices involves weighting of multiple criteria. It may result in a complex decision-making process for the practitioner, specifically in a time pressured industrial context. Visual information seeking has successfully been applied to reduce the complexity of datasets in healthcare and other fields. Information is presented visually and divided in ‘Overview’, representing the data by its characteristic criteria, and ‘Details’, which are presented on demand. This division reduces information load for the user and eases comparison based on characteristics. This project, first, aims to understand what criteria practitioners use to decide about the suitability of a measure for an in-vehicle evaluation. Secondly, criteria practitioners use to select measures are implemented in a new interface approach based on methods of visual information seeking to support users in the selection and comparison of human-factors related measures for in-vehicle evaluations. Overall, the interface exposes practitioners to new measures, enables them to rapidly compare measures, and obtain information to practically apply the

Visual Analysis of Spatio-Temporal Event Predictions: Investigating the Spread Dynamics of Invasive Species

Invasive species are a major cause of ecological damage and commercial losses. A current problem spreading in North America and Europe is the vinegar fly Drosophila suzukii. Unlike other Drosophila, it infests non-rotting and healthy fruits and is therefore of concern to fruit growers, such as vintners. Consequently, large amounts of data about infestations have been collected in recent years. However, there is a lack of interactive methods to investigate this data. We employ ensemble-based classification to predict areas susceptible to infestation by D. suzukii and bring them into a spatio-temporal context using maps and glyph-based visualizations. Following the information-seeking mantra, we provide a visual analysis system Drosophigator for spatio-temporal event prediction, enabling the investigation of the spread dynamics of invasive species. We demonstrate the usefulness of this approach in two use cases

Information Seeking in Context: Teachers' Content Selection during Lesson Planning Using the Shoah Foundation's Visual History Archive of Holocaust Survivor Testimony

This study explored the information seeking task of content selection. An integrative conceptual framework used existing models to examine the context and process of information seeking, evaluation, and selection. The conceptual framework incorporated three main elements of the information seeking process: * The information need context, * The information search process, * Relevance criteria. Among teachers' many duties are the creation, implementation, and revision of lesson plans. A subtask of lesson planning is content selection, which occurs when teachers seek outside content, such as readings or audio recordings, to incorporate into lesson plans. Content selection is seen here as a work-task-embedded information seeking process. A qualitative study was implemented within the setting of a week-long professional development workshop, during which eight teachers used a custom software product that combined a lesson-planning module with an information retrieval (IR) system. The IR system provided access to a subset of the Shoah Foundation's Visual History Archive. Data types included interviews, fly-on-the-wall transcripts, transaction logs, relevance judgments, and lesson plans. Analysis combined inductive and deductive techniques, including start codes, constant comparison, emergent themes, and matrix analysis. Findings depict associations among each component of the framework. 1. The information need context consists of five layers (Environment, Role, Person, Task, Information Source), each of which influences information search and relevance. 2. The ISP includes two cognitive-behavioral facets: Conceptualizing and Actualizing. 3. Relevance criteria are the situationally-driven embodiment of contextual elements that apply to information seeking. These findings have theoretical and practical implications for information studies and education. For information studies, this study contributes to understanding of the ISP as contextual, cognitive, and interactive. Information need, while unobservable in its native form, can be depicted in enough detail to supply meaningful requirements for the design of information systems and processes. Content selection is a form of exploratory search, and this study's implications suggest that the "traditional" reference interview should be used as an interaction model during exploratory search. For education, this study extends the discourse about consequences of standards-based education for teacher practice and contributes to models of teacher planning as an iterative, cognitive process

Netværksbaseret læring

This paper discusses pedagogical and technological aspects of networked learning and the interplay between pedagogical principles, information technology and the educa-tional and organisational frameworks in the context of a distance-teaching course for Danish graduates seeking research-based further education and training in English for medical purposes. The empirical basis is an evaluation report of a second generation distance-teaching course where applied information technologies are geared to maxi-mum pedagogical efficiency by presenting on-line course materials in a flat visual de-sign and by facilitating dialogue between course participants working in teams

Stacked Cross-modal Feature Consolidation Attention Networks for Image Captioning

Recently, the attention-enriched encoder-decoder framework has aroused great interest in image captioning due to its overwhelming progress. Many visual attention models directly leverage meaningful regions to generate image descriptions. However, seeking a direct transition from visual space to text is not enough to generate fine-grained captions. This paper exploits a feature-compounding approach to bring together high-level semantic concepts and visual information regarding the contextual environment fully end-to-end. Thus, we propose a stacked cross-modal feature consolidation (SCFC) attention network for image captioning in which we simultaneously consolidate cross-modal features through a novel compounding function in a multi-step reasoning fashion. Besides, we jointly employ spatial information and context-aware attributes (CAA) as the principal components in our proposed compounding function, where our CAA provides a concise context-sensitive semantic representation. To make better use of consolidated features potential, we further propose an SCFC-LSTM as the caption generator, which can leverage discriminative semantic information through the caption generation process. The experimental results indicate that our proposed SCFC can outperform various state-of-the-art image captioning benchmarks in terms of popular metrics on the MSCOCO and Flickr30K datasets

Information for inspiration: understanding information-seeking behaviour and library usage of students at the Hong Kong Design Institute

The process of information- and inspiration-seeking behaviour amongst artists and designers often involve direct observation, note-taking, collecting materials and image samples, recognising styles, analysing movements, patterns, textures, as well as experimenting with different materials and techniques. They also rely heavily on having access to a variety of visual resources, both physical and digital, during the process of inspiration-seeking. However, there have been few studies on how art and design students look for and use information in the digital age, especially in the context of the library. This paper reports on an empirical study of the inspiration-seeking process and other information-related behaviour of students at the Hong Kong Design Institute (HKDI). An online questionnaire was created to ask the HKDI students specific questions: the types of library preferred; students\u27 comfort level with the HKDI Library; student respondents\u27 information needs; and their preferred sources for inspiration. They were also asked which media and venues they looked to for information that was important to their creative process. A total of 327 current students at the HKDI completed the survey. The research findings suggest that information-seeking behaviour of the art and design students was reflective of the fluid and creative nature of the art and design domain. They were regular users of traditional printed resources as well as the physical libraries. They also placed heavy reliance on the Internet and a variety of social networks when it came to inspiration-seeking. Inspiration was found from a very diverse and ‘idiosyncratic’ set of sources; often via accidental discovery. The students\u27 status as emergent practitioners also suggested a strong need for career advice and interactions with peers practicing in the art and design field

What Is Known About the Impact of Impairments on Information Seeking and Searching?

Information seeking and access are essential for users in all walks of life, from addressing personal needs such as finding flights to locating information needed to complete work tasks. Over the past decade or so, the general needs of people with impairments have increasingly been recognized as something to be addressed, an issue embedded both in international treaties and in state legislation. The same tendency can be found in research, where a growing number of user studies including people with impairments have been conducted. The purpose of these studies is typically to uncover potential barriers for access to information, especially in the context of inaccessible search user interfaces. This literature review provides an overview of research on the information seeking and searching of users with impairments. The aim is to provide an overview to both researchers and practitioners who work with any of the user groups identified. Some diagnoses are relatively well represented in the literature (for instance, visual impairment), but there is very little work in other areas (for instance, autism) and in some cases no work at all (for instance, aphasia). Gaps are identified in the research, and suggestions are made regarding areas where further research is needed

Combining Visual and Textual Features for Semantic Segmentation of Historical Newspapers

The massive amounts of digitized historical documents acquired over the last decades naturally lend themselves to automatic processing and exploration. Research work seeking to automatically process facsimiles and extract information thereby are multiplying with, as a first essential step, document layout analysis. If the identification and categorization of segments of interest in document images have seen significant progress over the last years thanks to deep learning techniques, many challenges remain with, among others, the use of finer-grained segmentation typologies and the consideration of complex, heterogeneous documents such as historical newspapers. Besides, most approaches consider visual features only, ignoring textual signal. In this context, we introduce a multimodal approach for the semantic segmentation of historical newspapers that combines visual and textual features. Based on a series of experiments on diachronic Swiss and Luxembourgish newspapers, we investigate, among others, the predictive power of visual and textual features and their capacity to generalize across time and sources. Results show consistent improvement of multimodal models in comparison to a strong visual baseline, as well as better robustness to high material variance

Use of the Web by Visual Artists: An Exploration of How Online Information Seeking Informs Creative Practice

Visual artists' information-seeking behavior takes place in a broad context, involving interaction with a range of visual, textual, environmental, process-related and interpersonal sources. The World Wide Web (or Web) is one such resource that artists turn to within this vast information setting, but to-date, no known studies have examined how artists interact with information online. The present study addresses this gap by exploring non-academic visual artists' use of the Web as it relates to their creative activity. Diaries and interviews were used in order to understand participants' artistic practices and related information needs, as well as their sources, search strategies, and motivations for Web use. The artists' overall information needs matched those identified in previous studies. This study discovered that they use the Web primarily as a tool to promote their art, identify opportunities to further their careers, and socially network. Their use of the Web is connected to various offline information-seeking behaviors, showing that it serves to complement, rather than supplant, many of the sources they consult