Text–to–Video: Image Semantics and NLP

When aiming at automatically translating an arbitrary text into a visual story, the main challenge consists in finding a semantically close visual representation whereby the displayed meaning should remain the same as in the given text. Besides, the appearance of an image itself largely influences how its meaningful information is transported towards an observer. This thesis now demonstrates that investigating in both, image semantics as well as the semantic relatedness between visual and textual sources enables us to tackle the challenging semantic gap and to find a semantically close translation from natural language to a corresponding visual representation. Within the last years, social networking became of high interest leading to an enormous and still increasing amount of online available data. Photo sharing sites like Flickr allow users to associate textual information with their uploaded imagery. Thus, this thesis exploits this huge knowledge source of user generated data providing initial links between images and words, and other meaningful data. In order to approach visual semantics, this work presents various methods to analyze the visual structure as well as the appearance of images in terms of meaningful similarities, aesthetic appeal, and emotional effect towards an observer. In detail, our GPU-based approach efficiently finds visual similarities between images in large datasets across visual domains and identifies various meanings for ambiguous words exploring similarity in online search results. Further, we investigate in the highly subjective aesthetic appeal of images and make use of deep learning to directly learn aesthetic rankings from a broad diversity of user reactions in social online behavior. To gain even deeper insights into the influence of visual appearance towards an observer, we explore how simple image processing is capable of actually changing the emotional perception and derive a simple but effective image filter. To identify meaningful connections between written text and visual representations, we employ methods from Natural Language Processing (NLP). Extensive textual processing allows us to create semantically relevant illustrations for simple text elements as well as complete storylines. More precisely, we present an approach that resolves dependencies in textual descriptions to arrange 3D models correctly. Further, we develop a method that finds semantically relevant illustrations to texts of different types based on a novel hierarchical querying algorithm. Finally, we present an optimization based framework that is capable of not only generating semantically relevant but also visually coherent picture stories in different styles.Bei der automatischen Umwandlung eines beliebigen Textes in eine visuelle Geschichte, besteht die größte Herausforderung darin eine semantisch passende visuelle Darstellung zu finden. Dabei sollte die Bedeutung der Darstellung dem vorgegebenen Text entsprechen. Darüber hinaus hat die Erscheinung eines Bildes einen großen Einfluß darauf, wie seine bedeutungsvollen Inhalte auf einen Betrachter übertragen werden. Diese Dissertation zeigt, dass die Erforschung sowohl der Bildsemantik als auch der semantischen Verbindung zwischen visuellen und textuellen Quellen es ermöglicht, die anspruchsvolle semantische Lücke zu schließen und eine semantisch nahe Übersetzung von natürlicher Sprache in eine entsprechend sinngemäße visuelle Darstellung zu finden. Des Weiteren gewann die soziale Vernetzung in den letzten Jahren zunehmend an Bedeutung, was zu einer enormen und immer noch wachsenden Menge an online verfügbaren Daten geführt hat. Foto-Sharing-Websites wie Flickr ermöglichen es Benutzern, Textinformationen mit ihren hochgeladenen Bildern zu verknüpfen. Die vorliegende Arbeit nutzt die enorme Wissensquelle von benutzergenerierten Daten welche erste Verbindungen zwischen Bildern und Wörtern sowie anderen aussagekräftigen Daten zur Verfügung stellt. Zur Erforschung der visuellen Semantik stellt diese Arbeit unterschiedliche Methoden vor, um die visuelle Struktur sowie die Wirkung von Bildern in Bezug auf bedeutungsvolle Ähnlichkeiten, ästhetische Erscheinung und emotionalem Einfluss auf einen Beobachter zu analysieren. Genauer gesagt, findet unser GPU-basierter Ansatz effizient visuelle Ähnlichkeiten zwischen Bildern in großen Datenmengen quer über visuelle Domänen hinweg und identifiziert verschiedene Bedeutungen für mehrdeutige Wörter durch die Erforschung von Ähnlichkeiten in Online-Suchergebnissen. Des Weiteren wird die höchst subjektive ästhetische Anziehungskraft von Bildern untersucht und "deep learning" genutzt, um direkt ästhetische Einordnungen aus einer breiten Vielfalt von Benutzerreaktionen im sozialen Online-Verhalten zu lernen. Um noch tiefere Erkenntnisse über den Einfluss des visuellen Erscheinungsbildes auf einen Betrachter zu gewinnen, wird erforscht, wie alleinig einfache Bildverarbeitung in der Lage ist, tatsächlich die emotionale Wahrnehmung zu verändern und ein einfacher aber wirkungsvoller Bildfilter davon abgeleitet werden kann. Um bedeutungserhaltende Verbindungen zwischen geschriebenem Text und visueller Darstellung zu ermitteln, werden Methoden des "Natural Language Processing (NLP)" verwendet, die der Verarbeitung natürlicher Sprache dienen. Der Einsatz umfangreicher Textverarbeitung ermöglicht es, semantisch relevante Illustrationen für einfache Textteile sowie für komplette Handlungsstränge zu erzeugen. Im Detail wird ein Ansatz vorgestellt, der Abhängigkeiten in Textbeschreibungen auflöst, um 3D-Modelle korrekt anzuordnen. Des Weiteren wird eine Methode entwickelt die, basierend auf einem neuen hierarchischen Such-Anfrage Algorithmus, semantisch relevante Illustrationen zu Texten verschiedener Art findet. Schließlich wird ein optimierungsbasiertes Framework vorgestellt, das nicht nur semantisch relevante, sondern auch visuell kohärente Bildgeschichten in verschiedenen Bildstilen erzeugen kann

Annual Report: 2008

I submit herewith the annual report from the Agricultural and Forestry Experiment Station, School of Natural Resources and Agricultural Sciences, University of Alaska Fairbanks, for the period ending December 31, 2008. This is done in accordance with an act of Congress, approved March 2, 1887, entitled, “An act to establish agricultural experiment stations, in connection with the agricultural college established in the several states under the provisions of an act approved July 2, 1862, and under the acts supplementary thereto,” and also of the act of the Alaska Territorial Legislature, approved March 12, 1935, accepting the provisions of the act of Congress. The research reports are organized according to our strategic plan, which focuses on high-latitude soils, high-latitude agriculture, natural resources use and allocation, ecosystems management, and geographic information. These areas cross department and unit lines, linking them and unifying the research. We have also included in our financial statement information on the special grants we receive. These special grants allow us to provide research and outreach that is targeted toward economic development in Alaska. Research conducted by our graduate and undergraduate students plays an important role in these grants and the impact they make on Alaska.Financial statement -- Grants -- Students -- Research reports: Partners, Facilities, and Programs; Geographic Information; High-Latitude Agriculture; High-Latitude Soils, Management of Ecosystems; Natural Resources Use and Allocation; Index to Reports -- Publications -- Facult

Quarterly literature review of the remote sensing of natural resources

The Technology Application Center reviewed abstracted literature sources, and selected document data and data gathering techniques which were performed or obtained remotely from space, aircraft or groundbased stations. All of the documentation was related to remote sensing sensors or the remote sensing of the natural resources. Sensors were primarily those operating within the 10 to the minus 8 power to 1 meter wavelength band. Included are NASA Tech Briefs, ARAC Industrial Applications Reports, U.S. Navy Technical Reports, U.S. Patent reports, and other technical articles and reports

Annual Report: 2009

I submit herewith the annual report from the Agricultural and Forestry Experiment Station, School of Natural Resources and Agricultural Sciences, University of Alaska Fairbanks, for the period ending December 31, 2009. This is done in accordance with an act of Congress, approved March 2, 1887, entitled, “An act to establish agricultural experiment stations, in connection with the agricultural college established in the several states under the provisions of an act approved July 2, 1862, and under the acts supplementary thereto,” and also of the act of the Alaska Territorial Legislature, approved March 12, 1935, accepting the provisions of the act of Congress. The research reports are organized according to our strategic plan, which focuses on high-latitude soils, high-latitude agriculture, natural resources use and allocation, ecosystems management, and geographic information. These areas cross department and unit lines, linking them and unifying the research. We have also included in our financial statement information on the special grants we receive. These special grants allow us to provide research and outreach that is targeted toward economic development in Alaska. Research conducted by our graduate and undergraduate students plays an important role in these grants and the impact they make on Alaska.Financial statement -- Grants -- Students -- Research Reports: Partners, Facilities, and Programs; Geography; High-Latitude Agriculture; High-Latitude Soils; Management of Ecosystems; Natural Resources Use and Allocation; Index to Reports -- Publications -- Facult

Human experience in the natural and built environment : implications for research policy and practice

22nd IAPS conference. Edited book of abstracts. 427 pp. University of Strathclyde, Sheffield and West of Scotland Publication. ISBN: 978-0-94-764988-3

Digitizing Material Culture

Archaeology has always centred on material studies and has a long tradition of establishing methods for addressing the vast source material at hand. Typology is one of the most prominent of these methods, where the material is categorised into types according to morphology and geometry. There is an ongoing debate regarding the subjectivity of the typological approach and the problems that follow this issue. In recent years, there has also been an increase in the use of digital methods in archaeology to tackle many of the problems present in previous archaeological work, but this has mostly focused on field and buildings archaeology. This thesis therefore aimed at testing and discussing the application of digital methods of documentation and analysis within the field of material studies. It was investigated how a digital approach could aid and solve some of the current issues of the field of study, but also how it can further improve the science. This was done by establishing a digital work-flow, conducting analyses and discussing the theoretical and methodological aspects of the digital approach. It was concluded that there are several advantages to be gained from using the digital method, especially in detail-oriented studies and surface analyses, and that it can also be employed to greatly improve the typological method in regards to the debate of subjectivity, but that the method must be used in a proper way for this to be achieved. It can also aid in the spread of knowledge and documentation within the archaeological discipline, as well as providing the tools for deeper analysis and understanding into the material culture

Human-forest relationship in Finland

Our relationship with the forest can be defined as human-forest relationship (HFR). It is the result of our individual history, family history, cultural background, the society in which we live, and the forest surrounding us. This relationship, which combines both historical and modern values and practices, reflects the constantly evolving global, national, communal and individual attitudes towards forests. The aim of this article is to first, define the concept of HFR and second, to demonstrate how HFR has been, and continues to be integrated into Finnish society and culture. Finally, we will gather some ongoing societal discussion on changes in HFR. The Finnish National Inventory of Living Heritage, established in 2017 included HFR as one of its elements. In March 2018, according to a published survey, 83 per cent of Finns appraised forests either quite or very important for themselves. These results beckoned the question of what is their HFR. Do ageing private forest owners share similar HFR with city dwellers of generation Y? Despite the admitted importance of forests, it seems that the essence of the HFR is evolving and Finns are adapting various HFRs in accordance with the fading traditional economic importance of forests as new values are arising.Peer reviewe

A Low-cost Normalized Difference Vegetation Index (NDVI) Payload for Cubesats and Unmanned Aerial Vehicles (UAVs)

The focus of this research has been the design and fabrication of a Normalized Difference Vegetation Index (NDVI) payload configuration. This unique payload employs low-cost commercial off-the-shelf (COTS) hardware and equipment to assess photosynthetic activity and vegetation health through remote sensing on Cubesat or Unmanned Aerial Vehicle (UAV) platforms. The proposed NDVI imaging payload is comprised of three main subsystems: an electrical system, a software system, and a hardware system. The electrical system includes a custom designed printed circuit board (PCB), a single cell 3.7 V lithium-polymer battery, voltage regulator circuitry components, and wiring harnesses and connectors. The software system employs a master and slave system that communicates through general purpose input/output (GPIO) pin responses. Raspberry Pi Zero computer boards serve as the central processing units (CPUs) of the hardware subsystem, which also consists of the Pi-Cam standard red/green/blue (RGB) and Pi No-IR near-infrared (NIR) camera modules. A PCB was designed to be compatible with the Cubesat standard and lightweight component selections make it a desirable option for UAV flights. Open-source GIMP image processing software was used to analyze results from ground-based testing and flight testing on a UAV and general aviation (GA) aircraft at various altitudes to validate proof of concept. Raw NDVI and NDVI color map images were created from GIMP post-processing. Analysis of the results suggests that the angle of incidence of the sun with respect to the view angle of the imaging payload is a significant factor in the resulting NDVI values. Terrain also appeared to have an effect on the results where shadows were cast from the sun at low angles of incidence. Therefore, in the northern hemisphere it is recommended that image collection is performed roughly within the hours of 10 AM and 2 PM between the vernal and autumnal equinoxes to ensure a solar altitude of at least 35°. For best results, it has been verified that image data should be collected at the local time of maximum solar altitude for a particular date and location of interest (typically around noon). The information gather by this research can be used by scientists and technologist to potentially provide a means of enhancing their research and further developing technologies of UAV applications and space-based systems

Assessment of the CORONA series of satellite imagery for landscape archaeology: a case study from the Orontes valley, Syria

In 1995, a large database of satellite imagery with worldwide coverage taken from 1960 until 1972 was declassified. The main advantages of this imagery known as CORONA that made it attractive for archaeology were its moderate cost and its historical value. The main disadvantages were its unknown quality, format, geometry and the limited base of known applications. This thesis has sought to explore the properties and potential of CORONA imagery and thus enhance its value for applications in landscape archaeology. In order to ground these investigations in a real dataset, the properties and characteristics of CORONA imagery were explored through the case study of a landscape archaeology project working in the Orontes Valley, Syria. Present-day high-resolution IKONOS imagery was integrated within the study and assessed alongside CORONA imagery. The combination of these two image datasets was shown to provide a powerful set of tools for investigating past archaeological landscape in the Middle East. The imagery was assessed qualitatively through photointerpretation for its ability to detect archaeological remains, and quantitatively through the extraction of height information after the creation of stereomodels. The imagery was also assessed spectrally through fieldwork and spectroradiometric analysis, and for its Multiple View Angle (MVA) capability through visual and statistical analysis. Landscape archaeology requires a variety of data to be gathered from a large area, in an effective and inexpensive way. This study demonstrates an effective methodology for the deployment of CORONA and IKONOS imagery and raises a number of technical points of which the archaeological researcher community need to be aware of. Simultaneously, it identified certain limitations of the data and suggested solutions for the more effective exploitation of the strengths of CORONA imagery