Multimedia Forensics

This book is open access. Media forensics has never been more relevant to societal life. Not only media content represents an ever-increasing share of the data traveling on the net and the preferred communications means for most users, it has also become integral part of most innovative applications in the digital information ecosystem that serves various sectors of society, from the entertainment, to journalism, to politics. Undoubtedly, the advances in deep learning and computational imaging contributed significantly to this outcome. The underlying technologies that drive this trend, however, also pose a profound challenge in establishing trust in what we see, hear, and read, and make media content the preferred target of malicious attacks. In this new threat landscape powered by innovative imaging technologies and sophisticated tools, based on autoencoders and generative adversarial networks, this book fills an important gap. It presents a comprehensive review of state-of-the-art forensics capabilities that relate to media attribution, integrity and authenticity verification, and counter forensics. Its content is developed to provide practitioners, researchers, photo and video enthusiasts, and students a holistic view of the field

The role of stigma in writing charitable appeals

Indiana University-Purdue University Indianapolis (IUPUI)This study investigated choices made by fundraisers when crafting appeals to unknown potential donors. Specifically, it asked if and how fundraisers’ choices vary depending on whether they were raising money for a population that faced societal stigma. Research on fundraising often focuses on donor behavior, without considering the type of the beneficiary and the discretionary decisions made by fundraisers. This study drew on literature about stigma and literature about fundraising communication. It employed mixed methodologies to explore this research question. The first part of the study used an online experimental survey, in which 76 practicing fundraisers wrote an acquisition appeal letter for a nonprofit after random assignment to benefit either clients with mental illness (stigmatized population) or older adults (non-stigmatized population), then answered attitudinal questions about the beneficiary population. Participants believed individuals with mental illness were more stigmatized than older adults. Analysis of the letters using linguistic software showed that fundraisers used more humanizing language when writing about the non-stigmatized population, compared to the stigmatized population. Several aspects of the appeal letters, identified through existing theory, were examined but did not vary at statistically significant levels between the groups. Exploratory factor analysis showed several patterns of elements recurring within the letters. One of these patterns, addressing social expectations, varied significantly by client group. In the second part of the study, semi-structured interviews with fifteen participants showed that writing for the stigmatized client population raised special concerns in communicating with potential donors: many interviewees described identifying client stories and evidence to justify helping stigmatized clients in a way that wasn’t thought as necessary for non-stigmatized clients. They also attempted to mitigate threatening stereotypes to maintain readers’ comfort levels. Fundraisers regularly evaluated how readers were likely to think of different kinds of clients. Fundraisers’ own implicit assumptions also came into play

Assessing the sustainability of indigenous food systems in Pacific Small Island Developing States (PSIDS) : a dissertation presented in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Public Health Nutrition & Food Systems at Massey University, Wellington, New Zealand

Chapter 2 is reproduced with the publisher's permission. This article was published in Vogliano, C., Murray, L., Coad, J., Wham, C., Maelaua, J., Kafa, R., & Burlingame, B., Progress towards SDG 2: Zero hunger in Melanesia – A state of data scoping review, Global Food Security, 29, 100519, © Elsevier 2021. Chapter 3 is reproduced with permission. This article was published as Chapter 4, From the ocean to the mountains: Storytelling in the Pacific Islands, in FAO and Alliance of Bioversity International and CIAT, Indigenous Peoples’ food systems: Insights on sustainability and resilience from the front line of climate change, Rome, 2021, http://www.fao.org/documents/card/en/c/cb5131en. Chapters 4 & 5 are re-used under the terms of the Creative Commons Attribution 4.0 International (CC BY 4.0) license, https://creativecommons.org/licenses/by/4.0/. Appendices A & H are re-used under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs 3.0 IGO (CC BY-NC-ND 3.0 IGO) license, https://creativecommons.org/licenses/by-nc-nd/3.0/igo/. Appendix B was removed for copyright reasons. Appendix C is re-used under the terms of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).Indigenous Peoples living in Pacific Small Island Developing States (PSIDS) who have traditionally relied on locally grown, biodiverse foods for their primary source of nutrition are now seeing the adverse impacts of changing diets and climate change. Shifts away from traditional diets towards modern, imported and ultra-processed foods are likely giving rise to noncommunicable diseases such as cardiovascular disease and Type 2 Diabetes Mellitus, which are now the leading causes of mortality. Climate change is magnifying health inequities and challenging food and nutrition security through heavier rains, longer droughts, and rising sea levels. COVID-19 has highlighted additional challenges for those living in PSIDS, exposing vulnerabilities across global food systems. Using Solomon Islands as a proxy for the broader Pacific, this thesis aims to assess PSIDS food system sustainability, including diet quality and diversity, as well as perceived food system transitions. Findings from this thesis can help strengthen discourse around promoting sustainable and resilient food systems and help achieve food and nutrition security targets set by the United Nations Sustainable Development Goals (SDGs)

Landscape Visualization: Influence on Engagement for Climate Resilience

Research suggests an “Adaptation Deficit” exists in the realm of climate change mitigation and adaptation. There is a lack of climate adaptation goals, policies and projects implemented at the local level. Climate resilience relies on effective public engagement to ensure implementation. This type of engagement includes: (1) being aware of the issue and solutions; (2) feeling concerned about the problem; and (3) taking action. This research explores the impact of in situ 3D landscape visualization coupled with meaningful dialogue, on public engagement for climate change resilience. A mixed methods approach was used to undertake this research study using landscape visualization in an experiential outdoor setting in San Mateo County, California. San Mateo County was chosen as an optimal site for this research because of efforts underway to plan and prepare for sea level rise across the region. Since the research was part of a larger project with numerous stakeholders, many characteristics of Action Research (AR) were incorporated into the research design. This included working with local, regional, state and federal stakeholders to choose the exact site location, target audience, and project objectives to be accomplished from the research study. The overall goal of the project was to increase community concern about sea level rise and prompt target audience members to take an active role in their community on climate change adaptation. The research component of the project tested the use of landscape visualization to gauge impacts on concern and engagement levels, along with correlations between age, concern and engagement. The landscape visualization process used 3D imagery loaded into two viewfinders, called OWLS, that depicted current and future sea-level rise scenarios along with two possible solutions for Coyote Beach recreational area. Findings indicate that landscape visualization increases concern levels in participants that harbor low to no concern about existing sea-level rise, high tides, and storms. There was a statistically significant relationship between high concern levels and higher levels of engagement on the issue of climate adaptation. Lastly, data were collected to understand barriers to climate change engagement and adaptation and consider solutions that could overcome specific barriers identified. Using visual imagery along with meaningful dialogue allowed for a deep exploration of these barriers and solutions to be explored. Further research is needed to further test the application of landscape visualization along with meaningful dialogue on the issue of climate change in other locations, and to explore applicability in different settings and with different audiences

Convolutional Neural Networks for Image Steganalysis in the Spatial Domain

Esta tesis doctoral muestra los resultados obtenidos al aplicar Redes Neuronales Convolucionales (CNNs) para el estegoanálisis de imágenes digitales en el dominio espacial. La esteganografía consiste en ocultar mensajes dentro de un objeto conocido como portador para establecer un canal de comunicación encubierto para que el acto de comunicación pase desapercibido para los observadores que tienen acceso a ese canal. Steganalysis se dedica a detectar mensajes ocultos mediante esteganografía; estos mensajes pueden estar implícitos en diferentes tipos de medios, como imágenes digitales, archivos de video, archivos de audio o texto sin formato. Desde 2014, los investigadores se han interesado especialmente en aplicar técnicas de Deep Learning (DL) para lograr resultados que superen los métodos tradicionales de Machine Learning (ML).Is doctoral thesis shows the results obtained by applying Convolutional Neural Networks (CNNs) for the steganalysis of digital images in the spatial domain. Steganography consists of hiding messages inside an object known as a carrier to establish a covert communication channel so that the act of communication goes unnoticed by observers who have access to that channel. Steganalysis is dedicated to detecting hidden messages using steganography; these messages can be implicit in di.erent types of media, such as digital images, video €les, audio €les, or plain text. Since 2014 researchers have taken a particular interest in applying Deep Learning (DL) techniques to achieving results that surpass traditional Machine Learning (ML) methods

Multimedia Forensics

This book is open access. Media forensics has never been more relevant to societal life. Not only media content represents an ever-increasing share of the data traveling on the net and the preferred communications means for most users, it has also become integral part of most innovative applications in the digital information ecosystem that serves various sectors of society, from the entertainment, to journalism, to politics. Undoubtedly, the advances in deep learning and computational imaging contributed significantly to this outcome. The underlying technologies that drive this trend, however, also pose a profound challenge in establishing trust in what we see, hear, and read, and make media content the preferred target of malicious attacks. In this new threat landscape powered by innovative imaging technologies and sophisticated tools, based on autoencoders and generative adversarial networks, this book fills an important gap. It presents a comprehensive review of state-of-the-art forensics capabilities that relate to media attribution, integrity and authenticity verification, and counter forensics. Its content is developed to provide practitioners, researchers, photo and video enthusiasts, and students a holistic view of the field

The roles of extracellular matrix molecules matrilins and aggrecan in bone development and articular cartilage functions

Bones are important constituents of the organ systems of the vertebrates providing body support, physical protection for inner organs, movement facilitation, mineral storage and a niche for hematopoiesis [1, 2]. In human, there are more than 200 bones, which derive through two developmental pathways: 1) flat bones of the skull form directly from the condensation of the skeletogenic mesenchymal cells in the process of intramembranous ossification (IO); 2) while long bones of the appendicular and axial skeleton arise through a cartilaginous intermediates in the process of endochondral ossification (EO) [3]. EO starts with the condensation of the skeletogenic mesenchymal cells at the sites of the future bones, and the progenitor cells in these aggregates, under the guidance of various factors, differentiate into chondrocytes forming the cartilage template. Chondrocytes within the cartilage anlage begin to proliferate and synthesize cartilage-specific extracellular matrix, which is rich in collagen II and aggrecan [3]. The cartilage templates subsequently undergo maturation, hypertrophy, vascular invasion and mineralization, and the bones grow both laterally and longitudinally [4, 5]. As a result of the morphogenetic processes, embryonic cartilage is largely replaced by bone (transient cartilage), except the ends of long bones where it remains intact and forms the permanent articular cartilage [6, 7]. Articular cartilage (AC) is a highly hydrated, strong, resilient, avascular, alymphatic and aneural tissue. Covering the ends of long bones, AC is not only providing a lubricating, frictionless surface for the synovial, diarthrodial joints, but is also essential to distribute the mechanical loading generated during movement [8]. AC is composed of a relatively small number of chondrocytes, which lay down a specialized extracellular matrix (ECM). The major constituents of the ECM are the organic components (about 40%) and water (about 60%) [9, 10]. AC is characterized by a unique zonal structure with varying structural and biochemical properties. Generally, the AC is divided into four vertical layers: the superficial zone, the middle zone, the deep zone and the calcified cartilage zones[11]. All these zones have characteristic mechanical behavior for loading stimuli [12]. The ECM of both the transient and permanent cartilage provides physical support for chondrocytes, and acts as a sponge reserving growth factors and other cytokines, which in turn could modulate cell proliferation and differentiation. The ECM is predominantly composed of fibrillary collagens, proteoglycans, and non-collagenous molecules [6, 13, 14]. These constituents interact with each other forming a unique protein network, which maintain the biochemical and biomechanical characters of the cartilage. The collagen fibrils provide tensile strength, whereas proteoglycans are responsible for the osmotic swelling and the elastic properties of the tissue [13, 15]. The cartilage ECM is a dynamic network, which undergoes modeling and remodeling along the whole life. Homeostasis of cartilage is maintained by complex mechanisms controlling the turnover of the ECM by regulating the balance between anabolic and catabolic processes [16]. Mutations in matrix proteins resulting in abnormal organization of the ECM could eventually affect the development of endochondral bones and the function of articular cartilage [17, 18]. Abnormal development and growth of the transient cartilage lead to various chondrodysplasias; while degenerative diseases, such as osteoarthritis, are characteristic for the permanent AC [19]

A Data-driven Methodology Towards Mobility- and Traffic-related Big Spatiotemporal Data Frameworks

Human population is increasing at unprecedented rates, particularly in urban areas. This increase, along with the rise of a more economically empowered middle class, brings new and complex challenges to the mobility of people within urban areas. To tackle such challenges, transportation and mobility authorities and operators are trying to adopt innovative Big Data-driven Mobility- and Traffic-related solutions. Such solutions will help decision-making processes that aim to ease the load on an already overloaded transport infrastructure. The information collected from day-to-day mobility and traffic can help to mitigate some of such mobility challenges in urban areas. Road infrastructure and traffic management operators (RITMOs) face several limitations to effectively extract value from the exponentially growing volumes of mobility- and traffic-related Big Spatiotemporal Data (MobiTrafficBD) that are being acquired and gathered. Research about the topics of Big Data, Spatiotemporal Data and specially MobiTrafficBD is scattered, and existing literature does not offer a concrete, common methodological approach to setup, configure, deploy and use a complete Big Data-based framework to manage the lifecycle of mobility-related spatiotemporal data, mainly focused on geo-referenced time series (GRTS) and spatiotemporal events (ST Events), extract value from it and support decision-making processes of RITMOs. This doctoral thesis proposes a data-driven, prescriptive methodological approach towards the design, development and deployment of MobiTrafficBD Frameworks focused on GRTS and ST Events. Besides a thorough literature review on Spatiotemporal Data, Big Data and the merging of these two fields through MobiTraffiBD, the methodological approach comprises a set of general characteristics, technical requirements, logical components, data flows and technological infrastructure models, as well as guidelines and best practices that aim to guide researchers, practitioners and stakeholders, such as RITMOs, throughout the design, development and deployment phases of any MobiTrafficBD Framework. This work is intended to be a supporting methodological guide, based on widely used Reference Architectures and guidelines for Big Data, but enriched with inherent characteristics and concerns brought about by Big Spatiotemporal Data, such as in the case of GRTS and ST Events. The proposed methodology was evaluated and demonstrated in various real-world use cases that deployed MobiTrafficBD-based Data Management, Processing, Analytics and Visualisation methods, tools and technologies, under the umbrella of several research projects funded by the European Commission and the Portuguese Government.A população humana cresce a um ritmo sem precedentes, particularmente nas áreas urbanas. Este aumento, aliado ao robustecimento de uma classe média com maior poder económico, introduzem novos e complexos desafios na mobilidade de pessoas em áreas urbanas. Para abordar estes desafios, autoridades e operadores de transportes e mobilidade estão a adotar soluções inovadoras no domínio dos sistemas de Dados em Larga Escala nos domínios da Mobilidade e Tráfego. Estas soluções irão apoiar os processos de decisão com o intuito de libertar uma infraestrutura de estradas e transportes já sobrecarregada. A informação colecionada da mobilidade diária e da utilização da infraestrutura de estradas pode ajudar na mitigação de alguns dos desafios da mobilidade urbana. Os operadores de gestão de trânsito e de infraestruturas de estradas (em inglês, road infrastructure and traffic management operators — RITMOs) estão limitados no que toca a extrair valor de um sempre crescente volume de Dados Espaciotemporais em Larga Escala no domínio da Mobilidade e Tráfego (em inglês, Mobility- and Traffic-related Big Spatiotemporal Data —MobiTrafficBD) que estão a ser colecionados e recolhidos. Os trabalhos de investigação sobre os tópicos de Big Data, Dados Espaciotemporais e, especialmente, de MobiTrafficBD, estão dispersos, e a literatura existente não oferece uma metodologia comum e concreta para preparar, configurar, implementar e usar uma plataforma (framework) baseada em tecnologias Big Data para gerir o ciclo de vida de dados espaciotemporais em larga escala, com ênfase nas série temporais georreferenciadas (em inglês, geo-referenced time series — GRTS) e eventos espacio- temporais (em inglês, spatiotemporal events — ST Events), extrair valor destes dados e apoiar os RITMOs nos seus processos de decisão. Esta dissertação doutoral propõe uma metodologia prescritiva orientada a dados, para o design, desenvolvimento e implementação de plataformas de MobiTrafficBD, focadas em GRTS e ST Events. Além de uma revisão de literatura completa nas áreas de Dados Espaciotemporais, Big Data e na junção destas áreas através do conceito de MobiTrafficBD, a metodologia proposta contem um conjunto de características gerais, requisitos técnicos, componentes lógicos, fluxos de dados e modelos de infraestrutura tecnológica, bem como diretrizes e boas práticas para investigadores, profissionais e outras partes interessadas, como RITMOs, com o objetivo de guiá-los pelas fases de design, desenvolvimento e implementação de qualquer pla- taforma MobiTrafficBD. Este trabalho deve ser visto como um guia metodológico de suporte, baseado em Arqui- teturas de Referência e diretrizes amplamente utilizadas, mas enriquecido com as característi- cas e assuntos implícitos relacionados com Dados Espaciotemporais em Larga Escala, como no caso de GRTS e ST Events. A metodologia proposta foi avaliada e demonstrada em vários cenários reais no âmbito de projetos de investigação financiados pela Comissão Europeia e pelo Governo português, nos quais foram implementados métodos, ferramentas e tecnologias nas áreas de Gestão de Dados, Processamento de Dados e Ciência e Visualização de Dados em plataformas MobiTrafficB

Unsupervised quantification of entity consistency between photos and text in real-world news

Das World Wide Web und die sozialen Medien übernehmen im heutigen Informationszeitalter eine wichtige Rolle für die Vermittlung von Nachrichten und Informationen. In der Regel werden verschiedene Modalitäten im Sinne der Informationskodierung wie beispielsweise Fotos und Text verwendet, um Nachrichten effektiver zu vermitteln oder Aufmerksamkeit zu erregen. Kommunikations- und Sprachwissenschaftler erforschen das komplexe Zusammenspiel zwischen Modalitäten seit Jahrzehnten und haben unter Anderem untersucht, wie durch die Kombination der Modalitäten zusätzliche Informationen oder eine neue Bedeutungsebene entstehen können. Die Anzahl gemeinsamer Konzepte oder Entitäten (beispielsweise Personen, Orte und Ereignisse) zwischen Fotos und Text stellen einen wichtigen Aspekt für die Bewertung der Gesamtaussage und Bedeutung eines multimodalen Artikels dar. Automatisierte Ansätze zur Quantifizierung von Bild-Text-Beziehungen können für zahlreiche Anwendungen eingesetzt werden. Sie ermöglichen beispielsweise eine effiziente Exploration von Nachrichten, erleichtern die semantische Suche von Multimedia-Inhalten in (Web)-Archiven oder unterstützen menschliche Analysten bei der Evaluierung der Glaubwürdigkeit von Nachrichten. Allerdings gibt es bislang nur wenige Ansätze, die sich mit der Quantifizierung von Beziehungen zwischen Fotos und Text beschäftigen. Diese Ansätze berücksichtigen jedoch nicht explizit die intermodalen Beziehungen von Entitäten, welche eine wichtige Rolle in Nachrichten darstellen, oder basieren auf überwachten multimodalen Deep-Learning-Techniken. Diese überwachten Lernverfahren können ausschließlich die intermodalen Beziehungen von Entitäten detektieren, die in annotierten Trainingsdaten enthalten sind. Um diese Forschungslücke zu schließen, wird in dieser Arbeit ein unüberwachter Ansatz zur Quantifizierung der intermodalen Konsistenz von Entitäten zwischen Fotos und Text in realen multimodalen Nachrichtenartikeln vorgestellt. Im ersten Teil dieser Arbeit werden neuartige Verfahren auf Basis von Deep Learning zur Extrahierung von Informationen aus Fotos vorgestellt, um Ereignisse (Events), Orte, Zeitangaben und Personen automatisch zu erkennen. Diese Verfahren bilden eine wichtige Voraussetzung, um die Beziehungen von Entitäten zwischen Bild und Text zu bewerten. Zunächst wird ein Ansatz zur Ereignisklassifizierung präsentiert, der neuartige Optimierungsfunktionen und Gewichtungsschemata nutzt um Ontologie-Informationen aus einer Wissensdatenbank in ein Deep-Learning-Verfahren zu integrieren. Das Training erfolgt anhand eines neu vorgestellten Datensatzes, der 570.540 Fotos und eine Ontologie mit 148 Ereignistypen enthält. Der Ansatz übertrifft die Ergebnisse von Referenzsystemen die keine strukturierten Ontologie-Informationen verwenden. Weiterhin wird ein DeepLearning-Ansatz zur Schätzung des Aufnahmeortes von Fotos vorgeschlagen, der Kontextinformationen über die Umgebung (Innen-, Stadt-, oder Naturaufnahme) und von Erdpartitionen unterschiedlicher Granularität verwendet. Die vorgeschlagene Lösung übertrifft die bisher besten Ergebnisse von aktuellen Forschungsarbeiten, obwohl diese deutlich mehr Fotos zum Training verwenden. Darüber hinaus stellen wir den ersten Datensatz zur Schätzung des Aufnahmejahres von Fotos vor, der mehr als eine Million Bilder aus den Jahren 1930 bis 1999 umfasst. Dieser Datensatz wird für das Training von zwei Deep-Learning-Ansätzen zur Schätzung des Aufnahmejahres verwendet, welche die Aufgabe als Klassifizierungs- und Regressionsproblem behandeln. Beide Ansätze erzielen sehr gute Ergebnisse und übertreffen Annotationen von menschlichen Probanden. Schließlich wird ein neuartiger Ansatz zur Identifizierung von Personen des öffentlichen Lebens und ihres gemeinsamen Auftretens in Nachrichtenfotos aus der digitalen Bibliothek Internet Archiv präsentiert. Der Ansatz ermöglicht es unstrukturierte Webdaten aus dem Internet Archiv mit Metadaten, beispielsweise zur semantischen Suche, zu erweitern. Experimentelle Ergebnisse haben die Effektivität des zugrundeliegenden Deep-Learning-Ansatzes zur Personenerkennung bestätigt. Im zweiten Teil dieser Arbeit wird ein unüberwachtes System zur Quantifizierung von BildText-Beziehungen in realen Nachrichten vorgestellt. Im Gegensatz zu bisherigen Verfahren liefert es automatisch neuartige Maße der intermodalen Konsistenz für verschiedene Entitätstypen (Personen, Orte und Ereignisse) sowie den Gesamtkontext. Das System ist nicht auf vordefinierte Datensätze angewiesen, und kann daher mit der Vielzahl und Diversität von Entitäten und Themen in Nachrichten umgehen. Zur Extrahierung von Entitäten aus dem Text werden geeignete Methoden der natürlichen Sprachverarbeitung eingesetzt. Examplarbilder für diese Entitäten werden automatisch aus dem Internet beschafft. Die vorgeschlagenen Methoden zur Informationsextraktion aus Fotos werden auf die Nachrichten- und heruntergeladenen Exemplarbilder angewendet, um die intermodale Konsistenz von Entitäten zu quantifizieren. Es werden zwei Aufgaben untersucht um die Qualität des vorgeschlagenen Ansatzes in realen Anwendungen zu bewerten. Experimentelle Ergebnisse für die Dokumentverifikation und die Beschaffung von Nachrichten mit geringer (potenzielle Fehlinformation) oder hoher multimodalen Konsistenz zeigen den Nutzen und das Potenzial des Ansatzes zur Unterstützung menschlicher Analysten bei der Untersuchung von Nachrichten.In today’s information age, the World Wide Web and social media are important sources for news and information. Different modalities (in the sense of information encoding) such as photos and text are typically used to communicate news more effectively or to attract attention. Communication scientists, linguists, and semioticians have studied the complex interplay between modalities for decades and investigated, e.g., how their combination can carry additional information or add a new level of meaning. The number of shared concepts or entities (e.g., persons, locations, and events) between photos and text is an important aspect to evaluate the overall message and meaning of an article. Computational models for the quantification of image-text relations can enable many applications. For example, they allow for more efficient exploration of news, facilitate semantic search and multimedia retrieval in large (web) archives, or assist human assessors in evaluating news for credibility. To date, only a few approaches have been suggested that quantify relations between photos and text. However, they either do not explicitly consider the cross-modal relations of entities – which are important in the news – or rely on supervised deep learning approaches that can only detect the cross-modal presence of entities covered in the labeled training data. To address this research gap, this thesis proposes an unsupervised approach that can quantify entity consistency between photos and text in multimodal real-world news articles. The first part of this thesis presents novel approaches based on deep learning for information extraction from photos to recognize events, locations, dates, and persons. These approaches are an important prerequisite to measure the cross-modal presence of entities in text and photos. First, an ontology-driven event classification approach that leverages new loss functions and weighting schemes is presented. It is trained on a novel dataset of 570,540 photos and an ontology with 148 event types. The proposed system outperforms approaches that do not use structured ontology information. Second, a novel deep learning approach for geolocation estimation is proposed that uses additional contextual information on the environmental setting (indoor, urban, natural) and from earth partitions of different granularity. The proposed solution outperforms state-of-the-art approaches, which are trained with significantly more photos. Third, we introduce the first large-scale dataset for date estimation with more than one million photos taken between 1930 and 1999, along with two deep learning approaches that treat date estimation as a classification and regression problem. Both approaches achieve very good results that are superior to human annotations. Finally, a novel approach is presented that identifies public persons and their co-occurrences in news photos extracted from the Internet Archive, which collects time-versioned snapshots of web pages that are rarely enriched with metadata relevant to multimedia retrieval. Experimental results confirm the effectiveness of the deep learning approach for person identification. The second part of this thesis introduces an unsupervised approach capable of quantifying image-text relations in real-world news. Unlike related work, the proposed solution automatically provides novel measures of cross-modal consistency for different entity types (persons, locations, and events) as well as the overall context. The approach does not rely on any predefined datasets to cope with the large amount and diversity of entities and topics covered in the news. State-of-the-art tools for natural language processing are applied to extract named entities from the text. Example photos for these entities are automatically crawled from the Web. The proposed methods for information extraction from photos are applied to both news images and example photos to quantify the cross-modal consistency of entities. Two tasks are introduced to assess the quality of the proposed approach in real-world applications. Experimental results for document verification and retrieval of news with either low (potential misinformation) or high cross-modal similarities demonstrate the feasibility of the approach and its potential to support human assessors to study news

Circuits and Systems Advances in Near Threshold Computing

Modern society is witnessing a sea change in ubiquitous computing, in which people have embraced computing systems as an indispensable part of day-to-day existence. Computation, storage, and communication abilities of smartphones, for example, have undergone monumental changes over the past decade. However, global emphasis on creating and sustaining green environments is leading to a rapid and ongoing proliferation of edge computing systems and applications. As a broad spectrum of healthcare, home, and transport applications shift to the edge of the network, near-threshold computing (NTC) is emerging as one of the promising low-power computing platforms. An NTC device sets its supply voltage close to its threshold voltage, dramatically reducing the energy consumption. Despite showing substantial promise in terms of energy efficiency, NTC is yet to see widescale commercial adoption. This is because circuits and systems operating with NTC suffer from several problems, including increased sensitivity to process variation, reliability problems, performance degradation, and security vulnerabilities, to name a few. To realize its potential, we need designs, techniques, and solutions to overcome these challenges associated with NTC circuits and systems. The readers of this book will be able to familiarize themselves with recent advances in electronics systems, focusing on near-threshold computing