Smartphone picture organization: a hierarchical approach

We live in a society where the large majority of the population has a camera-equipped smartphone. In addition, hard drives and cloud storage are getting cheaper and cheaper, leading to a tremendous growth in stored personal photos. Unlike photo collections captured by a digital camera, which typically are pre-processed by the user who organizes them into event-related folders, smartphone pictures are automatically stored in the cloud. As a consequence, photo collections captured by a smartphone are highly unstructured and because smartphones are ubiquitous, they present a larger variability compared to pictures captured by a digital camera. To solve the need of organizing large smartphone photo collections automatically, we propose here a new methodology for hierarchical photo organization into topics and topic-related categories. Our approach successfully estimates latent topics in the pictures by applying probabilistic Latent Semantic Analysis, and automatically assigns a name to each topic by relying on a lexical database. Topic-related categories are then estimated by using a set of topic-specific Convolutional Neuronal Networks. To validate our approach, we ensemble and make public a large dataset of more than 8,000 smartphone pictures from 40 persons. Experimental results demonstrate major user satisfaction with respect to state of the art solutions in terms of organization.Peer ReviewedPreprin

An e-science infrastructure for collecting, sharing, retrieving, and analyzing heterogeneous scientific data

The process of collecting, sharing, retrieving, and analyzing data is common in many areas of scientific work. While each field has its own workflows and best practices, the general process can be aided by an e-Science infrastructure. The contribution of this thesis is to support the workflow of the scientists which can be split in four parts: In the first part, we introduce xBook, a framework which aids the creation of database application to collect, back-up, and share data. In the second part, we describe the synchronization which is a vital part of the xBook framework that, with the use of timestamps, allows data to be entered offline. The data then can be shared with coworkers for analyses or further processing. It also can be used as a backup system to avoid data loss. Third, we present an architecture allowing data from distributed data sources to be retrieved without a central managing instance. This is achieved with the use of minimal search parameters which are guaranteed to exist in all connected data sources. This architecture is based on the concept of mediators, but gives data owners full control over their data sources as opposed to the traditional mediator where the connected data sources are managed by a central administrator. Fourth we describe an embeddable analysis tool which can be integrated into a base application where the data is gathered. With the aid of simple modules, called ``Workers'', this tool empowers domain experts to easily create analyses particularly designed for their area of work in a familiar working environment. Additionally, we present another tool which allows the graphical display of temporal and spatial information of archaeological excavations. This tool uses an interactive Harris Matrix to order findings temporally and allows the comparison with their spatial location.Viele wissenschaftliche Bereiche haben gemeinsame Vorgänge, wie die Erfassung, das Teilen, das Abrufen und das Analysieren von Daten. Jeder Bereich hat zwar seine eigenen Vorgänge und bewährte Verfahren, jedoch kann der allgemeine Prozess durch eine e-Science Infrastruktur unterstützt werden. Der Beitrag dieser Dissertation ist die Unterstützung des typischen wissenschaftlichen Arbeitsablaufes, der in vier Teile unterteilt werden kann: Im ersten Teil stellen wir xBook vor, ein Framework zur Erstellung von Datenbankanwendungen, das Wissenschaftler dabei unterstützt, Daten zu erfassen, zu sichern und zu teilen. Im zweiten Teil beschreiben wir die Synchronisation, die ein wichtiger Teil des xBook Frameworks ist. Diese erlaubt, dass Daten offline bearbeitet werden können, indem Änderungen über Zeitstempel protokolliert werden. Diese Daten können dann mit Kollegen für Analysen oder weitere Eingaben geteilt werden. Die Synchronisation kann zusätzlich als Sicherungssystem verwendet werden, um Datenverlust zu verhindern. Im dritten Teil präsentieren wir eine Architektur, um Daten aus verteilten Datenquellen ohne ein zentrales Verwaltungssystem abrufen zu können. Dies wird mit Hilfe eines minimalen Suchparameters, der in allen angeschlossenen Datenquellen existieren muss, ermöglicht. Diese Architektur basiert auf dem Konzept des Mediators, benötigt aber im Gegensatz zum traditionellen Mediator keinen zentralen Administrator zur Verwaltung der Datenquellen und gibt deren Besitzern volle Kontrolle über ihre Daten. Abschließend, im vierten Teil, beschreiben wir ein einbettbares Analyse Tool, das in eine Hauptanwendung integriert werden kann, in der Daten erfasst werden. Dieses Tool ermöglicht Fachexperten auf einfache Weise, mit Hilfe von speziellen Modulen, Analysen in einer vertrauten Arbeitsumgebung zu erstellen, die genau für ihr Fachgebiet benötigt werden. Zusätzlich stellen wir ein weiteres Tool vor, das die zeitlichen und räumlichen Informationen archäologischer Ausgrabungen visualisiert. Dieses Tool verwendet eine interaktive Harris Matrix, um Funde zeitlich zu ordnen und erlaubt den Vergleich ihrer räumlichen Position

Supporting the workflow of archaeo-related sciences by providing storage, sharing, analysis, and retrieval methods

The recovery and analysis of material culture is the main focus of archaeo-related work. The corpus of findings like rest of buildings, artifacts, human burial remains, or faunal remains is excavated, described, categorized, and analyzed in projects all over the world. A huge amount of archaeo-related data is the basis for many analyses. The results of analyzing collected data make us learn about the past. All disciplines of archaeo-related sciences deal with similar challenges. The workflow of the disciplines is similar, however there are still differences in the nature of the data. These circumstances result in questions how to store, share, retrieve, and analyze these heterogeneous and distributed data. The contribution of this thesis is to support archaeologists and bioarchaeologists in their work by providing methods following the archaeo-related workflow which is split in five main parts. Therefore, the first part of this thesis describes the xBook framework that has been developed to gather and store archaeological data. It allows creating several database applications to provide necessary features for the archaeo-related context. The second part deals with methods to share information, collaborate with colleagues, and retrieve distributed data of cohesive archaeological contexts to bring together archaeo-related data. The third part addresses a dynamic framework for data analyses which features a flexible and easy to be used tool to support archaeologists and bioarchaeologists executing analyses on their data without any programming skills and without the necessity to get familiar with external technologies. The fourth part introduces an interactive tool to compare the temporal position of archaeological findings in form of a Harris Matrix with their spatial position as 2D and 3D site plan sketches by using the introduced data retrieval methods. Finally, the fifth part specifies an architecture for an information system which allows distributed and interdisciplinary data to be searched by using dynamic joins of results from heterogeneous data formats. This novel way of information retrieval enables scientists to cross-connect archaeological information with domain-extrinsic knowledge. However, the concept of this information system is not limited to the archaeo-related context. Other sciences could also benefit from this architecture.Die Wiederherstellung und Analyse von materieller Kultur ist der Schwerpunkt archäologischer Arbeit. Das Material von Funden wie Gebäudereste, Artefakte, menschliche Überreste aus Bestattungen oder tierische Reste wird in Projekten auf der ganzen Welt ausgegraben, beschrieben, kategorisiert und analysiert. Die große Anzahl an archäologischen Daten bildet die Grundlage für viele Analysen. Die Ergebnisse der Auswertung der gesammelten Daten gibt uns Aufschluss über die Vergangenheit. Alle Disziplinen der archäologischen Wissenschaften setzen sich mit ähnlichen Herausforderungen auseinander. Der Arbeitsablauf ist in den einzelnen Disziplinen ähnlich, jedoch gibt es aufgrund der Art der Daten Unterschiede. Das führt zu Fragestellungen, wie heterogene und verteilte Daten erfasst, geteilt, abgerufen und analysiert werden können. Diese Dissertation beschäftigt sich mit der Unterstützung von Archäologen und Bioarchäologen bei ihrer Arbeit, indem unterstützende Methoden bereitgestellt werden, die dem archäologischen Arbeitsablauf , der in fünf Schritte unterteilt ist, folgt. Der erste Teil dieser Arbeit beschreibt das xBook Framework, welches entwickelt wurde, um archäologische Daten zu erfassen und zu speichern. Es ermöglicht die Erstellung zahlreicher Datenbankanwendungen, um notwendige Funktionen für den archäologischen Kontext bereitzustellen. Der zweite Teil beschäftigt sich mit der Zusammentragung von archäologischen Daten und setzt sich mit Methoden zum Teilen von Informationen, Methoden zur Zusammenarbeit zwischen Kollegen und Methoden zum Abruf von verteilten, aber zusammenhängenden archäologischen Daten auseinander. Der dritte Teil stellt ein dynamisches Framework für Datenanalysen vor, welches ein flexibles und leicht zu bedienendes Tool bereitstellt, das Archäologen und Bioarchäologen in der Ausführung von Analysen ihrer Daten unterstützt, so dass weder Programmierkenntnisse noch die Einarbeitung in externe Technologien benötigt werden. Der vierte Teil führt ein interaktives Tool ein, mit dem – unter Verwendung der zuvor beschriebenen Methoden zur Datenabfrage – die zeitliche Position von archäologischen Funden in Form einer Harris Matrix mit ihrer räumlichen Position als 2D- und 3D-Lageplan verglichen werden kann. Abschließend spezifiziert der fünfte Teil eine Architektur für ein Informationssystem, das die Durchsuchung von verteilten und interdisziplinären Daten durch dynamische Joins von Suchergebnissen aus heterogenen Datenformaten ermöglicht. Diese neue Art an Informationsabfrage erlaubt Wissenschaftlern eine Querverbindung von archäologischen Informationen mit fachfremdem Wissen. Das Konzept für dieses Informationssystem ist jedoch nicht auf den archäologischen Kontext begrenzt. Auch andere wissenschaftliche Bereiche können von dieser Architektur profitieren

Estimating long-term growth-rate changes of southern bluefin tuna (Thunnus maccoyii) from two periods of tag-return data

Southern bluefin tuna (SBT) (Thunnus maccoyii) growth rates are estimated from tag-return data associated with two time periods, the 1960s and 1980s. The traditional von Bertalanffy growth model (VBG) and a two-phase VBG model were fitted to the data by maximum likelihood. The traditional VBG model did not provide an adequate representation of growth in SBT, and the two-phase VBG yielded a significantly better fit. The results indicated that significant change occurs in the pattern of growth in relation to a VBG curve during the juvenile stages of the SBT life cycle, which may be related to the transition from a tightly schooling fish that spends substantial time in near and surface shore waters to one that is found primarily in more offshore and deeper waters. The results suggest that more complex growth models should be considered for other tunas and for other species that show a marked change in habitat use with age. The likelihood surface for the two-phase VBG model was found to be bimodal and some implications of this are investigated. Significant and substantial differences were found in the growth for fish spawned in the 1960s and in the 1980s, such that after age four there is a difference of about one year in the expected age of a fish of similar length which persists over the size range for which meaningful recapture data are available. This difference may be a density-dependent response as a consequence of the marked reduction in the SBT population. Given the key role that estimates of growth have in most stock assessments, the results indicate that there is a need both for the regular monitoring of growth rates and for provisions for changes in growth over time (possibly related to changes in abundance) in the stock assessment models used for SBT and other species

Node Cardinality Estimation in the Internet of Things Using Privileged Feature Distillation

The Internet of Things (IoT) is emerging as a critical technology to connect resource-constrained devices such as sensors and actuators as well as appliances to the Internet. In this paper, we propose a novel methodology for node cardinality estimation in wireless networks such as the IoT and Radio-Frequency IDentification (RFID) systems, which uses the privileged feature distillation (PFD) technique and works using a neural network with a teacher-student model. The teacher is trained using both privileged and regular features, and the student is trained with predictions from the teacher and regular features. We propose node cardinality estimation algorithms based on the PFD technique for homogeneous as well as heterogeneous wireless networks. We show via extensive simulations that the proposed PFD based algorithms for homogeneous as well as heterogeneous networks achieve much lower mean squared errors in the computed node cardinality estimates than state-of-the-art protocols proposed in prior work, while taking the same number of time slots for executing the node cardinality estimation process as the latter protocols.Comment: 15 pages, 17 figures, journal pape

Model-Agnostic Decentralized Collaborative Learning for On-Device POI Recommendation

As an indispensable personalized service in Location-based Social Networks (LBSNs), the next Point-of-Interest (POI) recommendation aims to help people discover attractive and interesting places. Currently, most POI recommenders are based on the conventional centralized paradigm that heavily relies on the cloud to train the recommendation models with large volumes of collected users' sensitive check-in data. Although a few recent works have explored on-device frameworks for resilient and privacy-preserving POI recommendations, they invariably hold the assumption of model homogeneity for parameters/gradients aggregation and collaboration. However, users' mobile devices in the real world have various hardware configurations (e.g., compute resources), leading to heterogeneous on-device models with different architectures and sizes. In light of this, We propose a novel on-device POI recommendation framework, namely Model-Agnostic Collaborative learning for on-device POI recommendation (MAC), allowing users to customize their own model structures (e.g., dimension \& number of hidden layers). To counteract the sparsity of on-device user data, we propose to pre-select neighbors for collaboration based on physical distances, category-level preferences, and social networks. To assimilate knowledge from the above-selected neighbors in an efficient and secure way, we adopt the knowledge distillation framework with mutual information maximization. Instead of sharing sensitive models/gradients, clients in MAC only share their soft decisions on a preloaded reference dataset. To filter out low-quality neighbors, we propose two sampling strategies, performance-triggered sampling and similarity-based sampling, to speed up the training process and obtain optimal recommenders. In addition, we design two novel approaches to generate more effective reference datasets while protecting users' privacy

Ontologies on the semantic web

As an informational technology, the World Wide Web has enjoyed spectacular success. In just ten years it has transformed the way information is produced, stored, and shared in arenas as diverse as shopping, family photo albums, and high-level academic research. The “Semantic Web” was touted by its developers as equally revolutionary but has not yet achieved anything like the Web’s exponential uptake. This 17 000 word survey article explores why this might be so, from a perspective that bridges both philosophy and IT