Towards solving computer vision problems: datasets, labels, algorithms, and applications

The solution to a supervised computer vision problem consists of an application, algorithm, input data, and a set of human generated labels. Solving these kinds of tasks involves collecting large quantities of data, collecting appropriate labels, and developing machine vision algorithms tailored to the application. Progress on these problems has often benefited from large scale datasets with high fidelity labels. Successful algorithms display a synergy between application goals and the size and quality of the dataset. This thesis presents work highlighting the importance of each component of a supervised vision task.First, the problem of automatically classifying groups of people into social categories is introduced. This problem is called Urban Tribe Classification. To tackle this problem, each individual and the entire group of individuals are modeled. Since this was a newly introduced computer vision problem, a dataset for this task was created. On this dataset, the combined representation of group and individuals outperforms using only the person representations. This model showed promising results for automatic subculture classification.Second, the problem of creating perceptual embeddings based on human similarity judgements is tackled. This work focuses on triplet similarity comparisons of the form ``Is object $i$ more similar to $j$ or $k$?'', which have been useful for computer vision and machine learning applications. Unfortunately, triplet similarity comparisons, like many human labeling efforts, can be prohibitively expensive. This work proposes two techniques for dealing with this obstacle. First, an alternative display for collecting triplets is designed. This display shows a probe image and a grid of query images, allowing the user to collect multiple triplets simultaneously. The display is shown to reduce the cost and time of triplet collection. In addition, higher quality embeddings are created with the improved triplet collection UI. A 10,000-food item dataset of human taste similarity was created using this UI. Second, ``SNaCK,'' a low-dimensional perceptual embedding algorithm that combines human expertise with automatic machine kernels, is introduced. Both parts are complementary: human insight can capture relationships that are not apparent from the object's visual similarity and the machine can help relieve the human from having to exhaustively specify many constraints. Finally, the precise localization of key frames of an action is explored. This work focuses on detecting the exact starting frame of a behavior, an important task for neuroscience research. To address this problem, a loss designed to penalize extra and missed action start detections over small misalignments. Recurrent neural networks (RNN) are trained to optimize this loss. The model is shown to reduce the number of false positives, an important criteria defined by the neuroscientist. The performance of the model is evaluated on a new dataset, the Mouse Reach Dataset, a large, annotated video dataset of mice performing a sequence of actions. The dataset was created for neuroscience research. On this dataset, the proposed model outperforms related approaches and baseline methods using an unstructured loss

Graph neural networks for network analysis

With an increasing number of applications where data can be represented as graphs, graph neural networks (GNNs) are a useful tool to apply deep learning to graph data. Signed and directed networks are important forms of networks that are linked to many real-world problems, such as ranking from pairwise comparisons, and angular synchronization. In this report, we propose two spatial GNN methods for node clustering in signed and directed networks, a spectral GNN method for signed directed networks on both node clustering and link prediction, and two GNN methods for specific applications in ranking as well as angular synchronization. The methods are end-to-end in combining embedding generation and prediction without an intermediate step. Experimental results on various data sets, including several synthetic stochastic block models, random graph outlier models, and real-world data sets at different scales, demonstrate that our proposed methods can achieve satisfactory performance, for a wide range of noise and sparsity levels. The introduced models also complement existing methods through the possibility of including exogenous information, in the form of node-level features or labels. Their contribution not only aid the analysis of data which are represented by networks, but also form a body of work which presents novel architectures and task-driven loss functions for GNNs to be used in network analysis

4Sensing - decentralized processing for participatory sensing data

Trabalho apresentado no âmbito do Mestrado em Engenharia Informática, como requisito parcial para obtenção do grau de Mestre em Engenharia Informática.Participatory sensing is a new application paradigm, stemming from both technical and social drives, which is currently gaining momentum as a research domain. It leverages the growing adoption of mobile phones equipped with sensors, such as camera, GPS and accelerometer, enabling users to collect and aggregate data, covering a wide area without incurring in the costs associated with a large-scale sensor network. Related research in participatory sensing usually proposes an architecture based on a centralized back-end. Centralized solutions raise a set of issues. On one side, there is the implications of having a centralized repository hosting privacy sensitive information. On the other side, this centralized model has financial costs that can discourage grassroots initiatives. This dissertation focuses on the data management aspects of a decentralized infrastructure for the support of participatory sensing applications, leveraging the body of work on participatory sensing and related areas, such as wireless and internet-wide sensor networks, peer-to-peer data management and stream processing. It proposes a framework covering a common set of data management requirements - from data acquisition, to processing, storage and querying - with the goal of lowering the barrier for the development and deployment of applications. Alternative architectural approaches - RTree, QTree and NTree - are proposed and evaluated experimentally in the context of a case-study application - SpeedSense - supporting the monitoring and prediction of traffic conditions, through the collection of speed and location samples in an urban setting, using GPS equipped mobile phones

2020 Student Symposium Research and Creative Activity Book of Abstracts

The UMaine Student Symposium (UMSS) is an annual event that celebrates undergraduate and graduate student research and creative work. Students from a variety of disciplines present their achievements with video presentations. It’s the ideal occasion for the community to see how UMaine students’ work impacts locally – and beyond. The 2020 Student Symposium Research and Creative Activity Book of Abstracts includes a complete list of student presenters as well as abstracts related to their works

Music Culture and the Self-Presentation of Indigenous Musicians on Social Media in Contemporary Taiwan

The purpose of this research was to provide an indigenous perspective of popular culture in Taiwan as a means to re-examine Taiwanese contemporary identity. In-depth qualitative interviews and digital ethnography were adopted to collect data about indigenous musicians' self-presentation on social media. Being an indigenous musician in postmodern Taiwan is a highly contested phenomenon, as social media offers a double-edged sword requiring a conjunctional analysis that delves into both the past and the contemporary. This research unpacks the performance of contemporary indigenous musicians in the post-digital media age and offers five findings. Firstly, the indigenous musicians interviewed for the purpose of this research use social media to perform their indigenous identities to wider audiences, both indigenous and nonindigenous. Secondly, identity performances of indigenous musicians on social media are inspired by and reflect the richness and diversity of Taiwanese society. Thirdly, indigenous musicians act as spatio-temporal bridges commuting between urban and rural spaces, on- and offline and between tradition and contemporaneity. Fourthly, indigenous musicians in Taiwan do not only create and perform music, but also give a huge importance to defining and re-articulating what they think indigenous music is and what role it should play in contemporary Taiwanese society. Finally, online selfpresentation provides indigenous musicians with an opportunity to present their performed identities beyond the local to a global audience, allowing non-indigenous audiences to participate in their culture. Using empirical evidence from the interviews and the digital ethnography, this thesis demonstrates how identity performances by Taiwanese indigenous musicians oscillate between three different and inter-related identity processes: ‘doing’ indigenous, ‘being indigenous’, and ‘becoming’ indigenous

Highway construction for wireless sensor networks

Wireless Sensor Networks are a rapidly growing field of study with many open research topics. The aim of this project is to build a hierarchy of clusters in wireless sensor networks and to communicate them through distinguished paths. Those paths are known as highways, and simplify higher level node inter-communication while reducing energy and memory requirements. To achieve this goal several distributed algorithms were designed and tested either in simulators or in real hardware. The message delivery rate, through highways, measured in hardware was close to 70% and it effectively served as base for a higher level network module to make end to end communication between every node of the connected network. This opens a way for the development of more algorithms to make Wireless Sensor Networks communications on large deployments effective and troubleless.Postprint (published version

The Evolutionary Dynamics of Genes and Genomes: Copy Number Variation of the Chalcone Synthase Gene in the Context of Brassicaceae Evolution

The Brassicaceae (Mustards, Cruciferae) are a cosmopolitan family comprising 370 genera and around 3660 species assigned to lately 50 tribes. The tribal system was originally based on solely homoplasious morphological character traits and reaches back to the early 19th century. De Candolle introduced the first tribal classification of the family nearly 200 years ago (1821) containing 21 partly still utilised classifications nowadays. Although labelling seems to be up to date, generic delimitations have been under permanent significant substitution and replacement. The tribes are arranged in three major monophyletic lineages and some additional small groups. The relationships within and between these lineages have not been resolved very clearly yet, as the Brassicaceae are characterised by frequently occurring hybridisation and polyploidisation events. This could be either the result of early and rapid radiation events or perhaps be the product of reticulate evolution, lediang to conflicting gene trees (KOCH & AL-SHEHBAZ 2009). This lack of resolution could in parts be resolved via the application of the nuclear encoded chalcone synthase gene (chs) on 39 of the Cruciferous tribes. Several small-scale tribal-specific duplication events, including age estimations, could be detected giving insight into the evolutionary history of this molecular single- or low-copy gene. Most definitely a tendency towards diploidisation is proven by purifying selection as well as accelerated synonymous substitution rates among this family resulting sooner or later in the reduction of preliminary multiplied chs loci. Supposedly, chs is single-copy in most diploid mustard taxa. The determination of orthologous and paralogous gene copies exposed to be of essential cause as it could be proven that yet functional but fluctuating DNA sequences demonstrate a huge impact on divergence time estimates as well as on any other extrapolation applying nucleotide or amino acid data. However, all crown age estimations calculated with diverse approaches resulted in reasonable output, dating the most recent common ancestor (tmrca) of the family to the Late Miocene or Oligocene. Adjustments of the DNA sequences resulted in a well-resolved thoroughgoing gene tree phylogeny facilitating established taxonomic as well as phylogenetic achievements and do, moreover, hint to further ambiguities which have to be clarified by the commitment of additional marker systems

Game Theory Relaunched

The game is on. Do you know how to play? Game theory sets out to explore what can be said about making decisions which go beyond accepting the rules of a game. Since 1942, a well elaborated mathematical apparatus has been developed to do so; but there is more. During the last three decades game theoretic reasoning has popped up in many other fields as well - from engineering to biology and psychology. New simulation tools and network analysis have made game theory omnipresent these days. This book collects recent research papers in game theory, which come from diverse scientific communities all across the world; they combine many different fields like economics, politics, history, engineering, mathematics, physics, and psychology. All of them have as a common denominator some method of game theory. Enjoy

Can We `Feel' the Temperature of Knowledge? Modelling Scientific Popularity Dynamics via Thermodynamics

Just like everything in the nature, scientific topics flourish and perish. While existing literature well captures article's life-cycle via citation patterns, little is known about how scientific popularity and impact evolves for a specific topic. It would be most intuitive if we could `feel' topic's activity just as we perceive the weather by temperature. Here, we conceive knowledge temperature to quantify topic overall popularity and impact through citation network dynamics. Knowledge temperature includes 2 parts. One part depicts lasting impact by assessing knowledge accumulation with an analogy between topic evolution and isobaric expansion. The other part gauges temporal changes in knowledge structure, an embodiment of short-term popularity, through the rate of entropy change with internal energy, 2 thermodynamic variables approximated via node degree and edge number. Our analysis of representative topics with size ranging from 1000 to over 30000 articles reveals that the key to flourishing is topics' ability in accumulating useful information for future knowledge generation. Topics particularly experience temperature surges when their knowledge structure is altered by influential articles. The spike is especially obvious when there appears a single non-trivial novel research focus or merging in topic structure. Overall, knowledge temperature manifests topics' distinct evolutionary cycles