Super-resolution community detection for layer-aggregated multilayer networks

Applied network science often involves preprocessing network data before applying a network-analysis method, and there is typically a theoretical disconnect between these steps. For example, it is common to aggregate time-varying network data into windows prior to analysis, and the tradeoffs of this preprocessing are not well understood. Focusing on the problem of detecting small communities in multilayer networks, we study the effects of layer aggregation by developing random-matrix theory for modularity matrices associated with layer-aggregated networks with $N$ nodes and $L$ layers, which are drawn from an ensemble of Erd\H{o}s-R\'enyi networks. We study phase transitions in which eigenvectors localize onto communities (allowing their detection) and which occur for a given community provided its size surpasses a detectability limit $K^*$. When layers are aggregated via a summation, we obtain $K^*\varpropto \mathcal{O}(\sqrt{NL}/T)$, where $T$ is the number of layers across which the community persists. Interestingly, if $T$ is allowed to vary with $L$ then summation-based layer aggregation enhances small-community detection even if the community persists across a vanishing fraction of layers, provided that $T/L$ decays more slowly than $\mathcal{O}(L^{-1/2})$. Moreover, we find that thresholding the summation can in some cases cause $K^*$ to decay exponentially, decreasing by orders of magnitude in a phenomenon we call super-resolution community detection. That is, layer aggregation with thresholding is a nonlinear data filter enabling detection of communities that are otherwise too small to detect. Importantly, different thresholds generally enhance the detectability of communities having different properties, illustrating that community detection can be obscured if one analyzes network data using a single threshold.Comment: 11 pages, 8 figure

Elintarvikelainsäädännön implementoinnin haasteita

After a series of food incidents in the 1990s, the food business sector has become one of the most heavily regulated sectors in the European Union (EU), with ever-evolving regulations regarding both official food control and food business operators (FBOs). The regulatory framework is meaningless if the regulation is not implemented promptly according to transitional provisions and in a unified way. Dissenting implementation of food safety legislation may also endanger the equal treatment of FBOs and the principle of free trade of foodstuff in the EU. This research provides a new perspective on the challenges of implementing food legislation, with the phenomenon surveyed from the viewpoints of both control officials and FBOs. Resources and organization of food control affect actual control work and were thus included. Varying ways of reporting and handling food frauds on the local, national and EU levels were also investigated. Fulfilling food control requirements set in food legislation necessitates an adequate quantity and quality of personnel, whereas organization of food control can differ between countries or areas depending on socio-economic and political factors. In Finland, municipalities alone or as a joint control unit are responsible for local food control in their respective areas. According to the results, this may lead to varying implementation and interpretation of food legislation, endangering equal treatment of FBOs. There is an alarming shortage of food control personnel in some regions in Finland. Even when food safety is the responsibility of FBOs, scarce resources in food control result in a lower percentage of approved in-house control systems among FBOs. This research revealed a connection between the number of approved in-house control systems and the number of reported food- or waterborne outbreaks in the area, especially in regions with inadequate food control resources. EU legislation concerning quality systems, food control plans and food control fees are implemented in Finland at regionally different time points and with different contents directly influencing FBOs in regionally variable ways. Control officials support larger control units, with the rationale that they will increase equal treatment of FBOs. Both control officials and FBOs have problems in implementation of food legislation, and FBOs are also challenged with varying interpretation of legislation and requirements of control officials. As food safety is the responsibility of the FBOs, they need to understand and carefully comply with legislation. The challenges of fish and meat FBOs in implementation of legislation were therefore evaluated. According to this study, the most common problems concerning food safety legislation are related to layout of production premises and transport routes, control fees, requirements concerning in-house control and structures and maintenance of premises. Risk evaluation is problematic for both control officers and FBOs. Traditional food control measures are challenged, when requirements set by law are intentionally violated for financial gain by FBOs, with food deliberately placed on the market with the intention of deceiving the consumer (food fraud). Uniform methods to detect and report food fraud are needed. Hence patterns of food frauds published in the EU Rapid Alert System for Food and Feed (RASFF) in 2008-2012, recalls of notifications published by the Finnish Food Safety Authority Evira in 2008-2012 and local Finnish food fraud cases in 2003-2012 were analysed. Patterns of food fraud and manners of reporting frauds at the local, national and EU levels differ significantly. If the detection and reporting of frauds and the legal consequences incurred by FBOs for frauds differ among member states, it may create distortion of competition.Useat elintarvikekriisit 1990-luvulla aiheuttivat sen, että elintarvikesektorista on tullut yksi eniten säännellyistä aloista Euroopan unionissa (EU). Jatkuvasti kehittyvät ja uudistuvat säädökset koskevat sekä virallista elintarvikevalvontaa että elintarvikealan toimijoita. Säädösten toimivuus edellyttää, että sekä valvojat että toimijat alkavat soveltaa säädöksiä viipymättä ja samalla tavalla. Säädösten erilainen toimeenpano tai soveltaminen voi vaarantaa toimijoiden yhdenvertaisen kohtelun, aiheuttaa kilpailun vääristymistä tai vaikeuttaa elintarvikkeiden vapaata liikkuvuutta EU:n alueella. Tässä tutkimuksessa tutkittiin elintarvikelainsäädännön toimeenpanoon ja soveltamiseen liittyviä haasteita sekä elintarvikevalvojien että elintarvikealan toimijoiden näkökulmasta. Elintarvikevalvonnan resurssit ja organisointi vaikuttavat suoraan valvontatyöhön ja ovat siksi tutkimuksessa mukana. Tutkimus käsitteli myös elintarvikkeisiin liittyviä petoksia ja niiden vaihtelevia raportointi- ja käsittelytapoja paikallisella, kansallisella ja EU-tasolla. Elintarvikevalvonnalle asetettujen vaatimusten täyttäminen edellyttää riittävää henkilöstöä. Suomessa paikallisesta elintarvikevalvonnasta vastaavat kunnat tai kuntien yhteiset suuremmat valvontayksiköt. Suomessa elintarvikevalvonnan henkilöstön määrä vaihtelee alueittain ja joillain alueilla on huomattava henkilöstövaje. Valvojien mukaan henkilöstövaje huonontaa elintarvikevalvonnan laatua. Tutkimuksessa paljastui yhteys elintarvikevalvonnan rajallisten resurssien ja toimijoiden hyväksyttyjen omavalvontasuunnitelmien sekä raportoitujen elintarvike- ja vesivälitteisten epidemioiden määrässä. EU:n lainsäädännön määräykset elintarvikevalvonnan laatujärjestelmistä, valvontasuunnitelmista ja valvontamaksuista on otettu Suomessa käyttöön alueellisesti eri ajankohtina ja sisällöltään erilaisina, mikä aiheuttaa elintarviketoimijoiden alueellisen kohtelun vaihtelua. Valvojat kannattavat suuria valvontayksiköitä ja katsovat niiden lisäävän toimijoiden tasapuolista kohtelua ja yhdenmukaista lainsäädännön soveltamista. Sekä valvojilla että toimijoilla on vaikeuksia elintarvikelainsäädännön soveltamisessa. Valvojien vaihtelevat tulkinnat lainsäädännön vaatimuksista aiheuttavat toimijoille vaikeuksia. Elintarvikealan toimijat vastaavat käsittelemiensä elintarvikkeiden turvallisuudesta ja heidän täytyy tuntea lainsäädäntö ja osata soveltaa sitä. Tutkimuksessa selvitettiin liha- ja kala-alan toimijoiden suurimmat lainsäädännön vaatimuksia koskevat ongelmat. Suurimmat ongelmat liittyivät tuotantotilojen asetteluun, kuljetusreitteihin, valvontamaksuihin, omavalvonnan vaatimuksiin sekä tilojen rakenteisiin. Riskien arviointi oli vaikeaa niin valvojille kuin toimijoillekin. Perinteiset valvontakeinot ovat riittämättömiä, jos elintarvikealan toimija jättää tietoisesti noudattamatta lain vaatimuksia tai harhauttaa kuluttajia saadakseen taloudellista hyötyä. Tutkimuksessa analysoitiin EU:n elintarvikkeita ja rehuja koskevaan nopeaan hälytysjärjestelmään (RASFF) tehdyt elintarvikepetokset, Eviran julkaisemat vastaavat elintarvikkeiden takaisinvedot 2008-2012 sekä Suomessa havaittuja elintarvikepetoksia 2003-2012. Tulosten mukaan elintarvikepetoksia tutkitaan, raportoidaan ja sanktioidaan hyvin vaihtelevilla tavoilla EU:ssa. Elintarvikepetosten torjunta vaatii yhdenmukaisia menettelytapoja ja sääntöjä. Vaihteleva suhtautuminen elintarvikepetoksiin voi pahimmillaan aiheuttaa kilpailun vääristymistä

Super-Resolution Community Detection for Layer-Aggregated Multilayer Networks

Applied network science often involves preprocessing network data before applying a network-analysis method, and there is typically a theoretical disconnect between these steps. For example, it is common to aggregate time-varying network data into windows prior to analysis, and the trade-offs of this preprocessing are not well understood. Focusing on the problem of detecting small communities in multilayer networks, we study the effects of layer aggregation by developing random-matrix theory for modularity matrices associated with layer-aggregated networks with N nodes and L layers, which are drawn from an ensemble of Erdős–Rényi networks with communities planted in subsets of layers. We study phase transitions in which eigenvectors localize onto communities (allowing their detection) and which occur for a given community provided its size surpasses a detectability limit K*. When layers are aggregated via a summation, we obtain K∗∝O(NL/T), where T is the number of layers across which the community persists. Interestingly, if T is allowed to vary with L, then summation-based layer aggregation enhances small-community detection even if the community persists across a vanishing fraction of layers, provided that T/L decays more slowly than (L−1/2). Moreover, we find that thresholding the summation can, in some cases, cause K* to decay exponentially, decreasing by orders of magnitude in a phenomenon we call super-resolution community detection. In other words, layer aggregation with thresholding is a nonlinear data filter enabling detection of communities that are otherwise too small to detect. Importantly, different thresholds generally enhance the detectability of communities having different properties, illustrating that community detection can be obscured if one analyzes network data using a single threshold

Network based data oriented methods for application driven problems

Networks are amazing. If you think about it, some of them can be found in almost every single aspect of our life from sociological, financial and biological processes to the human body. Even considering entities that are not necessarily connected to each other in a natural sense, can be connected based on real life properties, creating a whole new aspect to express knowledge. A network as a structure implies not only interesting and complex mathematical questions, but the possibility to extract hidden and additional information from real life data. The data that is one of the most valuable resources of this century. The different activities of the society and the underlying processes produces a huge amount of data, which can be available for us due to the technological knowledge and tools we have nowadays. Nevertheless, the data without the contained knowledge does not represent value, thus the main focus in the last decade is to generate or extract information and knowledge from the data. Consequently, data analytics and science, as well as data-driven methodologies have become leading research fields both in scientific and industrial areas. In this dissertation, the author introduces efficient algorithms to solve application oriented optimization and data analysis tasks built on network science based models. The main idea is to connect these problems along graph based approaches, from virus modelling on an existing system through understanding the spreading mechanism of an infection/influence and maximize or minimize the effect, to financial applications, such as fraud detection or cost optimization in a case of employee rostering

The role of protozoan genetic diversity in human disease : implications for the epidemiology of cryptosporidiosis and giardiasis in New Zealand : a thesis submitted in partial fulfilment of the requirements for the degree of Doctor of Philosophy, Massey University, Palmerston North, New Zealand

Cryptosporidium and Giardia are two common causes of diarrhoea in humans and livestock and responsible for multiple outbreaks of gastroenteritis every year in New Zealand and around the globe. Despite their prevalence, there are few effective therapies or vaccines, against either parasite. This is largely due to the difficulty of manipulating these parasites in vitro. The understanding of the epidemiology of this parasite in New Zealand is incomplete, due to the presence of multiple dominant subtypes of each parasite within samples from the same outbreak. In this thesis, new techniques are employed to investigate the genetic diversity of these parasites within hosts and develop an in vitro assay for comparing the infectivity of multiple subtypes of Cryptosporidium. Current methods for the purification of Giardia cysts from faecal samples do not adequately remove debris from the sample and produce low numbers of purified cysts. This hampers molecular techniques that benefit from uncontaminated samples resulting in the use of expensive methods like immunomagnetic separation. Here, a novel method for the purification of cysts from faecal samples was developed, which produced purified oocysts with negligible debris and a 10-fold increase in yield over current techniques. Epidemiological and molecular investigations of past giardiasis and cryptosporidiosis outbreaks in New Zealand have highlighted inconsistent results, where epidemiologically linked cases can have different dominant subtypes identified through Sanger sequencing. Here, amplicon-based metabarcoding was utilised to resolve Giardia and Cryptosporidium outbreak epidemiology in New Zealand. Human faecal samples from past outbreaks previously classified using Sanger sequencing were analysed using next-generation sequencing. This strategy uncovered significant within-host diversity and identified potential emerging subtypes of Cryptosporidium that could have public health significance in the future. Analysis of diversity within outbreaks provided previously unidentified genetic links between samples from the same outbreak. Previous studies show that people experience different symptoms depending on the subtype of Cryptosporidium they are infected with. Also, the dominant subtypes of the parasite in a region, like the USA and Australia, have changed multiple times within the past 20 years. This suggests there are differences in infectivity between subtypes, but further analysis of this problem has been hampered by the lack of adequate cell culture systems that allow the complete development of the parsite in vitro. To better understand the differences in infectivity between subtypes of Cryptosporidium, an analysis of the expression of Cryptosporidium genes in the COLO-680N cell line at multiple timepoints during infection was carried out using the NanoString nCounter analysis system. This was done to investigate whether differences in gene expression could account for differences in infectivity. Furthermore, utilising flow cytometry a system was developed capable of identifying and quantifying infection in infected cells with and without the use of a fluorescent antibody. A novel signal was identified in the near-infra red range that was specific to Cryptosporidium infection and showed better signalling characteristics than the fluorophore

Analyzing Tweets For Predicting Mental Health States Using Data Mining And Machine Learning Algorithms

Tweets are usually the outcome of peoples’ feelings on various topics. Twitter allows users to post casual and emotional thoughts to share in real-time. Around 20% of U.S. adults use Twitter. Using the word-frequency and singular value decomposition methods, we identified the behavior of individuals through their tweets. We graded depressive and anti-depressive keywords using the tweet time-series, time-window, and time-stamp methods. We have collected around four million tweets since 2018. A parameter (Depressive Index) is computed using the F1 score and Mathews correlation coefficient (MCC) to indicate the depressive level. A framework showing the Depressive Index and the Happiness Index is prepared with the time, location, and keywords and delivers F1 Score, MCC, and CI values. COVID-19 changed the routines of most peoples\u27 lives and affected mental health. We studied the tweets and compared them with the COVID-19 growth. The Happiness Index from our work and World Happiness Report for Georgia, New York, and Sri Lanka is compared. An interactive framework is prepared to analyze the tweets, depict the happiness index, and compare it. Bad words in tweets are analyzed, and a map showing the Happiness Index is computed for all the US states and was compared with WalletHub data. We add tweets continuously and a framework delivering an atlas of maps based on the Happiness Index and make these maps available for further study. We forecasted tweets with real-time data. Our results of tweets and COVID-19 reports (WHO) are in a similar pattern. A new moving average method was presented; this unique process gave perfect results at peaks of the function and improved the error percentage. An interactive GUI portal computes the Happiness Index, depression index, feel-good- factors, prediction of the keywords, and prepares a Happiness Index map. We plan to create a public web portal to facilitate users to get these results. Upon completing the proposed GUI application, the users can get the Happiness Index, Depression Index values, Happiness map, and prediction of keywords of the desired dates and geographical locations instantaneously

Critical Market Crashes

This review is a partial synthesis of the book ``Why stock market crash'' (Princeton University Press, January 2003), which presents a general theory of financial crashes and of stock market instabilities that his co-workers and the author have developed over the past seven years. The study of the frequency distribution of drawdowns, or runs of successive losses shows that large financial crashes are ``outliers'': they form a class of their own as can be seen from their statistical signatures. If large financial crashes are ``outliers'', they are special and thus require a special explanation, a specific model, a theory of their own. In addition, their special properties may perhaps be used for their prediction. The main mechanisms leading to positive feedbacks, i.e., self-reinforcement, such as imitative behavior and herding between investors are reviewed with many references provided to the relevant literature outside the confine of Physics. Positive feedbacks provide the fuel for the development of speculative bubbles, preparing the instability for a major crash. We demonstrate several detailed mathematical models of speculative bubbles and crashes. The most important message is the discovery of robust and universal signatures of the approach to crashes. These precursory patterns have been documented for essentially all crashes on developed as well as emergent stock markets, on currency markets, on company stocks, and so on. The concept of an ``anti-bubble'' is also summarized, with two forward predictions on the Japanese stock market starting in 1999 and on the USA stock market still running. We conclude by presenting our view of the organization of financial markets.Comment: Latex 89 pages and 38 figures, in press in Physics Report

Application of nuclear magnetic resonance spectroscopy in the study of complex matrices

The aim of this PhD work was to apply the NMR based metabolomic approach to the study of complex matrices such as several food plants (pepper, celery, tomatoes, hemp, baobab, teas, blueberries and olive oils). A comprehensive description of the chemical composition in term of primary and secondary metabolites obtained by means of 1D and 2D experiments was reported and information regarding specific aspects (variety, type of production etc) were obtained. The study of stool samples of patients with liver cirrhosis was also carried out confirming the important contribution of the NMR approach in the disease investigation