Adaptive Partitioning for Large-Scale Dynamic Graphs

Abstract—In the last years, large-scale graph processing has gained increasing attention, with most recent systems placing particular emphasis on latency. One possible technique to improve runtime performance in a distributed graph processing system is to reduce network communication. The most notable way to achieve this goal is to partition the graph by minimizing the num-ber of edges that connect vertices assigned to different machines, while keeping the load balanced. However, real-world graphs are highly dynamic, with vertices and edges being constantly added and removed. Carefully updating the partitioning of the graph to reflect these changes is necessary to avoid the introduction of an extensive number of cut edges, which would gradually worsen computation performance. In this paper we show that performance degradation in dynamic graph processing systems can be avoided by adapting continuously the graph partitions as the graph changes. We present a novel highly scalable adaptive partitioning strategy, and show a number of refinements that make it work under the constraints of a large-scale distributed system. The partitioning strategy is based on iterative vertex migrations, relying only on local information. We have implemented the technique in a graph processing system, and we show through three real-world scenarios how adapting graph partitioning reduces execution time by over 50 % when compared to commonly used hash-partitioning. I

xDGP: A Dynamic Graph Processing System with Adaptive Partitioning

13 pagesMany real-world systems, such as social networks, rely on mining efficiently large graphs, with hundreds of millions of vertices and edges. This volume of information requires partitioning the graph across multiple nodes in a distributed system. This has a deep effect on performance, as traversing edges cut between partitions incurs a significant performance penalty due to the cost of communication. Thus, several systems in the literature have attempted to improve computational performance by enhancing graph partitioning, but they do not support another characteristic of real-world graphs: graphs are inherently dynamic, their topology evolves continuously, and subsequently the optimum partitioning also changes over time. In this work, we present the first system that dynamically repartitions massive graphs to adapt to structural changes. The system optimises graph partitioning to prevent performance degradation without using data replication. The system adopts an iterative vertex migration algorithm that relies on local information only, making complex coordination unnecessary. We show how the improvement in graph partitioning reduces execution time by over 50%, while adapting the partitioning to a large number of changes to the graph in three real-world scenarios

Individuation and holistic processing of faces in rhesus monkeys

Despite considerable evidence that neural activity in monkeys reflects various aspects of face perception, relatively little is known about monkeys' face processing abilities. Two characteristics of face processing observed in humans are a subordinate-level entry point, here, the default recognition of faces at the subordinate, rather than basic, level of categorization, and holistic effects, i.e. perception of facial displays as an integrated whole. The present study used an adaptation paradigm to test whether untrained rhesus macaques (Macaca mulatta) display these hallmarks of face processing. In experiments 1 and 2, macaques showed greater rebound from adaptation to conspecific faces than to other animals at the individual or subordinate level. In experiment 3, exchanging only the bottom half of a monkey face produced greater rebound in aligned than in misaligned composites, indicating that for normal, aligned faces, the new bottom half may have influenced the perception of the whole face. Scan path analysis supported this assertion: during rebound, fixation to the unchanged eye region was renewed, but only for aligned stimuli. These experiments show that macaques naturally display the distinguishing characteristics of face processing seen in humans and provide the first clear demonstration that holistic information guides scan paths for conspecific faces

Ready ... Go: Amplitude of the fMRI Signal Encodes Expectation of Cue Arrival Time

What happens when the brain awaits a signal of uncertain arrival time, as when a sprinter waits for the starting pistol? And what happens just after the starting pistol fires? Using functional magnetic resonance imaging (fMRI), we have discovered a novel correlate of temporal expectations in several brain regions, most prominently in the supplementary motor area (SMA). Contrary to expectations, we found little fMRI activity during the waiting period; however, a large signal appears after the “go” signal, the amplitude of which reflects learned expectations about the distribution of possible waiting times. Specifically, the amplitude of the fMRI signal appears to encode a cumulative conditional probability, also known as the cumulative hazard function. The fMRI signal loses its dependence on waiting time in a “countdown” condition in which the arrival time of the go cue is known in advance, suggesting that the signal encodes temporal probabilities rather than simply elapsed time. The dependence of the signal on temporal expectation is present in “no-go” conditions, demonstrating that the effect is not a consequence of motor output. Finally, the encoding is not dependent on modality, operating in the same manner with auditory or visual signals. This finding extends our understanding of the relationship between temporal expectancy and measurable neural signals

Long-Term Stability of Visual Pattern Selective Responses of Monkey Temporal Lobe Neurons

Many neurons in primate inferotemporal (IT) cortex respond selectively to complex, often meaningful, stimuli such as faces and objects. An important unanswered question is whether such response selectivity, which is thought to arise from experience-dependent plasticity, is maintained from day to day, or whether the roles of individual cells are continually reassigned based on the diet of natural vision. We addressed this question using microwire electrodes that were chronically implanted in the temporal lobe of two monkeys, often allowing us to monitor activity of individual neurons across days. We found that neurons maintained their selectivity in both response magnitude and patterns of spike timing across a large set of visual images throughout periods of stable signal isolation from the same cell that sometimes exceeded two weeks. These results indicate that stimulus-selectivity of responses in IT is stable across days and weeks of visual experience

Using NMF for analyzing war logs

We investigate a semi-automated identification of technical problems occurred by armed forces weapon systems during mission of war. The proposed methodology is based on a semantic analysis of textual information in reports from soldiers (war logs). Latent semantic indexing (LSI) with non-negative matrix factorization (NMF) as technique from multivariate analysis and linear algebra is used to extract hidden semantic textual patterns from the reports. NMF factorizes the term-by-war log matrix - that consists of weighted term frequencies into two non-negative matrices. This enables natural parts-based representation of the report information and it leads to an easy evaluation by human experts because human brain also uses parts-based representation. For an improved research and technology planning, the identified technical problems are a valuable source of information. A case study extracts technical problems from military logs of the Afghanistan war. Results are compared to a manual analysis written by journalists of 'Der Spiegel'

WannaCry Ransomware: Analysis of Infection, Persistence, Recovery Prevention and Propagation Mechanisms, Journal of Telecommunications and Information Technology, 2019, nr 1

In recent years, we have been experiencing fast proliferation of different types of ransomware targeting home users, companies and even critical telecommunications infrastructure elements. Modern day ransomware relies on sophisticated infection, persistence and recovery prevention mechanisms. Some recent examples that received significant attention include WannaCry, Petya and BadRabbit. To design and develop appropriate defense mechanisms, it is important to understand the characteristics and the behavior of different types of ransomware. Dynamic analysis techniques are typically used to achieve that purpose, where the malicious binaries are executed in a controlled environment and are then observed. In this work, the dynamic analysis results focusing on the infamous WannaCry ransomware are presented. In particular, WannaCry is examined, during its execution in a purpose-built virtual lab environment, in order to analyze its infection, persistence, recovery prevention and propagation mechanisms. The results obtained may be used for developing appropriate detection and defense solutions for WannaCry and other ransomware families that exhibit similar behavior

Loss of high-level perceptual knowledge of object structure in DAT

Peer reviewe