3,686 research outputs found
Sparse Cholesky factorization by greedy conditional selection
Dense kernel matrices resulting from pairwise evaluations of a kernel
function arise naturally in machine learning and statistics. Previous work in
constructing sparse approximate inverse Cholesky factors of such matrices by
minimizing Kullback-Leibler divergence recovers the Vecchia approximation for
Gaussian processes. These methods rely only on the geometry of the evaluation
points to construct the sparsity pattern. In this work, we instead construct
the sparsity pattern by leveraging a greedy selection algorithm that maximizes
mutual information with target points, conditional on all points previously
selected. For selecting points out of , the naive time complexity is
, but by maintaining a partial Cholesky factor we reduce
this to . Furthermore, for multiple () targets we
achieve a time complexity of , which is
maintained in the setting of aggregated Cholesky factorization where a selected
point need not condition every target. We apply the selection algorithm to
image classification and recovery of sparse Cholesky factors. By minimizing
Kullback-Leibler divergence, we apply the algorithm to Cholesky factorization,
Gaussian process regression, and preconditioning with the conjugate gradient,
improving over -nearest neighbors selection
Cohesive subgraph identification in large graphs
Graph data is ubiquitous in real world applications, as the relationship among entities in the applications can be naturally captured by the graph model. Finding cohesive subgraphs is a fundamental problem in graph mining with diverse applications. Given the important roles of cohesive subgraphs, this thesis focuses on cohesive subgraph identification in large graphs.
Firstly, we study the size-bounded community search problem that aims to find a subgraph with the largest min-degree among all connected subgraphs that contain the query vertex q and have at least l and at most h vertices, where q, l, h are specified by the query. As the problem is NP-hard, we propose a branch-reduce-and-bound algorithm SC-BRB by developing nontrivial reducing techniques, upper bounding techniques, and branching techniques.
Secondly, we formulate the notion of similar-biclique in bipartite graphs which is a special kind of biclique where all vertices from a designated side are similar to each other, and aim to enumerate all maximal similar-bicliques. We propose a backtracking algorithm MSBE to directly enumerate maximal similar-bicliques, and power it by vertex reduction and optimization techniques. In addition, we design a novel index structure to speed up a time-critical operation of MSBE, as well as to speed up vertex reduction. Efficient index construction algorithms are developed.
Thirdly, we consider balanced cliques in signed graphs --- a clique is balanced if its vertex set can be partitioned into CL and CR such that all negative edges are between CL and CR --- and study the problem of maximum balanced clique computation. We propose techniques to transform the maximum balanced clique problem over G to a series of maximum dichromatic clique problems over small subgraphs of G. The transformation not only removes edge signs but also sparsifies the edge set
Empowering Cloud Data Centers with Network Programmability
Cloud data centers are a critical infrastructure for modern Internet services such as web search, social networking and e-commerce. However, the gradual slow-down of Moore’s law has put a burden on the growth of data centers’ performance and energy efficiency. In addition, the increasing of millisecond-scale and microsecond-scale tasks also bring higher requirements to the throughput and latency for the cloud applications. Today’s server-based solutions are hard to meet the performance requirements in many scenarios like resource management, scheduling, high-speed traffic monitoring and testing.
In this dissertation, we study these problems from a network perspective. We investigate a new architecture that leverages the programmability of new-generation network switches to improve the performance and reliability of clouds. As programmable switches only provide very limited memory and functionalities, we exploit compact data structures and deeply co-design software and hardware to best utilize the resource. More specifically, this dissertation presents four systems:
(i) NetLock: A new centralized lock management architecture that co-designs programmable switches and servers to simultaneously achieve high performance and rich policy support. It provides orders-of-magnitude higher throughput than existing systems with microsecond-level latency, and supports many commonly-used policies such as performance isolation.
(ii) HCSFQ: A scalable and practical solution to implement hierarchical fair queueing on commodity hardware at line rate. Instead of relying on a hierarchy of queues with complex queue management, HCSFQ does not keep per-flow states and uses only one queue to achieve hierarchical fair queueing.
(iii) AIFO: A new approach for programmable packet scheduling that only uses a single FIFO queue. AIFO utilizes an admission control mechanism to approximate PIFO which is theoretically ideal but hard to implement with commodity devices.
(iv) Lumina: A tool that enables fine-grained analysis of hardware network stack. By exploiting network programmability to emulate various network scenarios, Lumina is able to help users understand the micro-behaviors of hardware network stacks
Sex for sale : a comparative analysis of legal models and the socio-economic determinants informing law reform in South Africa
The overarching purpose of this research is to establish the need for legislative and policy reform in respect of the exchange of sexual acts for reward and peripheral crimes in South Africa. It has sought to do so through interrogating the context and the socio-economic determinants at work in South Africa, Canada, Sweden and India; and how each country’s unique context intersects with the chosen policy and legal framework. As a starting point the current criminalised legal framework is positioned as a constitutionally permissible legislative choice. In light of the fact that legislatures the world around draft legislation which regulates sensitive areas of morality, it would seem that the underlying question is which interpretation or understanding of morality should be used to inform legislative reform. This thesis is based on the assumption that the chosen legal framework is a policy choice and argues that this policy should be informed by available evidence and rational analysis, as opposed to political ideology. Evidenced-based research has been used to consider the complexity of the primary socio-economic drivers of poverty, inequality, unemployment and intersecting vulnerabilities such as early exposure to sexual violence as causal to entry into the sale of sexual services. Three global themes underpinning the need for law reform have been considered comparatively, including documenting the unfolding developments around law reform. This research has considered South Africa’s obligations in terms of binding international instruments to bring about normative change in respect of gender-based violence; recognise economic and social rights; curtail aspects associated with the exploitation of prostitution; guard against sex tourism; suppress all forms of trafficking in women, and address patriarchy and traditional stereotypes of women as sex objects. It is axiomatic that the solution to a problem sustained by socio-economic drivers is to be found in the disruption of those very same drivers. The outcome of this research is a recommendation in favour of substantive equality in the form of a country-specific legal framework of partial criminalisation coupled with the realisation of socio-economic rights.Public, Constitutional, and International LawLL. D
LIPIcs, Volume 274, ESA 2023, Complete Volume
LIPIcs, Volume 274, ESA 2023, Complete Volum
What Matters in Model Training to Transfer Adversarial Examples
Despite state-of-the-art performance on natural data, Deep Neural Networks (DNNs) are highly vulnerable to adversarial examples, i.e., imperceptible, carefully crafted perturbations of inputs applied at test time. Adversarial examples can transfer: an adversarial example against one model is likely to be adversarial against another independently trained model. This dissertation investigates the characteristics of the surrogate weight space that lead to the transferability of adversarial examples. Our research covers three complementary aspects of the weight space exploration: the multimodal exploration to obtain multiple models from different vicinities, the local exploration to obtain multiple models in the same vicinity, and the point selection to obtain a single transferable representation.
First, from a probabilistic perspective, we argue that transferability is fundamentally related to uncertainty. The unknown weights of the target DNN can be treated as random variables. Under a specified threat model, deep ensemble can produce a surrogate by sampling from the distribution of the target model. Unfortunately, deep ensembles are computationally expensive. We propose an efficient alternative by approximately sampling surrogate models from the posterior distribution using cSGLD, a state-of-the-art Bayesian deep learning technique. Our extensive experiments show that our approach improves and complements four attacks, three transferability techniques, and five more training methods significantly on ImageNet, CIFAR-10, and MNIST (up to 83.2 percentage points), while reducing training computations from 11.6 to 2.4 exaflops compared to deep ensemble on ImageNet.
Second, we propose transferability from Large Geometric Vicinity (LGV), a new technique based on the local exploration of the weight space. LGV starts from a pretrained model and collects multiple weights in a few additional training epochs with a constant and high learning rate. LGV exploits two geometric properties that we relate to transferability. First, we show that LGV explores a flatter region of the weight space and generates flatter adversarial examples in the input space. We present the surrogate-target misalignment hypothesis to explain why flatness could increase transferability. Second, we show that the LGV weights span a dense weight subspace whose geometry is intrinsically connected to transferability. Through extensive experiments, we show that LGV alone outperforms all (combinations of) four established transferability techniques by 1.8 to 59.9 percentage points.
Third, we investigate how to train a transferable representation, that is, a single model for transferability. First, we refute a common hypothesis from previous research to explain why early stopping improves transferability. We then establish links between transferability and the exploration dynamics of the weight space, in which early stopping has an inherent effect. More precisely, we observe that transferability peaks when the learning rate decays, which is also the time at which the sharpness of the loss significantly drops. This leads us to propose RFN, a new approach to transferability that minimises the sharpness of the loss during training. We show that by searching for large flat neighbourhoods, RFN always improves over early stopping (by up to 47 points of success rate) and is competitive to (if not better than) strong state-of-the-art baselines.
Overall, our three complementary techniques provide an extensive and practical method to obtain highly transferable adversarial examples from the multimodal and local exploration of flatter vicinities in the weight space. Our probabilistic and geometric approaches demonstrate that the way to train the surrogate model has been overlooked, although both the training noise and the flatness of the loss landscape are important elements of transfer-based attacks
C++ Design Patterns for Low-latency Applications Including High-frequency Trading
This work aims to bridge the existing knowledge gap in the optimisation of
latency-critical code, specifically focusing on high-frequency trading (HFT)
systems. The research culminates in three main contributions: the creation of a
Low-Latency Programming Repository, the optimisation of a market-neutral
statistical arbitrage pairs trading strategy, and the implementation of the
Disruptor pattern in C++. The repository serves as a practical guide and is
enriched with rigorous statistical benchmarking, while the trading strategy
optimisation led to substantial improvements in speed and profitability. The
Disruptor pattern showcased significant performance enhancement over
traditional queuing methods. Evaluation metrics include speed, cache
utilisation, and statistical significance, among others. Techniques like Cache
Warming and Constexpr showed the most significant gains in latency reduction.
Future directions involve expanding the repository, testing the optimised
trading algorithm in a live trading environment, and integrating the Disruptor
pattern with the trading algorithm for comprehensive system benchmarking. The
work is oriented towards academics and industry practitioners seeking to
improve performance in latency-sensitive applications
- …