3,703 research outputs found
Recommended from our members
High performance latent dirichlet allocation for text mining
This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Latent Dirichlet Allocation (LDA), a total probability generative model, is a three-tier Bayesian model. LDA computes the latent topic structure of the data and obtains the significant information of documents. However, traditional LDA has several limitations in practical applications. LDA cannot be directly used in classification because it is a non-supervised learning model. It needs to be embedded into appropriate classification algorithms. LDA is a generative model as it normally generates the latent topics in the categories where the target documents do not belong to, producing the deviation in computation and reducing the classification accuracy. The number of topics in LDA influences the learning process of model parameters greatly. Noise samples in the training data also affect the final text classification result. And, the quality of LDA based classifiers depends on the quality of the training samples to a great extent. Although parallel LDA algorithms are proposed to deal with huge amounts of data, balancing computing loads in a computer cluster poses another challenge. This thesis presents a text classification method which combines the LDA model and Support Vector Machine (SVM) classification algorithm for an improved accuracy in classification when reducing the dimension of datasets. Based on Density-Based Spatial Clustering of Applications with Noise (DBSCAN), the algorithm automatically optimizes the number of topics to be selected which reduces the number of iterations in computation. Furthermore, this thesis presents a noise data reduction scheme to process noise data. When the noise ratio is large in the training data set, the noise reduction scheme can always produce a high level of accuracy in classification. Finally, the thesis parallelizes LDA using the MapReduce model which is the de facto computing standard in supporting data intensive applications. A genetic algorithm based load balancing algorithm is designed to balance the workloads among computers in a heterogeneous MapReduce cluster where the computers have a variety of computing resources in terms of CPU speed, memory space and hard disk space
Load Balancing in Heterogeneous Cloud Environments by Using PROMETHEE Method
Abstract: Efficient Scheduling of tasks in a cloud environment improves resources utilization thereby meeting users' requirements. One of the most important objectives of a scheduling algorithm in cloud environment is a balanced load distribution over various resources for enhancing the overall performance of the cloud. Such a scheduling is complex in nature due to the dynamicity of resources and incoming application specifications. In this paper, we employ PROMETHEE decision making model to design a scheduling algorithm, called PROMETHEE Load Balancing (PLB).This paper formulates the load balancing issue as a multi-criteria decision making problem and aims to achieve well-balanced load across virtual machines for maximizing the overall throughput of the cloud. Extensive simulation results in CloudSim environment show that the proposed algorithm outperforms existing algorithms in terms of load balancing index (LBI), VM load variation, makespan, average execution time and waiting time
Networking - A Statistical Physics Perspective
Efficient networking has a substantial economic and societal impact in a
broad range of areas including transportation systems, wired and wireless
communications and a range of Internet applications. As transportation and
communication networks become increasingly more complex, the ever increasing
demand for congestion control, higher traffic capacity, quality of service,
robustness and reduced energy consumption require new tools and methods to meet
these conflicting requirements. The new methodology should serve for gaining
better understanding of the properties of networking systems at the macroscopic
level, as well as for the development of new principled optimization and
management algorithms at the microscopic level. Methods of statistical physics
seem best placed to provide new approaches as they have been developed
specifically to deal with non-linear large scale systems. This paper aims at
presenting an overview of tools and methods that have been developed within the
statistical physics community and that can be readily applied to address the
emerging problems in networking. These include diffusion processes, methods
from disordered systems and polymer physics, probabilistic inference, which
have direct relevance to network routing, file and frequency distribution, the
exploration of network structures and vulnerability, and various other
practical networking applications.Comment: (Review article) 71 pages, 14 figure
The cosmological simulation code GADGET-2
We discuss the cosmological simulation code GADGET-2, a new massively
parallel TreeSPH code, capable of following a collisionless fluid with the
N-body method, and an ideal gas by means of smoothed particle hydrodynamics
(SPH). Our implementation of SPH manifestly conserves energy and entropy in
regions free of dissipation, while allowing for fully adaptive smoothing
lengths. Gravitational forces are computed with a hierarchical multipole
expansion, which can optionally be applied in the form of a TreePM algorithm,
where only short-range forces are computed with the `tree'-method while
long-range forces are determined with Fourier techniques. Time integration is
based on a quasi-symplectic scheme where long-range and short-range forces can
be integrated with different timesteps. Individual and adaptive short-range
timesteps may also be employed. The domain decomposition used in the
parallelisation algorithm is based on a space-filling curve, resulting in high
flexibility and tree force errors that do not depend on the way the domains are
cut. The code is efficient in terms of memory consumption and required
communication bandwidth. It has been used to compute the first cosmological
N-body simulation with more than 10^10 dark matter particles, reaching a
homogeneous spatial dynamic range of 10^5 per dimension in a 3D box. It has
also been used to carry out very large cosmological SPH simulations that
account for radiative cooling and star formation, reaching total particle
numbers of more than 250 million. We present the algorithms used by the code
and discuss their accuracy and performance using a number of test problems.
GADGET-2 is publicly released to the research community.Comment: submitted to MNRAS, 31 pages, 20 figures (reduced resolution), code
available at http://www.mpa-garching.mpg.de/gadge
Correlation-based communication in wireless multimedia sensor networks
Wireless multimedia sensor networks (WMSNs) are networks of interconnected devices that allow retrieving video and audio streams, still images, and scalar data from the environment. In a densely deployed WMSN, there exists correlation among the observations of camera sensors with overlapped coverage areas, which introduces substantial data redundancy in the network. In this dissertation, efficient communication schemes are designed for WMSNs by leveraging the correlation of visual information observed by camera sensors. First, a spatial correlation model is developed to estimate the correlation of visual information and the joint entropy of multiple correlated camera sensors. The compression performance of correlated visual information is then studied. An entropy-based divergence measure is proposed to predict the compression efficiency of performing joint coding on the images from correlated cameras. Based on the predicted compression efficiency, a clustered coding technique is proposed that maximizes the overall compression gain of the visual information gathered in WMSNs. The correlation of visual information is then utilized to design a network scheduling scheme to maximize the lifetime of WMSNs. Furthermore, as many WMSN applications require QoS support, a correlation-aware QoS routing algorithm is introduced that can efficiently deliver visual information under QoS constraints. Evaluation results show that, by utilizing the correlation of visual information in the communication process, the energy efficiency and networking performance of WMSNs could be improved significantly.PhDCommittee Chair: Akyildiz, Ian; Committee Member: Ammar, Mostafa; Committee Member: Ji, Chuanyi; Committee Member: Li, Ye; Committee Member: Romberg, Justi
Scaling Expected Force: Efficient Identification of Key Nodes in Network-based Epidemic Models
Centrality measures are fundamental tools of network analysis as they
highlight the key actors within the network. This study focuses on a newly
proposed centrality measure, Expected Force (EF), and its use in identifying
spreaders in network-based epidemic models. We found that EF effectively
predicts the spreading power of nodes and identifies key nodes and immunization
targets. However, its high computational cost presents a challenge for its use
in large networks. To overcome this limitation, we propose two parallel
scalable algorithms for computing EF scores: the first algorithm is based on
the original formulation, while the second one focuses on a cluster-centric
approach to improve efficiency and scalability. Our implementations
significantly reduce computation time, allowing for the detection of key nodes
at large scales. Performance analysis on synthetic and real-world networks
demonstrates that the GPU implementation of our algorithm can efficiently scale
to networks with up to 44 million edges by exploiting modern parallel
architectures, achieving speed-ups of up to 300x, and 50x on average, compared
to the simple parallel solution
- …