1,192 research outputs found
A Survey From Distributed Machine Learning to Distributed Deep Learning
Artificial intelligence has achieved significant success in handling complex
tasks in recent years. This success is due to advances in machine learning
algorithms and hardware acceleration. In order to obtain more accurate results
and solve more complex problems, algorithms must be trained with more data.
This huge amount of data could be time-consuming to process and require a great
deal of computation. This solution could be achieved by distributing the data
and algorithm across several machines, which is known as distributed machine
learning. There has been considerable effort put into distributed machine
learning algorithms, and different methods have been proposed so far. In this
article, we present a comprehensive summary of the current state-of-the-art in
the field through the review of these algorithms. We divide this algorithms in
classification and clustering (traditional machine learning), deep learning and
deep reinforcement learning groups. Distributed deep learning has gained more
attention in recent years and most of studies worked on this algorithms. As a
result, most of the articles we discussed here belong to this category. Based
on our investigation of algorithms, we highlight limitations that should be
addressed in future research
Rapid Prototyping of Embedded Vision Systems: Embedding Computer Vision Applications into Low-Power Heterogeneous Architectures
Embedded vision is a disruptive new technology in the vision industry. It is a revolutionary concept with far reaching implications, and it is opening up new applications and shaping the future of entire industries. It is applied in self-driving cars, autonomous vehicles in agriculture, digital dermascopes that help specialists make more accurate diagnoses, among many other unique and cutting-edge applications. The design of such systems gives rise to new challenges for embedded Software developers. Embedded vision applications are characterized by stringent performance constraints to guarantee real-time behaviours and, at the same time, energy constraints to save battery on the mobile platforms. In this paper, we address such challenges by proposing an overall view of the problem and by analysing current solutions. We present our last results on embedded vision design automation over two main aspects: the adoption of the model-based paradigm for the embedded vision rapid prototyping, and the application of heterogeneous programming languages to improve the system performance. The paper presents our recent results on the design of a localization and mapping application combined with image recognition based on deep learning optimized for an NVIDIA Jetson TX2
Parallel programming paradigms and frameworks in big data era
With Cloud Computing emerging as a promising new approach for ad-hoc parallel data processing, major companies have started to integrate frameworks for parallel data processing in their product portfolio, making it easy for customers to access these services and to deploy their programs. We have entered the Era of Big Data. The explosion and profusion of available data in a wide range of application domains rise up new challenges and opportunities in a plethora of disciplines-ranging from science and engineering to biology and business. One major challenge is how to take advantage of the unprecedented scale of data-typically of heterogeneous nature-in order to acquire further insights and knowledge for improving the quality of the offered services. To exploit this new resource, we need to scale up and scale out both our infrastructures and standard techniques. Our society is already data-rich, but the question remains whether or not we have the conceptual tools to handle it. In this paper we discuss and analyze opportunities and challenges for efficient parallel data processing. Big Data is the next frontier for innovation, competition, and productivity, and many solutions continue to appear, partly supported by the considerable enthusiasm around the MapReduce paradigm for large-scale data analysis. We review various parallel and distributed programming paradigms, analyzing how they fit into the Big Data era, and present modern emerging paradigms and frameworks. To better support practitioners interesting in this domain, we end with an analysis of on-going research challenges towards the truly fourth generation data-intensive science.Peer ReviewedPostprint (author's final draft
EvoX: A Distributed GPU-accelerated Library towards Scalable Evolutionary Computation
During the past decades, evolutionary computation (EC) has demonstrated
promising potential in solving various complex optimization problems of
relatively small scales. Nowadays, however, ongoing developments in modern
science and engineering are bringing increasingly grave challenges to the
conventional EC paradigm in terms of scalability. As problem scales increase,
on the one hand, the encoding spaces (i.e., dimensions of the decision vectors)
are intrinsically larger; on the other hand, EC algorithms often require
growing numbers of function evaluations (and probably larger population sizes
as well) to work properly. To meet such emerging challenges, not only does it
require delicate algorithm designs, but more importantly, a high-performance
computing framework is indispensable. Hence, we develop a distributed
GPU-accelerated algorithm library -- EvoX. First, we propose a generalized
workflow for implementing general EC algorithms. Second, we design a scalable
computing framework for running EC algorithms on distributed GPU devices.
Third, we provide user-friendly interfaces to both researchers and
practitioners for benchmark studies as well as extended real-world
applications. To comprehensively assess the performance of EvoX, we conduct a
series of experiments, including: (i) scalability test via numerical
optimization benchmarks with problem dimensions/population sizes up to
millions; (ii) acceleration test via a neuroevolution task with multiple GPU
nodes; (iii) extensibility demonstration via the application to reinforcement
learning tasks on the OpenAI Gym. The code of EvoX is available at
https://github.com/EMI-Group/EvoX
HPC Cloud for Scientific and Business Applications: Taxonomy, Vision, and Research Challenges
High Performance Computing (HPC) clouds are becoming an alternative to
on-premise clusters for executing scientific applications and business
analytics services. Most research efforts in HPC cloud aim to understand the
cost-benefit of moving resource-intensive applications from on-premise
environments to public cloud platforms. Industry trends show hybrid
environments are the natural path to get the best of the on-premise and cloud
resources---steady (and sensitive) workloads can run on on-premise resources
and peak demand can leverage remote resources in a pay-as-you-go manner.
Nevertheless, there are plenty of questions to be answered in HPC cloud, which
range from how to extract the best performance of an unknown underlying
platform to what services are essential to make its usage easier. Moreover, the
discussion on the right pricing and contractual models to fit small and large
users is relevant for the sustainability of HPC clouds. This paper brings a
survey and taxonomy of efforts in HPC cloud and a vision on what we believe is
ahead of us, including a set of research challenges that, once tackled, can
help advance businesses and scientific discoveries. This becomes particularly
relevant due to the fast increasing wave of new HPC applications coming from
big data and artificial intelligence.Comment: 29 pages, 5 figures, Published in ACM Computing Surveys (CSUR
Computational Methods in Science and Engineering : Proceedings of the Workshop SimLabs@KIT, November 29 - 30, 2010, Karlsruhe, Germany
In this proceedings volume we provide a compilation of article contributions equally covering applications from different research fields and ranging from capacity up to capability computing. Besides classical computing aspects such as parallelization, the focus of these proceedings is on multi-scale approaches and methods for tackling algorithm and data complexity. Also practical aspects regarding the usage of the HPC infrastructure and available tools and software at the SCC are presented
- …