323 research outputs found
Tools Used in Big Data Analytics
Big data is the current state of the art topic creating its unique place in the research and industry minds to look into depth of topic to get valuable results needed to meet the future data mining and analysis needs. Big data refers to enormous amounts of unstructured data created as a result of high performance applications ranging from scientific to social networks, from e-government to medical information system and so on. So, there also prevails the need of to analyze the data to get valuable data results from it. This paper deals with analytic emphasis on big data and what are the different tools used for big data analysis In this paper, different sections through an overlook on different aspects on big data such as big data analysis, big data storage techniques and tools used for big data analysis
Data analytics in IoT FaaS with DataFlasks
Dissertação de mestrado em Computer ScienceThe current exponential growth of data demands new strategies for processing and analyzing information.
Increased Internet usage, as well as the everyday appearance of new sources of data, is
generating data volumes to be processed by Cloud applications that are growing much faster than
available Cloud computing power.
These issues, combined with the appearance of new devices with relatively low computational
power (such as smartphones), have pushed for the development of new applications able to make
use of this power as a complement to the Cloud, pushing the frontier of computing applications,
data storage and services to the edge of the network.
However, the environment in Edge computing is very unstable. It requires leveraging resources
that may not be continuously connected to a network and device failure is a certainty. The system
has to be aware of the processing capabilities of each node to achieve proper task distribution as it
may exist a high level of heterogeneity between the system devices.
A recent approach for developing applications in the Cloud, named Function as a Service (FaaS),
proposes a way to enable data processing in these environments. FaaS services adhere to the principles
of serverless architectures, providing stateless computing containers that allow users to run
code without provisioning or managing servers.
In this dissertation we present OpenFlasks, a new approach to the management and processing
of data in a decentralized manner across Cloud and Edge. We build upon these types of architectures
and other data storage tools and combine them in a novel way to create a flexible system
capable of balancing data storage and data analytics needs in both environments. In addition, we
call for a new approach to provide task execution both in Edge and Cloud environments that is able
to handle high churn and heterogeneity of the system.
Our evaluation shows an increase in the percentage of task execution success under high churn
environments of up to 18%withOpenFlasks relatively to other FaaS systems. In addition, it denotes
improvements in load balancing and average resource usage in the system for the execution of
simple analytics at the Edge.O atual crescimento exponencial de dados exige novas estratégias para processar e analisar informação.
O aumento do uso da Internet, assim como o aparecimento diário de novas fontes de
dados, produz volumes de dados a ser processados por aplicações Cloud que crescem a umamaior
velocidade do que o poder de computação aí disponível.
Este problema, combinado com o surgir de novos dispositivos com poder computacional relativamente
baixo (como smartphones), tem motivado o desenvolvimento de novas aplicações capazes
de usar esse poder como complemento a Cloud computing, expandindo a fronteira dos
serviços de processamento e armazenamento de dados atuais para o limite da rede (Edge).
No entanto, o ambiente de Edge computing é muito instável. Requer a gestão de recursos que
podem não estar continuamente conectados à rede e a falha de dispositivos é uma certeza. O
sistema deve estar ciente das capacidades de processamento de cada dispositivo para obter uma
distribuição de tarefas adequada, dado que pode existir um alto nível de heterogeneidade entre os
dispositivos do sistema.
Uma abordagem recente para o desenvolvimento de aplicações de Cloud computing, denominada
Function as a Service (FaaS), propõe uma forma de permitir o processamento de dados neste
tipo de ambientes. Os serviços FaaS aderem aos princípios de arquiteturas serverless, fornecendo
containers de computação que nãomantêmestado e que permitemaos utilizadores executar código
sem a necessidade de instanciar e gerir servidores.
Nesta dissertação apresentamos OpenFlasks, uma nova abordagem para a gestão e processamento
de dados de forma descentralizada em ambientes Cloud e Edge. Baseamo-nos neste tipo
de arquiteturas, assimcomo outros serviços atuais de armazenamento de dados e combinamo-los
de forma a criar um sistema flexível, capaz de equilibrar o armazenamento e as necessidades de
análise de dados em ambos ambientes. Além disso, propomos uma nova abordagem para possibilitar
a execução de tarefas tanto em ambientes de Edge como de Cloud, capaz de lidar com o
elevado dinamismo e heterogeneidade do sistema.
A nossa avaliação mostra um aumento na percentagem de sucesso da execução de tarefas sob
ambientes de elevado dinamismo de até 18% relativamente a outros sistemas FaaS. Além disso,
denotamelhorias na distribuição de carga e no uso médio de recursos do sistema para a execução
de data analytics simples em ambientes Edge
Sensing as a service: A cloud computing system for mobile phone sensing
Sensors on (or attached to) mobile phones can enable attractive sensing applications in different domains such as environmental monitoring, social networking, healthcare, etc. We introduce a new concept, Sensing-as-a-Service (S2aaS), i.e., providing sensing services using mobile phones via a cloud computing system. An S2aaS cloud should meet the following requirements:
1) It must be able to support various mobile phone sensing applications on different smartphone platforms.
2) It must be energy-efficient. 3) It must have effective incentive mechanisms that can be used to attract mobile users to participate in sensing activities. In this paper, we identify unique challenges of designing and implementing an S2aaS cloud, review existing systems and methods, present viable solutions, and point out future research directions
Review of performance of various Big Databases
Relational databases have been the main model for information data storage, retrieval and administration.A relational database is a table-based data system where there is no scalability, insignificant information duplication, computationally costly table joins and trouble in managing complex information. The greatest inspiration of NoSQL is adaptability. NoSQL information stores are broadly used to store and recover potentially a lot of information.In this paper, we assess four most famous NoSQL databases: Cassandra, MongoDB, and CouchDB
Big Data and the Internet of Things
Advances in sensing and computing capabilities are making it possible to
embed increasing computing power in small devices. This has enabled the sensing
devices not just to passively capture data at very high resolution but also to
take sophisticated actions in response. Combined with advances in
communication, this is resulting in an ecosystem of highly interconnected
devices referred to as the Internet of Things - IoT. In conjunction, the
advances in machine learning have allowed building models on this ever
increasing amounts of data. Consequently, devices all the way from heavy assets
such as aircraft engines to wearables such as health monitors can all now not
only generate massive amounts of data but can draw back on aggregate analytics
to "improve" their performance over time. Big data analytics has been
identified as a key enabler for the IoT. In this chapter, we discuss various
avenues of the IoT where big data analytics either is already making a
significant impact or is on the cusp of doing so. We also discuss social
implications and areas of concern.Comment: 33 pages. draft of upcoming book chapter in Japkowicz and Stefanowski
(eds.) Big Data Analysis: New algorithms for a new society, Springer Series
on Studies in Big Data, to appea
- …