748 research outputs found
Reliable Messaging to Millions of Users with MigratoryData
Web-based notification services are used by a large range of businesses to
selectively distribute live updates to customers, following the
publish/subscribe (pub/sub) model. Typical deployments can involve millions of
subscribers expecting ordering and delivery guarantees together with low
latencies. Notification services must be vertically and horizontally scalable,
and adopt replication to provide a reliable service. We report our experience
building and operating MigratoryData, a highly-scalable notification service.
We discuss the typical requirements of MigratoryData customers, and describe
the architecture and design of the service, focusing on scalability and fault
tolerance. Our evaluation demonstrates the ability of MigratoryData to handle
millions of concurrent connections and support a reliable notification service
despite server failures and network disconnections
Impliance: A Next Generation Information Management Appliance
ably successful in building a large market and adapting to the changes of the
last three decades, its impact on the broader market of information management
is surprisingly limited. If we were to design an information management system
from scratch, based upon today's requirements and hardware capabilities, would
it look anything like today's database systems?" In this paper, we introduce
Impliance, a next-generation information management system consisting of
hardware and software components integrated to form an easy-to-administer
appliance that can store, retrieve, and analyze all types of structured,
semi-structured, and unstructured information. We first summarize the trends
that will shape information management for the foreseeable future. Those trends
imply three major requirements for Impliance: (1) to be able to store, manage,
and uniformly query all data, not just structured records; (2) to be able to
scale out as the volume of this data grows; and (3) to be simple and robust in
operation. We then describe four key ideas that are uniquely combined in
Impliance to address these requirements, namely the ideas of: (a) integrating
software and off-the-shelf hardware into a generic information appliance; (b)
automatically discovering, organizing, and managing all data - unstructured as
well as structured - in a uniform way; (c) achieving scale-out by exploiting
simple, massive parallel processing, and (d) virtualizing compute and storage
resources to unify, simplify, and streamline the management of Impliance.
Impliance is an ambitious, long-term effort to define simpler, more robust, and
more scalable information systems for tomorrow's enterprises.Comment: This article is published under a Creative Commons License Agreement
(http://creativecommons.org/licenses/by/2.5/.) You may copy, distribute,
display, and perform the work, make derivative works and make commercial use
of the work, but, you must attribute the work to the author and CIDR 2007.
3rd Biennial Conference on Innovative Data Systems Research (CIDR) January
710, 2007, Asilomar, California, US
Introducing the new paradigm of Social Dispersed Computing: Applications, Technologies and Challenges
[EN] If last decade viewed computational services as a utility then surely
this decade has transformed computation into a commodity. Computation
is now progressively integrated into the physical networks in
a seamless way that enables cyber-physical systems (CPS) and the
Internet of Things (IoT) meet their latency requirements. Similar to
the concept of ¿platform as a service¿ or ¿software as a service¿, both
cloudlets and fog computing have found their own use cases. Edge
devices (that we call end or user devices for disambiguation) play the
role of personal computers, dedicated to a user and to a set of correlated
applications. In this new scenario, the boundaries between
the network node, the sensor, and the actuator are blurring, driven
primarily by the computation power of IoT nodes like single board
computers and the smartphones. The bigger data generated in this
type of networks needs clever, scalable, and possibly decentralized
computing solutions that can scale independently as required. Any
node can be seen as part of a graph, with the capacity to serve as a
computing or network router node, or both. Complex applications can
possibly be distributed over this graph or network of nodes to improve
the overall performance like the amount of data processed over time.
In this paper, we identify this new computing paradigm that we call
Social Dispersed Computing, analyzing key themes in it that includes
a new outlook on its relation to agent based applications. We architect
this new paradigm by providing supportive application examples that
include next generation electrical energy distribution networks, next
generation mobility services for transportation, and applications for
distributed analysis and identification of non-recurring traffic congestion
in cities. The paper analyzes the existing computing paradigms
(e.g., cloud, fog, edge, mobile edge, social, etc.), solving the ambiguity
of their definitions; and analyzes and discusses the relevant foundational
software technologies, the remaining challenges, and research
opportunities.Garcia Valls, MS.; Dubey, A.; Botti, V. (2018). Introducing the new paradigm of Social Dispersed Computing: Applications, Technologies and Challenges. Journal of Systems Architecture. 91:83-102. https://doi.org/10.1016/j.sysarc.2018.05.007S831029
Software Defined Application Delivery Networking
In this thesis we present the architecture, design, and prototype implementation details of AppFabric. AppFabric is a next generation application delivery platform for easily creating, managing and controlling massively distributed and very dynamic application deployments that may span multiple datacenters.
Over the last few years, the need for more flexibility, finer control, and automatic management of large (and messy) datacenters has stimulated technologies for virtualizing the infrastructure components and placing them under software-based management and control; generically called Software-defined Infrastructure (SDI). However, current applications are not designed to leverage this dynamism and flexibility offered by SDI and they mostly depend on a mix of different techniques including manual configuration, specialized appliances (middleboxes), and (mostly) proprietary middleware solutions together with a team of extremely conscientious and talented system engineers to get their applications deployed and running. AppFabric, 1) automates the whole control and management stack of application deployment and delivery, 2) allows application architects to define logical workflows consisting of application servers, message-level middleboxes, packet-level middleboxes and network services (both, local and wide-area) composed over application-level routing policies, and 3) provides the abstraction of an application cloud that allows the application to dynamically (and automatically) expand and shrink its distributed footprint across multiple geographically distributed datacenters operated by different cloud providers. The architecture consists of a hierarchical control plane system called Lighthouse and a fully distributed data plane design (with no special hardware components such as service orchestrators, load balancers, message brokers, etc.) called OpenADN . The current implementation (under active development) consists of ~10000 lines of python and C code.
AppFabric will allow applications to fully leverage the opportunities provided by modern virtualized Software-Defined Infrastructures. It will serve as the platform for deploying massively distributed, and extremely dynamic next generation application use-cases, including:
Internet-of-Things/Cyber-Physical Systems: Through support for managing distributed gather-aggregate topologies common to most Internet-of-Things(IoT) and Cyber-Physical Systems(CPS) use-cases. By their very nature, IoT and CPS use cases are massively distributed and have different levels of computation and storage requirements at different locations. Also, they have variable latency requirements for their different distributed sites. Some services, such as device controllers, in an Iot/CPS application workflow may need to gather, process and forward data under near-real time constraints and hence need to be as close to the device as possible. Other services may need more computation to process aggregated data to drive long term business intelligence functions. AppFabric has been designed to provide support for such very dynamic, highly diversified and massively distributed application use-cases.
Network Function Virtualization: Through support for heterogeneous workflows, application-aware networking, and network-aware application deployments, AppFabric will enable new partnerships between Application Service Providers (ASPs) and Network Service Providers (NSPs). An application workflow in AppFabric may comprise of application services, packet and message-level middleboxes, and network transport services chained together over an application-level routing substrate. The Application-level routing substrate allows policy-based service chaining where the application may specify policies for routing their application traffic over different services based on application-level content or context.
Virtual worlds/multiplayer games: Through support for creating, managing and controlling dynamic and distributed application clouds needed by these applications. AppFabric allows the application to easily specify policies to dynamically grow and shrink the application\u27s footprint over different geographical sites, on-demand.
Mobile Apps: Through support for extremely diversified and very dynamic application contexts typical of such applications. Also, AppFabric provides support for automatically managing massively distributed service deployment and controlling application traffic based on application-level policies. This allows mobile applications to provide the best Quality-of-Experience to its users without
This thesis is the first to handle and provide a complete solution for such a complex and relevant architectural problem that is expected to touch each of our lives by enabling exciting new application use-cases that are not possible today. Also, AppFabric is a non-proprietary platform that is expected to spawn lots of innovations both in the design of the platform itself and the features it provides to applications. AppFabric still needs many iterations, both in terms of design and implementation maturity. This thesis is not the end of journey for AppFabric but rather just the beginning
Managing IT Operations in a Cloud-driven Enterprise: Case Studies
Enterprise IT needs a new approach to manage processes, applications and infrastructure which are distributed across a mix of environments. In an Enterprise traditionally a request to deliver an application to business could take weeks or months due to decision-making functions, multiple approval bodies and processes that exist within IT departments. These delays in delivering a requested service can lead to dissatisfaction, with the result that the line-of-business group may seek alternative sources of IT capabilities. Also the complex IT infrastructure of these enterprises cannot keep up with the demand of new applications and services from an increasingly dispersed and mobile workforce which results in slower rollout of critical applications and services, limited resources, poor operation visibility and control. In such scenarios, it’s better to adopt cloud services to substitute for new application deployment otherwise most Enterprise IT organizations face the risk of losing 'market share' to the Public Cloud. Using Cloud Model the organizations should increase ROI, lower TCO and operate with seamless IT operations. It also helps to beat shadow IT and the practice of resource over-or under provisioning. In this research paper we have given two case studies where we migrated two Enterprise IT application to public clouds for the purpose of lower TCO and higher ROI. By migrating, the IT organizations improved IT agility, enterprise-class software for performance, security and control. In this paper, we also focus on the advantages and challenges while adopting cloud services
- …