1,341 research outputs found
Collaborative Reuse of Streaming Dataflows in IoT Applications
Distributed Stream Processing Systems (DSPS) like Apache Storm and Spark
Streaming enable composition of continuous dataflows that execute persistently
over data streams. They are used by Internet of Things (IoT) applications to
analyze sensor data from Smart City cyber-infrastructure, and make active
utility management decisions. As the ecosystem of such IoT applications that
leverage shared urban sensor streams continue to grow, applications will
perform duplicate pre-processing and analytics tasks. This offers the
opportunity to collaboratively reuse the outputs of overlapping dataflows,
thereby improving the resource efficiency. In this paper, we propose
\emph{dataflow reuse algorithms} that given a submitted dataflow, identifies
the intersection of reusable tasks and streams from a collection of running
dataflows to form a \emph{merged dataflow}. Similar algorithms to unmerge
dataflows when they are removed are also proposed. We implement these
algorithms for the popular Apache Storm DSPS, and validate their performance
and resource savings for 35 synthetic dataflows based on public OPMW workflows
with diverse arrival and departure distributions, and on 21 real IoT dataflows
from RIoTBench.Comment: To appear in IEEE eScience Conference 201
THE COLUMBUS GROUND SEGMENT โ A PRECURSOR FOR FUTURE MANNED MISSIONS
In the beginning the space programs were self standing national activities, often in competition to other nations. Today space flight becomes more and more an international task. Complex space mission and deep space explorations are not longer to be stemmed by one agency or nation alone but are joint activities of several nations. The best example for such a joint (ad-) venture at the moment is the International Space Station ISS.
Such international activities define complete new requirements for the supporting ground segments. The world-wide distribution of a ground segment is not any longer limited to a network of ground stations with the aim to provide a good coverage of the space craft. The coverage is sometimes โ like for the ISSanyway ensured by using a relay satellite system instead. In addition to the enhanced down- and uplink methods a ground segment is aimed to connect the different centres of competence of all participating agencies/nations.
From the space craft operations point of view such transnational ground segments are required to support distributed and shared operations in a predefined decision/commanding hierarchy. This has to be taken into account in the technical topology as well as for the operational set-up and teaming.
Last not least increases the duration of missions, which requires a certain flexibility of the ground segment and long-term maintenance strategies for the ground segment with a special emphasis on nonintrusive replacements. The Russian space station MIR has been in the orbit for about 15 years, the ISS is currently targeted for 2020, to be for over 20 years in space
Adaptive Energy-aware Scheduling of Dynamic Event Analytics across Edge and Cloud Resources
The growing deployment of sensors as part of Internet of Things (IoT) is
generating thousands of event streams. Complex Event Processing (CEP) queries
offer a useful paradigm for rapid decision-making over such data sources. While
often centralized in the Cloud, the deployment of capable edge devices on the
field motivates the need for cooperative event analytics that span Edge and
Cloud computing. Here, we identify a novel problem of query placement on edge
and Cloud resources for dynamically arriving and departing analytic dataflows.
We define this as an optimization problem to minimize the total makespan for
all event analytics, while meeting energy and compute constraints of the
resources. We propose 4 adaptive heuristics and 3 rebalancing strategies for
such dynamic dataflows, and validate them using detailed simulations for 100 -
1000 edge devices and VMs. The results show that our heuristics offer
O(seconds) planning time, give a valid and high quality solution in all cases,
and reduce the number of query migrations. Furthermore, rebalance strategies
when applied in these heuristics have significantly reduced the makespan by
around 20 - 25%.Comment: 11 pages, 7 figure
Knowledge visualizations: a tool to achieve optimized operational decision making and data integration
The overabundance of data created by modern information systems (IS) has led to a breakdown in cognitive decision-making. Without authoritative source data, commandersโ decision-making processes are hindered as they attempt to paint an accurate shared operational picture (SOP). Further impeding the decision-making process is the lack of proper interface interaction to provide a visualization that aids in the extraction of the most relevant and accurate data. Utilizing the DSS to present visualizations based on OLAP cube integrated data allow decision-makers to rapidly glean information and build their situation awareness (SA). This yields a competitive advantage to the organization while in garrison or in combat. Additionally, OLAP cube data integration enables analysis to be performed on an organizationโs data-flows. This analysis is used to identify the critical path of data throughout the organization. Linking a decision-maker to the authoritative data along this critical path eliminates the many decision layers in a hierarchal command structure that can introduce latency or error into the decision-making process. Furthermore, the organization has an integrated SOP from which to rapidly build SA, and make effective and efficient decisions.http://archive.org/details/knowledgevisuali1094545877Outstanding ThesisOutstanding ThesisMajor, United States Marine CorpsCaptain, United States Marine CorpsApproved for public release; distribution is unlimited
A Survey on the Evolution of Stream Processing Systems
Stream processing has been an active research field for more than 20 years,
but it is now witnessing its prime time due to recent successful efforts by the
research community and numerous worldwide open-source communities. This survey
provides a comprehensive overview of fundamental aspects of stream processing
systems and their evolution in the functional areas of out-of-order data
management, state management, fault tolerance, high availability, load
management, elasticity, and reconfiguration. We review noteworthy past research
findings, outline the similarities and differences between early ('00-'10) and
modern ('11-'18) streaming systems, and discuss recent trends and open
problems.Comment: 34 pages, 15 figures, 5 table
Towards Digital Twin-enabled DevOps for CPS providing Architecture-Based Service Adaptation & Verification at Runtime
Industrial Product-Service Systems (IPSS) denote a service-oriented (SO) way
of providing access to CPS capabilities. The design of such systems bears high
risk due to uncertainty in requirements related to service function and
behavior, operation environments, and evolving customer needs. Such risks and
uncertainties are well known in the IT sector, where DevOps principles ensure
continuous system improvement through reliable and frequent delivery processes.
A modular and SO system architecture complements these processes to facilitate
IT system adaptation and evolution. This work proposes a method to use and
extend the Digital Twins (DTs) of IPSS assets for enabling the continuous
optimization of CPS service delivery and the latter's adaptation to changing
needs and environments. This reduces uncertainty during design and operations
by assuring IPSS integrity and availability, especially for design and service
adaptations at CPS runtime. The method builds on transferring IT DevOps
principles to DT-enabled CPS IPSS. The chosen design approach integrates,
reuses, and aligns the DT processing and communication resources with DevOps
requirements derived from literature. We use these requirements to propose a
DT-enabled self-adaptive CPS model, which guides the realization of DT-enabled
DevOps in CPS IPSS. We further propose detailed design models for
operation-critical DTs that integrate CPS closed-loop control and
architecture-based CPS adaptation. This integrated approach enables the
implementation of A/B testing as a use case and central concept to enable CPS
IPSS service adaptation and reconfiguration. The self-adaptive CPS model and DT
design concept have been validated in an evaluation environment for
operation-critical CPS IPSS. The demonstrator achieved sub-millisecond cycle
times during service A/B testing at runtime without causing CPS operation
interferences and downtime.Comment: Final published version appearing in 17th Symposium on Software
Engineering for Adaptive and Self-Managing Systems (SEAMS 2022
ํ์ ๋ก๋ด์ ์ํ ์๋น์ค ๊ธฐ๋ฐ๊ณผ ๋ชจ๋ธ ๊ธฐ๋ฐ์ ์ํํธ์จ์ด ๊ฐ๋ฐ ๋ฐฉ๋ฒ๋ก
ํ์๋
ผ๋ฌธ(๋ฐ์ฌ)--์์ธ๋ํ๊ต ๋ํ์ :๊ณต๊ณผ๋ํ ์ ๊ธฐยท์ปดํจํฐ๊ณตํ๋ถ,2020. 2. ํ์ํ.๊ฐ๊น์ด ๋ฏธ๋์๋ ๋ค์ํ ๋ก๋ด์ด ๋ค์ํ ๋ถ์ผ์์ ํ๋์ ์๋ฌด๋ฅผ ํ๋ ฅํ์ฌ ์ํํ๋ ๋ชจ์ต์ ํํ ๋ณผ ์ ์๊ฒ ๋ ๊ฒ์ด๋ค. ๊ทธ๋ฌ๋ ์ค์ ๋ก ์ด๋ฌํ ๋ชจ์ต์ด ์คํ๋๊ธฐ์๋ ๋ ๊ฐ์ง์ ์ด๋ ค์์ด ์๋ค. ๋จผ์ ๋ก๋ด์ ์ด์ฉํ๊ธฐ ์ํ ์ํํธ์จ์ด๋ฅผ ๋ช
์ธํ๋ ๊ธฐ์กด ์ฐ๊ตฌ๋ค์ ๋๋ถ๋ถ ๊ฐ๋ฐ์๊ฐ ๋ก๋ด์ ํ๋์จ์ด์ ์ํํธ์จ์ด์ ๋ํ ์ง์์ ์๊ณ ์๋ ๊ฒ์ ๊ฐ์ ํ๊ณ ์๋ค. ๊ทธ๋์ ๋ก๋ด์ด๋ ์ปดํจํฐ์ ๋ํ ์ง์์ด ์๋ ์ฌ์ฉ์๋ค์ด ์ฌ๋ฌ ๋์ ๋ก๋ด์ด ํ๋ ฅํ๋ ์๋๋ฆฌ์ค๋ฅผ ์์ฑํ๊ธฐ๋ ์ฝ์ง ์๋ค. ๋ํ, ๋ก๋ด์ ์ํํธ์จ์ด๋ฅผ ๊ฐ๋ฐํ ๋ ๋ก๋ด์ ํ๋์จ์ด์ ํน์ฑ๊ณผ ๊ด๋ จ์ด ๊น์ด์, ๋ค์ํ ๋ก๋ด์ ์ํํธ์จ์ด๋ฅผ ๊ฐ๋ฐํ๋ ๊ฒ๋ ๊ฐ๋จํ์ง ์๋ค. ๋ณธ ๋
ผ๋ฌธ์์๋ ์์ ์์ค์ ๋ฏธ์
๋ช
์ธ์ ๋ก๋ด์ ํ์ ํ๋ก๊ทธ๋๋ฐ์ผ๋ก ๋๋์ด ์๋ก์ด ์ํํธ์จ์ด ๊ฐ๋ฐ ํ๋ ์์ํฌ๋ฅผ ์ ์ํ๋ค. ๋ํ, ๋ณธ ํ๋ ์์ํฌ๋ ํฌ๊ธฐ๊ฐ ์์ ๋ก๋ด๋ถํฐ ๊ณ์ฐ ๋ฅ๋ ฅ์ด ์ถฉ๋ถํ ๋ก๋ด๋ค์ด ์๋ก ๊ตฐ์ง์ ์ด๋ฃจ์ด ๋ฏธ์
์ ์ํํ ์ ์๋๋ก ์ง์ํ๋ค.
๋ณธ ์ฐ๊ตฌ์์๋ ๋ก๋ด์ ํ๋์จ์ด๋ ์ํํธ์จ์ด์ ๋ํ ์ง์์ด ๋ถ์กฑํ ์ฌ์ฉ์๋ ๋ก๋ด์ ๋์์ ์์ ์์ค์์ ๋ช
์ธํ ์ ์๋ ์คํฌ๋ฆฝํธ ์ธ์ด๋ฅผ ์ ์ํ๋ค. ์ ์ํ๋ ์ธ์ด๋ ๊ธฐ์กด์ ์คํฌ๋ฆฝํธ ์ธ์ด์์๋ ์ง์ํ์ง ์๋ ๋ค ๊ฐ์ง์ ๊ธฐ๋ฅ์ธ ํ์ ๊ตฌ์ฑ, ๊ฐ ํ์ ์๋น์ค ๊ธฐ๋ฐ ํ๋ก๊ทธ๋๋ฐ, ๋์ ์ผ๋ก ๋ชจ๋ ๋ณ๊ฒฝ, ๋ค์ค ์์
(๋ฉํฐ ํ์คํน)์ ์ง์ํ๋ค. ์ฐ์ ๋ก๋ด์ ํ์ผ๋ก ๊ทธ๋ฃน ์ง์ ์ ์๊ณ , ๋ก๋ด์ด ์ํํ ์ ์๋ ๊ธฐ๋ฅ์ ์๋น์ค ๋จ์๋ก ์ถ์ํํ์ฌ ์๋ก์ด ๋ณตํฉ ์๋น์ค๋ฅผ ๋ช
์ธํ ์ ์๋ค. ๋ํ ๋ก๋ด์ ๋ฉํฐ ํ์คํน์ ์ํด 'ํ๋' ์ด๋ผ๋ ๊ฐ๋
์ ๋์
ํ์๊ณ , ๋ณตํฉ ์๋น์ค ๋ด์์ ์ด๋ฒคํธ๋ฅผ ๋ฐ์์์ผ์ ๋์ ์ผ๋ก ๋ชจ๋๊ฐ ๋ณํํ ์ ์๋๋ก ํ์๋ค. ๋์๊ฐ ์ฌ๋ฌ ๋ก๋ด์ ํ๋ ฅ์ด ๋์ฑ ๊ฒฌ๊ณ ํ๊ณ , ์ ์ฐํ๊ณ , ํ์ฅ์ฑ์ ๋์ด๊ธฐ ์ํด, ๊ตฐ์ง ๋ก๋ด์ ์ด์ฉํ ๋ ๋ก๋ด์ด ์๋ฌด๋ฅผ ์ํํ๋ ๋์ค์ ๋ฌธ์ ๊ฐ ์๊ธธ ์ ์์ผ๋ฉฐ, ์ํฉ์ ๋ฐ๋ผ ๋ก๋ด์ ๋์ ์ผ๋ก ๋ค๋ฅธ ํ์๋ฅผ ์ํํ ์ ์๋ค๊ณ ๊ฐ์ ํ๋ค. ์ด๋ฅผ ์ํด ๋์ ์ผ๋ก๋ ํ์ ๊ตฌ์ฑํ ์ ์๊ณ , ์ฌ๋ฌ ๋์ ๋ก๋ด์ด ํ๋์ ์๋น์ค๋ฅผ ์ํํ๋ ๊ทธ๋ฃน ์๋น์ค๋ฅผ ์ง์ํ๊ณ , ์ผ๋ ๋ค ํต์ ๊ณผ ๊ฐ์ ์๋ก์ด ๊ธฐ๋ฅ์ ์คํฌ๋ฆฝํธ ์ธ์ด์ ๋ฐ์ํ์๋ค. ๋ฐ๋ผ์ ํ์ฅ๋ ์์ ์์ค์ ์คํฌ๋ฆฝํธ ์ธ์ด๋ ๋น์ ๋ฌธ๊ฐ๋ ๋ค์ํ ์ ํ์ ํ๋ ฅ ์๋ฌด๋ฅผ ์ฝ๊ฒ ๋ช
์ธํ ์ ์๋ค.
๋ก๋ด์ ํ์๋ฅผ ํ๋ก๊ทธ๋๋ฐํ๊ธฐ ์ํด ๋ค์ํ ์ํํธ์จ์ด ๊ฐ๋ฐ ํ๋ ์์ํฌ๊ฐ ์ฐ๊ตฌ๋๊ณ ์๋ค. ํนํ ์ฌ์ฌ์ฉ์ฑ๊ณผ ํ์ฅ์ฑ์ ์ค์ ์ผ๋ก ๋ ์ฐ๊ตฌ๋ค์ด ์ต๊ทผ ๋ง์ด ์ฌ์ฉ๋๊ณ ์์ง๋ง, ๋๋ถ๋ถ์ ์ด๋ค ์ฐ๊ตฌ๋ ๋ฆฌ๋
์ค ์ด์์ฒด์ ์ ๊ฐ์ด ๋ง์ ํ๋์จ์ด ์์์ ํ์๋ก ํ๋ ์ด์์ฒด์ ๋ฅผ ๊ฐ์ ํ๊ณ ์๋ค. ๋ํ, ํ๋ก๊ทธ๋จ์ ๋ถ์ ๋ฐ ์ฑ๋ฅ ์์ธก ๋ฑ์ ๊ณ ๋ คํ์ง ์๊ธฐ ๋๋ฌธ์, ์์ ์ ์ฝ์ด ์ฌํ ํฌ๊ธฐ๊ฐ ์์ ๋ก๋ด์ ์ํํธ์จ์ด๋ฅผ ๊ฐ๋ฐํ๊ธฐ์๋ ์ด๋ ต๋ค. ๊ทธ๋์ ๋ณธ ์ฐ๊ตฌ์์๋ ์๋ฒ ๋๋ ์ํํธ์จ์ด๋ฅผ ์ค๊ณํ ๋ ์ฐ์ด๋ ์ ํ์ ์ธ ๋ชจ๋ธ์ ์ด์ฉํ๋ค. ์ด ๋ชจ๋ธ์ ์ ์ ๋ถ์๊ณผ ์ฑ๋ฅ ์์ธก์ด ๊ฐ๋ฅํ์ง๋ง, ๋ก๋ด์ ํ์๋ฅผ ํํํ๊ธฐ์๋ ์ ์ฝ์ด ์๋ค. ๊ทธ๋์ ๋ณธ ๋
ผ๋ฌธ์์ ์ธ๋ถ์ ์ด๋ฒคํธ์ ์ํด ์ํ ์ค๊ฐ์ ํ์๋ฅผ ๋ณ๊ฒฝํ๋ ๋ก๋ด์ ์ํด ์ ํ ์ํ ๋จธ์ ๋ชจ๋ธ๊ณผ ๋ฐ์ดํฐ ํ๋ก์ฐ ๋ชจ๋ธ์ด ๊ฒฐํฉํ์ฌ ๋์ ํ์๋ฅผ ๋ช
์ธํ ์ ์๋ ํ์ฅ๋ ๋ชจ๋ธ์ ์ ์ฉํ๋ค. ๊ทธ๋ฆฌ๊ณ ๋ฅ๋ฌ๋๊ณผ ๊ฐ์ด ๊ณ์ฐ๋์ ๋ง์ด ํ์๋ก ํ๋ ์์ฉ์ ๋ถ์ํ๊ธฐ ์ํด, ๋ฃจํ ๊ตฌ์กฐ๋ฅผ ๋ช
์์ ์ผ๋ก ํํํ ์ ์๋ ๋ชจ๋ธ์ ์ ์ํ๋ค. ๋ง์ง๋ง์ผ๋ก ์ฌ๋ฌ ๋ก๋ด์ ํ์
์ด์ฉ์ ์ํด ๋ก๋ด ์ฌ์ด์ ๊ณต์ ๋๋ ์ ๋ณด๋ฅผ ๋ํ๋ด๊ธฐ ์ํด ๋ ๊ฐ์ง ๋ชจ๋ธ์ ์ฌ์ฉํ๋ค. ๋จผ์ ์ค์์์ ๊ณต์ ์ ๋ณด๋ฅผ ๊ด๋ฆฌํ๊ธฐ ์ํด ๋ผ์ด๋ธ๋ฌ๋ฆฌ ํ์คํฌ๋ผ๋ ํน๋ณํ ํ์คํฌ๋ฅผ ํตํด ๊ณต์ ์ ๋ณด๋ฅผ ๋ํ๋ธ๋ค. ๋ํ, ๋ก๋ด์ด ์์ ์ ์ ๋ณด๋ฅผ ๊ฐ๊น์ด ๋ก๋ด๋ค๊ณผ ๊ณต์ ํ๊ธฐ ์ํด ๋ฉํฐ์บ์คํ
์ ์ํ ์๋ก์ด ํฌํธ๋ฅผ ์ถ๊ฐํ๋ค. ์ด๋ ๊ฒ ํ์ฅ๋ ์ ํ์ ์ธ ๋ชจ๋ธ์ ์ค์ ๋ก๋ด ์ฝ๋๋ก ์๋ ์์ฑ๋์ด, ์ํํธ์จ์ด ์ค๊ณ ์์ฐ์ฑ ๋ฐ ๊ฐ๋ฐ ํจ์จ์ฑ์ ์ด์ ์ ๊ฐ์ง๋ค.
๋น์ ๋ฌธ๊ฐ๊ฐ ๋ช
์ธํ ์คํฌ๋ฆฝํธ ์ธ์ด๋ ์ ํ์ ์ธ ํ์คํฌ ๋ชจ๋ธ๋ก ๋ณํํ๊ธฐ ์ํด ์ค๊ฐ ๋จ๊ณ์ธ ์ ๋ต ๋จ๊ณ๋ฅผ ์ถ๊ฐํ์๋ค. ์ ์ํ๋ ๋ฐฉ๋ฒ๋ก ์ ํ๋น์ฑ์ ๊ฒ์ฆํ๊ธฐ ์ํด, ์๋ฎฌ๋ ์ด์
๊ณผ ์ฌ๋ฌ ๋์ ์ค์ ๋ก๋ด์ ์ด์ฉํ ํ์
ํ๋ ์๋๋ฆฌ์ค์ ๋ํด ์คํ์ ์งํํ์๋ค.In the near future, it will be common that a variety of robots are cooperating to perform a mission in various fields. There are two software challenges when deploying collaborative robots: how to specify a cooperative mission and how to program each robot to accomplish its mission. In this paper, we propose a novel software development framework that separates mission specification and robot behavior programming, which is called service-oriented and model-based (SeMo) framework. Also, it can support distributed robot systems, swarm robots, and their hybrid.
For mission specification, a novel scripting language is proposed with the expression capability. It involves team composition and service-oriented behavior specification of each team, allowing dynamic mode change of operation and multi-tasking. Robots are grouped into teams, and the behavior of each team is defined with a composite service. The internal behavior of a composite service is defined by a sequence of services that the robots will perform. The notion of plan is applied to express multi-tasking. And the robot may have various operating modes, so mode change is triggered by events generated in a composite service. Moreover, to improve the robustness, scalability, and flexibility of robot collaboration, the high-level mission scripting language is extended with new features such as team hierarchy, group service, one-to-many communication. We assume that any robot fails during the execution of scenarios, and the grouping of robots can be made at run-time dynamically. Therefore, the extended mission specification enables a casual user to specify various types of cooperative missions easily.
For robot behavior programming, an extended dataflow model is used for task-level behavior specification that does not depend on the robot hardware platform. To specify the dynamic behavior of the robot, we apply an extended task model that supports a hybrid specification of dataflow and finite state machine models. Furthermore, we propose a novel extension to allow the explicit specification of loop structures. This extension helps the compute-intensive application, which contains a lot of loop structures, to specify explicitly and analyze at compile time. Two types of information sharing, global information sharing and local knowledge sharing, are supported for robot collaboration in the dataflow graph. For global information, we use the library task, which supports shared resource management and server-client interaction. On the other hand, to share information locally with near robots, we add another type of port for multicasting and use the knowledge sharing technique. The actual robot code per robot is automatically generated from the associated task graph, which minimizes the human efforts in low-level robot programming and improves the software design productivity significantly.
By abstracting the tasks or algorithms as services and adding the strategy description layer in the design flow, the mission specification is refined into task-graph specification automatically. The viability of the proposed methodology is verified with preliminary experiments with three cooperative mission scenarios with heterogeneous robot platforms and robot simulator.Chapter 1. Introduction 1
1.1 Motivation 1
1.2 Contribution 7
1.3 Dissertation Organization 9
Chapter 2. Background and Existing Research 11
2.1 Terminologies 11
2.2 Robot Software Development Frameworks 25
2.3 Parallel Embedded Software Development Framework 31
Chapter 3. Overview of the SeMo Framework 41
3.1 Motivational Examples 45
Chapter 4. Robot Behavior Programming 47
4.1 Related works 48
4.2 Model-based Task Graph Specification for Individual Robots 56
4.3 Model-based Task Graph Specification for Cooperating Robots 70
4.4 Automatic Code Generation 74
4.5 Experiments 78
Chapter 5. High-level Mission Specification 81
5.1 Service-oriented Mission Specification 82
5.2 Strategy Description 93
5.3 Automatic Task Graph Generation 96
5.4 Related works 99
5.5 Experiments 104
Chapter 6. Conclusion 114
6.1 Future Research 116
Bibliography 118
Appendices 133
์์ฝ 158Docto
- โฆ