257 research outputs found
The ANTARES Astronomical Time-Domain Event Broker
We describe the Arizona-NOIRLab Temporal Analysis and Response to Events
System (ANTARES), a software instrument designed to process large-scale streams
of astronomical time-domain alerts. With the advent of large-format CCDs on
wide-field imaging telescopes, time-domain surveys now routinely discover tens
of thousands of new events each night, more than can be evaluated by
astronomers alone. The ANTARES event broker will process alerts, annotating
them with catalog associations and filtering them to distinguish customizable
subsets of events. We describe the data model of the system, the overall
architecture, annotation, implementation of filters, system outputs, provenance
tracking, system performance, and the user interface.Comment: 24 Pages, 8 figures, Accepted by A
Intelligent event broker: a complex event processing system in big data contexts
In Big Data contexts, many batch and streaming oriented technologies have emerged to deal with the high valuable sources of events, such as Internet of Things (IoT) platforms, the Web, several types of databases, among others. The huge amount of heterogeneous data being constantly generated by a world of interconnected things and the need for (semi)-automated decision-making processes through Complex Event Processing (CEP) and Machine Learning (ML) have raised the need for innovative architectures capable of processing events in a streamlined, scalable, analytical, and integrated way. This paper presents the Intelligent Event Broker, a CEP system built upon flexible and scalable Big Data techniques and technologies, highlighting its system architecture, software packages, and classes. A demonstration case in Bosch’s Industry 4.0 context is presented, detailing how the system can be used to manage and improve the quality of the manufacturing process, showing its usefulness for solving real-world event-oriented problems.This work has been supported by FCT –Fundação para a Ciência e Tecnologiawithin the Project Scope: UID/CEC/00319/2019 and the Doctoral scholarship PD/BDE/135101/2017. This paper uses icons made by Freepik, from www.flaticon.com
SafeWeb: A Middleware for Securing Ruby-Based Web Applications
Web applications in many domains such as healthcare and finance must process sensitive data, while complying with legal policies regarding the release of different classes of data to different parties. Currently, software bugs may lead to irreversible disclosure of confidential data in multi-tier web applications. An open challenge is how developers can guarantee these web applications only ever release sensitive data to authorised users without costly, recurring security audits.
Our solution is to provide a trusted middleware that acts as a “safety net” to event-based enterprise web applications by preventing harmful data disclosure before it happens. We describe the design and implementation of SafeWeb, a Ruby-based middleware that associates data with security labels and transparently tracks their propagation at different granularities across a multi-tier web architecture with storage and complex event processing. For efficiency, maintainability and ease-of-use, SafeWeb exploits the dynamic features of the Ruby programming language to achieve label propagation and data flow enforcement. We evaluate SafeWeb by reporting our experience of implementing a web-based cancer treatment application and deploying it as part of the UK National Health Service (NHS)
Evaluation of standard monitoring tools(including log analysis) for control systems at Cern
Project Specification:
The goal of this Openlab Summer Student project was to assess the implications and the benefits of integrating two standard IT tools, namely Icinga and Splunkstorm with the existing production setup for monitoring and management of control systems at CERN.
Icinga – an open source monitoring software based on Nagios would need to be integrated with an in-house developed WinCC OA application called MOON, that is currently used for monitoring and managing all the components that make up the control systems.
Splunkstorm – a data analysis and log management online application would be used stand alone, so it didn’t need integration with other software, only understanding of features and installation procedure.
Abstract:
The aim of this document is to provide insights into installation procedures, key features and functionality and projected implementation effort of Icinga and Splunkstorm IT tools. Focus will be on presenting the most feasible implementation paths that surfaced once both software were well understood
Autonomous software: Myth or magic?
We discuss work by the eSTAR project which demonstrates a fully closed loop
autonomous system for the follow up of possible micro-lensing anomalies. Not
only are the initial micro-lensing detections followed up in real time, but
ongoing events are prioritised and continually monitored, with the returned
data being analysed automatically. If the ``smart software'' running the
observing campaign detects a planet-like anomaly, further follow-up will be
scheduled autonomously and other telescopes and telescope networks alerted to
the possible planetary detection. We further discuss the implications of this,
and how such projects can be used to build more general autonomous observing
and control systems.Comment: 3 pages, 4 figures, to appear in proceedings of Hot-wiring the
Transient Universe (HTU) 2007, Astronomische Nachrichten, March 200
A Big Data perspective on Cyber-Physical Systems for Industry 4.0: modernizing and scaling complex event processing
Doctoral program in Advanced Engineering Systems for IndustryNowadays, the whole industry makes efforts to find the most productive ways of working and it already
understood that using the data that is being produced inside and outside the factories is a way to improve
the business performance. A set of modern technologies combined with sensor-based communication
create the possibility to act according to our needs, precisely at the moment when the data is being
produced and processed. Considering the diversity of processes existing in a factory, all of them producing
data, Complex Event Processing (CEP) with the capabilities to process that amount of data is needed in
the daily work of a factory, to process different types of events and find patterns between them. Although
the integration of the Big Data and Complex Event Processing topics is already present in the literature,
open challenges in this area were identified, hence the reason for the contribution presented in this thesis.
Thereby, this doctoral thesis proposes a system architecture that integrates the CEP concept with a rulebased
approach in the Big Data context: the Intelligent Event Broker (IEB). This architecture proposes the
use of adequate Big Data technologies in its several components. At the same time, some of the gaps
identified in this area were fulfilled, complementing Event Processing with the possibility to use Machine
Learning Models that can be integrated in the rules' verification, and also proposing an innovative
monitoring system with an immersive visualization component to monitor the IEB and prevent its
uncontrolled growth, since there are always several processes inside a factory that can be integrated in
the system. The proposed architecture was validated with a demonstration case using, as an example,
the Active Lot Release Bosch's system. This demonstration case revealed that it is feasible to implement
the proposed architecture and proved the adequate functioning of the IEB system to process Bosch's
business processes data and also to monitor its components and the events flowing through those
components.Hoje em dia as indústrias esforçam-se para encontrar formas de serem mais produtivas. A utilização dos
dados que são produzidos dentro e fora das fábricas já foi identificada como uma forma de melhorar o
desempenho do negócio. Um conjunto de tecnologias atuais combinado com a comunicação baseada
em sensores cria a possibilidade de se atuar precisamente no momento em que os dados estão a ser
produzidos e processados, assegurando resposta às necessidades do negócio. Considerando a
diversidade de processos que existem e produzem dados numa fábrica, as capacidades do
Processamento de Eventos Complexos (CEP) revelam-se necessárias no quotidiano de uma fábrica,
processando diferentes tipos de eventos e encontrando padrões entre os mesmos. Apesar da integração
do conceito CEP na era de Big Data ser um tópico já presente na literatura, existem ainda desafios nesta
área que foram identificados e que dão origem às contribuições presentes nesta tese. Assim, esta tese
de doutoramento propõe uma arquitetura para um sistema que integre o conceito de CEP na era do Big
Data, seguindo uma abordagem baseada em regras: o Intelligent Event Broker (IEB). Esta arquitetura
propõe a utilização de tecnologias de Big Data que sejam adequadas aos seus diversos componentes.
As lacunas identificadas na literatura foram consideradas, complementando o processamento de eventos
com a possibilidade de utilizar modelos de Machine Learning com vista a serem integrados na verificação
das regras, propondo também um sistema de monitorização inovador composto por um componente de
visualização imersiva que permite monitorizar o IEB e prevenir o seu crescimento descontrolado, o que
pode acontecer devido à integração do conjunto significativo de processos existentes numa fábrica. A
arquitetura proposta foi validada através de um caso de demonstração que usou os dados do Active Lot
Release, um sistema da Bosch. Os resultados revelaram a viabilidade da implementação da arquitetura
e comprovaram o adequado funcionamento do sistema no que diz respeito ao processamento dos dados
dos processos de negócio da Bosch e à monitorização dos componentes do IEB e eventos que fluem
através desses.This work has been supported by FCT – Fundação para a Ciência e Tecnologia within the R&D Units
Project Scope: UIDB/00319/2020, the Doctoral scholarship PD/BDE/135101/2017 and by European
Structural and Investment Funds in the FEDER component, through the Operational Competitiveness and
Internationalization Programme (COMPETE 2020) [Project nº 039479; Funding Reference: POCI-01-
0247-FEDER-039479]
- …