A study of System Interface Sets (SIS) for the host, target and integration environments of the Space Station Program (SSP)

System interface sets (SIS) for large, complex, non-stop, distributed systems are examined. The SIS of the Space Station Program (SSP) was selected as the focus of this study because an appropriate virtual interface specification of the SIS is believed to have the most potential to free the project from four life cycle tyrannies which are rooted in a dependance on either a proprietary or particular instance of: operating systems, data management systems, communications systems, and instruction set architectures. The static perspective of the common Ada programming support environment interface set (CAIS) and the portable common execution environment (PCEE) activities are discussed. Also, the dynamic perspective of the PCEE is addressed

Modernization of a legacy system:event streaming with Apache Kafka and Spring Boot

Abstract. In this thesis, we will design, implement, and evaluate a brand new replacement, the Watcher, for a legacy system built over two decades ago. The Watcher is able to track changes in our PDM system, and notify users of the changes by email or as a push notifcation using SSE. Functional requirements for the new system come from the legacy system including the possibility to create subscriptions with a wide range of options to flter out redundant data traffc. The Watcher will also be able to carry out all operations than the predecessor with increased performance and effciency. The main focus is on scalability, maintainability, and fault tolerance. The reason for building a new system is mainly the cost of maintainability and further development of the legacy system as well as features removed due to obsolete technologies. In the literature review, we go through the theory of the technologies related to the project. We create a REST API with Spring Boot for interactions between users and the system, implement powerful event streaming and processing environment using Apache Kafka, and build a message service responsible for providing information via scheduled emails or SSE. In the end, we will use Docker to containerize all the services. In the project design, we present functional as well as technical requirements that we use later on to evaluate the project’s success. We also compare the legacy system to the new one using metrics such as speed and ease of the installation process. In the end, we discuss the project’s future including steps before going to production such as automatic testing, and further development for years to come such as orchestration.Legacy-palvelun uudistaminen : reaaliaikajärjestelmä Apache Kafkaa ja Spring Bootia hyödyntäen. Tiivistelmä. Tässä työssä suunnittelemme, toteutamme ja arvioimme uuden järjestelmän, joka tulee korvaamaan yli kaksi vuosikymmentä sitten luodun legacy-järjestelmän. Tämä uusi järjestelmä, "the Watcher", kykenee seuraamaan muutoksia meidän PDM järjestelmässämme, ja ilmoittamaan muutoksista käyttäjille sähköpostilla, sekä push-ilmoituksilla. Hyödynnämme tässä työssä toiminnallisia vaatimuksia, jotka ovat määritelty jo vanhalle järjestelmälle. Esimerkiksi tilausten luominen käyttäen useita suodattimia vähentäen samalla tarpeetonta dataliikennettä. "The Watcher"kykenee suoriutumaan kaikista tehtävistä joista vanha järjestelmäkin, sekä lisäominaisuutena se tarjoaa paremman suorituskyvyn, sekä tehokkuuden. Pääpainona järjestelmässä on skaalautuvuus, ylläpidettävyys ja vikasietoisuus. Kirjallisuuskatsauksessa käymme läpi projektiin liittyvien teknologioiden teorian. Toteutamme Spring Boot ohjelmointikehyksen avulla REST-rajapinnan, jonka välityksellä käyttäjät voivat kommunikoida järjestelmän kanssa. Rakennamme myös tehokkaan ympäristön datan käsittelyyn ja reaaliaikaiseen viestintään käyttäen Apache Kafkaa. Viimeiseksi luomme viestipalvelun, joka vastaa käyttäjien informoimisesta hyödyntäen SSE:ksi kutsuttua teknologiaa, sekä lähettämällä sähköpostiviestejä käyttäjien toivomana ajankohtana. Lopuksi vielä sijoitamme kaikki palvelut kontteihin Dockerin avulla. Projektin suunnitteluosiossa esittelemme niin toiminnalliset, kuin teknisetkin vaatimukset, joiden avulla arvioimme myöhemmin projektin onnistumista. Vertaamme myös vanhaa ja uutta järjestelmää käyttäen metriikoita kuten nopeus ja asennusprosessin yksinkertaisuus. Lopussa keskustelemme projektin tulevaisuudesta sisältäen vaiheet jotka tulisi suorittaa ennen kuin järjestelmä voidaan ottaa tuotantokäyttöön kuten automaattinen testaus, sekä toiminnallisuuksien kehitys tulevina vuosina

The ALPS project release 2.0: Open source software for strongly correlated systems

We present release 2.0 of the ALPS (Algorithms and Libraries for Physics Simulations) project, an open source software project to develop libraries and application programs for the simulation of strongly correlated quantum lattice models such as quantum magnets, lattice bosons, and strongly correlated fermion systems. The code development is centered on common XML and HDF5 data formats, libraries to simplify and speed up code development, common evaluation and plotting tools, and simulation programs. The programs enable non-experts to start carrying out serial or parallel numerical simulations by providing basic implementations of the important algorithms for quantum lattice models: classical and quantum Monte Carlo (QMC) using non-local updates, extended ensemble simulations, exact and full diagonalization (ED), the density matrix renormalization group (DMRG) both in a static version and a dynamic time-evolving block decimation (TEBD) code, and quantum Monte Carlo solvers for dynamical mean field theory (DMFT). The ALPS libraries provide a powerful framework for programers to develop their own applications, which, for instance, greatly simplify the steps of porting a serial code onto a parallel, distributed memory machine. Major changes in release 2.0 include the use of HDF5 for binary data, evaluation tools in Python, support for the Windows operating system, the use of CMake as build system and binary installation packages for Mac OS X and Windows, and integration with the VisTrails workflow provenance tool. The software is available from our web server at http://alps.comp-phys.org/.Comment: 18 pages + 4 appendices, 7 figures, 12 code examples, 2 table

Consensus clustering with differential evolution

summary:Consensus clustering algorithms are used to improve properties of traditional clustering methods, especially their accuracy and robustness. In this article, we introduce our approach that is based on a refinement of the set of initial partitions and uses differential evolution algorithm in order to find the most valid solution. Properties of the algorithm are demonstrated on four benchmark datasets

Privacy-Enhanced Query Processing in a Cloud-Based Encrypted DBaaS (Database as a Service)

In this dissertation, we researched techniques to support trustable and privacy enhanced solutions for on-line applications accessing to “always encrypted” data in remote DBaaS (data-base-as-a-service) or Cloud SQL-enabled backend solutions. Although solutions for SQL-querying of encrypted databases have been proposed in recent research, they fail in providing: (i) flexible multimodal query facilities includ ing online image searching and retrieval as extended queries to conventional SQL-based searches, (ii) searchable cryptographic constructions for image-indexing, searching and retrieving operations, (iii) reusable client-appliances for transparent integration of multi modal applications, and (iv) lack of performance and effectiveness validations for Cloud based DBaaS integrated deployments. At the same time, the study of partial homomorphic encryption and multimodal searchable encryption constructions is yet an ongoing research field. In this research direction, the need for a study and practical evaluations of such cryptographic is essential, to evaluate those cryptographic methods and techniques towards the materialization of effective solutions for practical applications. The objective of the dissertation is to design, implement and perform experimental evaluation of a security middleware solution, implementing a client/client-proxy/server appliance software architecture, to support the execution of applications requiring on line multimodal queries on “always encrypted” data maintained in outsourced cloud DBaaS backends. In this objective we include the support for SQL-based text-queries enhanced with searchable encrypted image-retrieval capabilities. We implemented a prototype of the proposed solution and we conducted an experimental benchmarking evaluation, to observe the effectiveness, latency and performance conditions in support ing those queries. The dissertation addressed the envisaged security middleware solution, as an experimental and usable solution that can be extended for future experimental testbench evaluations using different real cloud DBaaS deployments, as offered by well known cloud-providers.Nesta dissertação foram investigadas técnicas para suportar soluções com garantias de privacidade para aplicações que acedem on-line a dados que são mantidos sempre cifrados em nuvens que disponibilizam serviços de armazenamento de dados, nomeadamente soluções do tipo bases de dados interrogáveis por SQL. Embora soluções para suportar interrogações SQL em bases de dados cifradas tenham sido propostas anteriormente, estas falham em providenciar: (i) capacidade de efectuar pesquisas multimodais que possam incluir pesquisa combinada de texto e imagem com obtenção de imagens online, (ii) suporte de privacidade com base em construções criptograficas que permitam operações de indexacao, pesquisa e obtenção de imagens como dados cifrados pesquisáveis, (iii) suporte de integração para aplicações de gestão de dados em contexto multimodal, e (iv) ausência de validações experimentais com benchmarking dobre desempenho e eficiência em soluções DBaaS em que os dados sejam armazenados e manipulados na sua forma cifrada. A pesquisa de soluções de privacidade baseada em primitivas de cifras homomórficas parciais, tem sido vista como uma possível solução prática para interrogação de dados e bases de dados cifradas. No entanto, este é ainda um campo de investigação em desenvolvimento. Nesta direção de investigação, a necessidade de estudar e efectuar avaliações experimentais destas primitivas em bibliotecas de cifras homomórficas, reutilizáveis em diferentes contextos de aplicação e como solução efetiva para uso prático mais generalizado, é um aspeto essencial. O objectivo da dissertação e desenhar, implementar e efectuar avalições experimentais de uma proposta de solução middleware para suportar pesquisas multimodais em bases de dados mantidas cifradas em soluções de nuvens de armazenamento. Esta proposta visa a concepção e implementação de uma arquitectura de software client/client-proxy/server appliance para suportar execução eficiente de interrogações online sobre dados cifrados, suportando operações multimodais sobre dados mantidos protegidos em serviços de nuvens de armazenamento. Neste objectivo incluímos o suporte para interrogações estendidas de SQL, com capacidade para pesquisa e obtenção de dados cifrados que podem incluir texto e pesquisa de imagens por similaridade. Foi implementado um prototipo da solução proposta e foi efectuada uma avaliação experimental do mesmo, para observar as condições de eficiencia, latencia e desempenho do suporte dessas interrogações. Nesta avaliação incluímos a análise experimental da eficiência e impacto de diferentes construções criptográficas para pesquisas cifradas (searchable encryption) e cifras parcialmente homomórficas e que são usadas como componentes da solução proposta. A dissertaçao aborda a soluçao de seguranca projectada, como uma solução experimental que pode ser estendida e utilizavel para futuras aplcações e respetivas avaliações experimentais. Estas podem vir a adoptar soluções do tipo DBaaS, oferecidos como serviços na nuvem, por parte de diversos provedores ou fornecedores

Discovery of Web Attacks by Inspecting HTTPS Network Traffic with Machine Learning and Similarity Search

Tese de mestrado, Segurança Informática, Universidade de Lisboa, Faculdade de Ciências, 2022Web applications are the building blocks of many services, from social networks to banks. Network security threats have remained a permanent concern since the advent of data communication. Not withstanding, security breaches are still a serious problem since web applications incorporate both company information and private client data. Traditional Intrusion Detection Systems (IDS) inspect the payload of the packets looking for known intrusion signatures or deviations from nor mal behavior. However, this Deep Packet Inspection (DPI) approach cannot inspect encrypted network traffic of Hypertext Transfer Protocol Secure (HTTPS), a protocol that has been widely adopted nowadays to protect data communication. We are interested in web application attacks, and to accurately detect them, we must access the payload. Network flows are able to aggregate flows of traffic with common properties, so they can be employed for inspecting large amounts of traffic. The main objective of this thesis is to develop a system to discover anomalous HTTPS traffic and confirm that the payloads included in it contains web applications attacks. We propose a new reliable method and system to identify traffic that may include web application attacks by analysing HTTPS network flows (netflows) and discovering payload content similarities. We resort to unsupervised machine learning algorithms to cluster netflows and identify anomalous traffic and to Locality Sensitive Hashing (LSH) algorithms to create a Similarity Search Engine (SSE) capable of correctly identifying the presence of known web applications attacks over this traffic. We involve the system in a continuous improvement process to keep a reliable detection as new web applications attacks are discovered. We evaluated the system, which showed that it could detect anomalous traffic, the SSE was able to confirm the presence of web attacks into that anomalous traffic, and the continuous improvement process was able to increase the accuracy of the SSE

Utilização da Norma JPEG2000 para codificar proteger e comercializar Produtos de Observação Terrestre

Applications like, change detection, global monitoring, disaster detection and management have emerging requirements that need the availability of large amounts of data. This data is currently being capture by a multiplicity of instruments and EO (Earth Observation) sensors originating large volumes of data that needs to be stored, processed and accessed in order to be useful – as an example, ENVISAT accumulates, in a yearly basis, several hundred terabytes of data. This need to recover, store, process and access brings some interesting challenges, like storage space, processing power, bandwidth and security, just to mention a few. These challenges are still very important on today’s technological world. If we take a look for example at the number of subscribers of ISP (Internet Service Providers) broadband services on the developed world today, one can notice that broadband services are still far from being common and dominant. On the underdeveloped countries the picture is even dimmer, not only from a bandwidth point of view but also in all other aspects regarding information and communication technologies (ICTs). All this challenges need to be taken into account if a service is to reach the broadest audience possible. Obviously protection and securing of services and contents is an extra asset that helps on the preservation of possible business values, especially if we consider such a costly business as the space industry. This thesis presents and describes a system which allows, not only the encoding and decoding of several EO products into a JPEG2000 format, but also supports some of the security requirements identified previously that allows ESA (European Space Agency) and related EO services to define and apply efficient EO data access security policies and even to exploit new ways to commerce EO products over the Internet.Aplicações como, detecção de mudanças no terreno, monitorização planetária, detecção e gestão de desastres, têm necessidades prementes que necessitam de vastas quantidades de dados. Estes dados estão presentemente a ser capturados por uma multiplicidade de instrumentos e sensores de observação terrestre, que originam uma enormidade de dados que necessitam de ser armazenados processados e acedidos de forma a se tornarem úteis – por exemplo, a ENVISAT acumula anualmente varias centenas de terabytes de dados. Esta necessidade de recuperar, armazenar, processar e aceder introduz alguns desafios interessantes como o espaço de armazenamento, poder de processamento, largura de banda e segurança dos dados só para mencionar alguns. Estes desafios são muito importantes no mundo tecnológico de hoje. Se olharmos, por exemplo, ao número actual de subscritores de ISP (Internet Service Providers) de banda larga nos países desenvolvidos podemos ficar surpreendidos com o facto do número de subscritores desses serviços ainda não ser uma maioria da população ou dos agregados familiares. Nos países subdesenvolvidos o quadro é ainda mais negro não só do ponto de vista da largura de banda mas também de todos os outros aspectos relacionados com Tecnologias da Informação e Comunicação (TICs). Todos estes aspectos devem ser levados em consideração se se pretende que um serviço se torne o mais abrangente possível em termos de audiências. Obviamente a protecção e segurança dos conteúdos é um factor extra que ajuda a preservar possíveis valores de negócio, especialmente considerando industrias tão onerosas como a Industria Espacial. Esta tese apresenta e descreve um sistema que permite, não só a codificação e descodificação de diversos produtos de observação terrestre para formato JPEG2000 mas também o suporte de alguns requisitos de segurança identificados previamente que permitem, á Agência Espacial Europeia e a outros serviços relacionados com observação terrestre, a aplicação de politicas eficientes de acesso seguro a produtos de observação terrestre, permitindo até o aparecimento de novas forma de comercialização de produtos de observação terrestre através da Internet

Data Publication with the Structural Biology Data Grid Supports Live Analysis

Access to experimental X-ray diffraction image data is fundamental for validation and reproduction of macromolecular models and indispensable for development of structural biology processing methods. Here, we established a diffraction data publication and dissemination system, Structural Biology Data Grid (SBDG; data.sbgrid.org), to preserve primary experimental data sets that support scientific publications. Data sets are accessible to researchers through a community driven data grid, which facilitates global data access. Our analysis of a pilot collection of crystallographic data sets demonstrates that the information archived by SBDG is sufficient to reprocess data to statistics that meet or exceed the quality of the original published structures. SBDG has extended its services to the entire community and is used to develop support for other types of biomedical data sets. It is anticipated that access to the experimental data sets will enhance the paradigm shift in the community towards a much more dynamic body of continuously improving data analysis