Data engineering and best practices

Mestrado Bolonha em Data Analytics for BusinessThis report presents the results of a study on the current state of data engineering at LGG Advisors company. Analyzing existing data, we identified several key trends and challenges facing data engineers in this field. Our study's key findings include a lack of standardization and best practices for data engineering processes, a growing need for more sophisticated data management and analysis tools and data security, and a lack of trained and experienced data engineers to meet the increasing demand for data-driven solutions. Based on these findings, we recommend several steps that organizations at LGG Advisors company can take to improve their data engineering capabilities, including investing in training and education programs, adopting best practices for data management and analysis, and collaborating with other organizations to share knowledge and resources. Data security is also an essential concern for data engineers, as data breaches can have significant consequences for organizations, including financial losses, reputational damage, and regulatory penalties. In this thesis, we will review and evaluate some of the best software tools for securing data in data engineering environments. We will discuss these tools' key features and capabilities and their strengths and limitations to help data engineers choose the best software for protecting their data. Some of the tools we will consider include encryption software, access control systems, network security tools, and data backup and recovery solutions. We will also discuss best practices for implementing and managing these tools to ensure data security in data engineering environments. We engineer data using intuition and rules of thumb. Many of these rules are folklore. Given the rapid technological changes, these rules must be constantly reevaluated.info:eu-repo/semantics/publishedVersio

-

In the last ten years, blockchain technology has transformed from just a digital currency system to supporting complex smart contracts and arbitrary use cases in business. Public blockchains have paved the way for decentralized, tamper-resistant, and transparent systems, offering a solution for the single point of failure found in centralized enterprise systems today. With a literature review conducted on blockchain, enterprise resource planning systems, and their collective usage, this thesis aims to familiarize the reader with the technology and explores the most potential use cases of blockchain in regular enterprise resource planning functions. This thesis identified supply chain management and accounting processes as the most viable applications of blockchain technology within the context of ERP systems. Furthermore, permissioned blockchains have demonstrated better suitability for enterprise use due to their relative efficiency and closed environment compared to public blockchains. The reviewed literature also emphasized the significance of smart contract functionalities creating possibilities of automation and enhanced process efficiency, as well as presented decentralized access controls as a way to prevent unauthorized access to the ERP data