A comparative assessment of deep learning models for day-ahead load forecasting: Investigating key accuracy drivers

Short-term load forecasting (STLF) is vital for the effective and economic operation of power grids and energy markets. However, the non-linearity and non-stationarity of electricity demand as well as its dependency on various external factors renders STLF a challenging task. To that end, several deep learning models have been proposed in the literature for STLF, reporting promising results. In order to evaluate the accuracy of said models in day-ahead forecasting settings, in this paper we focus on the national net aggregated STLF of Portugal and conduct a comparative study considering a set of indicative, well-established deep autoregressive models, namely multi-layer perceptrons (MLP), long short-term memory networks (LSTM), neural basis expansion coefficient analysis (N-BEATS), temporal convolutional networks (TCN), and temporal fusion transformers (TFT). Moreover, we identify factors that significantly affect the demand and investigate their impact on the accuracy of each model. Our results suggest that N-BEATS consistently outperforms the rest of the examined models. MLP follows, providing further evidence towards the use of feed-forward networks over relatively more sophisticated architectures. Finally, certain calendar and weather features like the hour of the day and the temperature are identified as key accuracy drivers, providing insights regarding the forecasting approach that should be used per case.Comment: Keywords: Short-Term Load Forecasting, Deep Learning, Ensemble, N-BEATS, Temporal Convolution, Forecasting Accurac

DeepTSF: Codeless machine learning operations for time series forecasting

This paper presents DeepTSF, a comprehensive machine learning operations (MLOps) framework aiming to innovate time series forecasting through workflow automation and codeless modeling. DeepTSF automates key aspects of the ML lifecycle, making it an ideal tool for data scientists and MLops engineers engaged in machine learning (ML) and deep learning (DL)-based forecasting. DeepTSF empowers users with a robust and user-friendly solution, while it is designed to seamlessly integrate with existing data analysis workflows, providing enhanced productivity and compatibility. The framework offers a front-end user interface (UI) suitable for data scientists, as well as other higher-level stakeholders, enabling comprehensive understanding through insightful visualizations and evaluation metrics. DeepTSF also prioritizes security through identity management and access authorization mechanisms. The application of DeepTSF in real-life use cases of the I-NERGY project has already proven DeepTSF's efficacy in DL-based load forecasting, showcasing its significant added value in the electrical power and energy systems domain

Targeted demand response for flexible energy communities using clustering techniques

The present study proposes clustering techniques for designing demand response (DR) programs for commercial and residential prosumers. The goal is to alter the consumption behavior of the prosumers within a distributed energy community in Italy. This aggregation aims to: a) minimize the reverse power flow at the primary substation, occuring when generation from solar panels in the local grid exceeds consumption, and b) shift the system wide peak demand, that typically occurs during late afternoon. Regarding the clustering stage, we consider daily prosumer load profiles and divide them across the extracted clusters. Three popular machine learning algorithms are employed, namely k-means, k-medoids and agglomerative clustering. We evaluate the methods using multiple metrics including a novel metric proposed within this study, namely peak performance score (PPS). The k-means algorithm with dynamic time warping distance considering 14 clusters exhibits the highest performance with a PPS of 0.689. Subsequently, we analyze each extracted cluster with respect to load shape, entropy, and load types. These characteristics are used to distinguish the clusters that have the potential to serve the optimization objectives by matching them to proper DR schemes including time of use, critical peak pricing, and real-time pricing. Our results confirm the effectiveness of the proposed clustering algorithm in generating meaningful flexibility clusters, while the derived DR pricing policy encourages consumption during off-peak hours. The developed methodology is robust to the low availability and quality of training datasets and can be used by aggregator companies for segmenting energy communities and developing personalized DR policies

Bridging the skill gap between the labor market and education in technology through recommender systems

In the last few years technology evolves rapidly, and as a result, the same applies to technology-related professions, concerning the need for specific skills. On the other hand, technological educational institutions oftentimes have difficulties to keep track and align with the labour market needs. Consequently, they sometimes end up teaching skills that are considered obsolete. Regarding students and lifelong learners, an important objective is to avoid courses that teach such skills, as they aim at landing a job in which they will utilise the skills they have acquired during their studies. Furthermore, with regards to educational institutions, an important challenge is to comply with the labour market needs, by providing courses that can be useful for the career of their students. In this context, this dissertation’s main objective is to bridge the gap between the skills taught by educational institutions and the ones required by the labour market in the technology sector, through the use of state-of-the-art data analytics techniques, including data mining, recommender systems and natural language processing. Specifically, in this dissertation, several studies and solutions that have already been proposed are examined in depth. As a next step, a skill and course recommender system has been developed for students and lifelong learners, in order to help them select courses that will improve their skills in accordance with the labour market needs. Additionally, a curriculum redesign recommender system has been developed for professors and educational institutions, for the evaluation and improvement of their organisation’s curriculum, to be compliant with the needs of the labour market. These recommender systems along with their methodology have been applied in the Greek labour market and the sector of software engineering, and the results prove to be very promising. The proposed methodology and services can be applied with little to no modifications to other sectors as well. It is worth mentioning that during the design and development phase of the aforementioned services, several additional problems have emerged. These problems and their solutions are also part of this dissertation. Specifically, this dissertation also deals with the problem of fairness among providers in recommender systems in terms of coverage and diversity in users for each provider’s items. This problem is modeled as an optimisation problem in literature, and it is an NP-complete problem. To this end, we developed a heuristic algorithm with polynomial computational complexity that finds solutions very close to the optimal one, as illustrated in the experimental results. In addition, the problem of recommender system operation and particularly their ethical implications to users is thoroughly examined in this dissertation. In this context, several widely known recommender systems were analysed and proved to be ethically problematic. To this end, we propose a series of regulatory actions. Their goal is the betterment of the recommender systems’ function and the harmonisation of their progress with social prosperity.Τα τελευταία χρόνια η τεχνολογία εξελίσσεται ραγδαία και, κατά συνέπεια, το ίδιο ισχύει και για τα σχετικά επαγγέλματα, όσον αφορά τις απαιτήσεις σε συγκεκριμένες τεχνολογικές δεξιότητες. Από την άλλη πλευρά, τα τεχνολογικά εκπαιδευτικά ιδρύματα συχνά δυσκολεύονται να συμβαδίσουν με τις ανάγκες της αγοράς εργασίας. Κατ’ επέκταση, καταλήγουν να προσφέρουν γνώσεις και δεξιότητες που σε πολλές περιπτώσεις θεωρούνται ξεπερασμένες. Όσον αφορά τους φοιτητές και τους δια βίου μαθητές, ο στόχος τους είναι να αποφύγουν μαθήματα που προσφέρουν ξεπερασμένες γνώσεις και δεξιότητες, καθώς μακροπρόθεσμα στοχεύουν στην εύρεση κάποιας θέσης εργασίας, στην οποία θα αξιοποιούν τις γνώσεις και τις δεξιότητες που έχουν αποκτήσει. Ακόμη, όσον αφορά τα εκπαιδευτικά ιδρύματα μια σημαντική πρόκληση είναι να συμβαδίσουν με τις ανάγκες της αγοράς εργασίας παρέχοντας μαθήματα και δεξιότητες που θα είναι χρήσιμα στην καριέρα των φοιτητών τους. Σε αυτό το πλαίσιο, αντικείμενο της διδακτορικής διατριβής είναι η γεφύρωση του χάσματος μεταξύ των διδασκόμενων γνώσεων και δεξιοτήτων στα εκπαιδευτικά ιδρύματα και των ζητούμενων γνώσεων και δεξιοτήτων από την αγορά εργασίας στον τομέα της τεχνολογίας, με τη χρήση σύγχρονων τεχνικών ανάλυσης δεδομένων όπως είναι η εξόρυξη γνώσης, τα συστήματα συστάσεων και η επεξεργασία φυσικής γλώσσας. Συγκεκριμένα, διερευνώνται σε βάθος εργασίες και λύσεις που έχουν αναπτυχθεί προς αυτή την κατεύθυνση και αναπτύσσονται συστήματα συστάσεων που απευθύνονται σε φοιτητές και δια βίου μαθητές για την επιλογή μαθημάτων και δεξιοτήτων που είναι σε υψηλή ζήτηση από την αγορά εργασίας, και σε εκπαιδευτικούς οργανισμούς, για την αξιολόγηση και βελτίωση του προγράμματος σπουδών τους με βάση τις ανάγκες της αγοράς εργασίας. Τα συγκεκριμένα συστήματα και η προτεινόμενη μεθοδολογία εφαρμόζονται στην ελληνική αγορά εργασίας στον τομέα της ανάπτυξης λογισμικού με αξιόλογα αποτελέσματα. Η προτεινόμενη μεθοδολογία και τα εργαλεία που αναπτύχθηκαν μπορούν εύκολα να χρησιμοποιηθούν με κατάλληλες τροποποιήσεις και σε άλλα επιστημονικά πεδία.Κατά την ανάπτυξη των προαναφερθέντων συστημάτων προέκυψαν κάποια επιπλέον προβλήματα που επίσης αποτελούν αντικείμενο της διδακτορικής διατριβής. Συγκεκριμένα, ασχοληθήκαμε με το πρόβλημα της δικαιοσύνης μεταξύ παρόχων αντικειμένων σε συστήματα συστάσεων, όσον αφορά την κάλυψη των αντικειμένων κάθε παρόχου σε χρήστες και τη διαφορετικότητα μεταξύ αυτών. Το πρόβλημα αυτό αποτελεί NP-πλήρες πρόβλημα σύμφωνα με τη βιβλιογραφία. Σε αυτό το πλαίσιο, αναπτύχθηκε ευριστικός αλγόριθμος πολυωνυμικού χρόονου που βρίσκει λύσεις πολύ κοντά στη βέλτιστη, όπως φαίνεται από τα πειραματικά αποτελέσματα. Επιπλέον, πρόβλημα της παρούσας διατριβής αποτελεί και η λειτουργία των συστημάτων συστάσεων ως προς τον αντίκτυπο που έχει στους χρήστες, και συγκεκριμένα, κατά πόσο είναι ηθική. Σε αυτό το πλαίσιο, αναλύθηκαν διάφορα ευρέως διαδεδομένα συστήματα συστάσεων τα οποία κρίνονται ηθικά προβληματικά. Για το λόγο αυτό, προτείνονται ρυθμιστικές ενέργειες για την εξυγίανση των συστημάτων συστάσεων, ως προς τον ηθικό τους αντίκτυπο

A Reasoning Engine Architecture for Building Energy Metadata Management

<p>During the Buildings’ lifecycle, massive amounts of data, that contain information related to their energy consumption, are generated. Towards the creation of smart building networks, this produced information must be intercepted and harmonized according to building ontologies and schemas. The pattern recognition from building metadata is based on inferencing and intelligent querying, that can be achieved with the utilization of graph and property databases that deploy and host building information. This paper presents a Reasoning Engine Architecture implemented in the context of the H2020 project called MATRYCS that persists building semantic information. It will be leveraged to support real life applications by improving the inference operations.</p&gt

An Advanced Visualisation Engine with Role-Based Access Control for Building Energy Visual Analytics

<p>One of the main challenges of today’s societies is the avoidance of the climate change since the climate crisis in now more evident than ever. Buildings have a large share of total energy consumption and, thus, it is obvious that actions should be taken to reduce their needs. Taking into consideration that nowadays data related to building’s metrics are available in significantly higher rate than in the past, due to the advance of the related technologies, it is necessary to find ways to exploit them in order to draw useful inferences regarding their consumptions and how they can be reduced. For that reason, in this paper we present a Visualisation Engine, which offers a variety of visualisations over stored data. With the usage of the proposed Visualisation Engine, we envision to be able to conduct sufficient research over the data, to generate insightful information regarding their behaviour, and to assist the development of useful solutions towards the direction of more energy efficient buildings.</p&gt

Application and Evaluation of a Blockchain-Centric Platform for Smart Badge Accreditation in Higher Education Institutions

Since its conceptualization in 2008, blockchain technology has advanced rapidly and been applied in multiple domains. In higher education, blockchain can be applied to develop ICT systems that can revolutionize student accreditation through certificate verification and micro-accreditations, which represent skills and other learning outcomes, in the form of digital/smart badges. While there are multiple studies that highlight the significance of blockchain in higher education and propose digital systems, few of those studies include the evaluation of such proposed systems by real users. As such, the research question of how useful a higher education blockchain system would be for its relevant stakeholders remains largely unanswered. In the research publication at hand, a blockchain-powered higher education platform was applied in the School of Electrical and Computer Engineering of the National Technical University of Athens, where it was used and evaluated by students and professors at the school. The evaluation of the platform was positive, and participants found that the smart badge functionality was among the most useful. Finally, the execution and evaluation of the pilot led to several lessons learned and policy recommendations towards dealing with existing barriers and further promoting blockchain in higher education

Bridging the Gap between Technological Education and Job Market Requirements through Data Analytics and Decision Support Services

In the 21st century, technology evolves extremely fast. The same applies to technology-related professions, mostly in terms of skills requirements. Contradictorily, higher education technological institutions are not always in the position to keep up with the labor market requirements. As a result, some of the skills taught in their courses are oftentimes outdated. From a learner’s perspective, the main goal should be to avoid such outdated courses, as for most university students, the long-term objective is to land a job, where they will utilize the skills they acquired from their studies. On the other hand, from an educational decision maker’s perspective, the most important goal is to keep up with the changes in the labor market, offering courses that will be valuable for the prospective careers of students. The work conducted in the context of this publication aims to bridge the gap between education offered in universities and job market skills’ requirements in technology. Specifically, a skill and course recommender system was developed to help learners select courses that are valuable for the job market, as well as a curriculum design service, which recommends updates to a given curriculum based on the job market needs. Both services are built on top of a text mining service that retrieves job posts from several online sources and performs skill extraction from them based on text analytics techniques. Moreover, a decision support service was developed to facilitate optimal decisions for both learners and education decision makers. All services were evaluated positively by 31 early users