Effective demand response gathering and deployment in smart grids for intensive renewable integration using aggregation and machine learning

Tesis por compendio de publicaciones.[EN] Distributed generation, namely renewables-based technologies, have emerged as a crucial component in the transition to mitigate the effects of climate change, providing a decentralized approach to electricity production. However, the volatile behavior of distributed generation has created new challenges in maintaining system balance and reliability. In this context, the demand response concept and corresponding programs arise giving the local energy communities prominence. In demand response concept, it is expected an empowerment of the consumer in the electricity sector. This has a significant impact on grid operations and brings complex interactions due to the volatile behavior, privacy concerns, and lack of consumer knowledge in the energy market context. For this, aggregators play a crucial role addressing these challenges. It is crucial to develop tools that allow the aggregators helping consumers to make informed decisions, maximize the benefits of their flexibility resources, and contribute to the overall success of grid operations. This thesis, through innovative solutions and resorting to artificial intelligence models, addresses the integration of renewables, promoting fair participation among all demand response providers. The thesis ultimately results in an innovative decision support system - MAESTRO, the Machine learning Assisted Energy System management Tool for Renewable integration using demand respOnse. MAESTRO is composed by a set of diversified models that together contribute for handling the complexity of managing energy communities with distributed generation resources, demand response providers, energy storage systems and electric vehicles. This PhD thesis comprises a comprehensive analysis of state-of-the-art techniques, system design and development, experimental results, and key findings. In this research were published twenty-six scientific papers, in both international journals and conference proceedings. Contributions to international projects and Portuguese projects was accomplished. [ES] La generación distribuida, en particular las tecnologías basadas en energías renovables, se ha convertido en un componente crucial en la transición para mitigar los efectos del cambio climático, al proporcionar un enfoque descentralizado para la producción de electricidad. Sin embargo, el comportamiento volátil de la generación distribuida ha generado nuevos desafíos para mantener el equilibrio y la confiabilidad del sistema. En este contexto, surge el concepto de respuesta de la demanda y los programas correspondientes, otorgando prominencia a las comunidades energéticas locales. En el concepto de "respuesta a la demanda" (DR por sus siglas en inglés), se espera un empoderamiento del consumidor en el sector eléctrico. Esto tiene un impacto significativo en la operación de la red y genera interacciones complejas debido al comportamiento volátil, las preocupaciones de privacidad y la falta de conocimiento del consumidor en el contexto del mercado energético. Para esto, los agregadores desempeñan un papel crucial al abordar estos desafíos. Es fundamental desarrollar herramientas que permitan a los agregadores ayudar a los consumidores a tomar decisiones informadas, maximizar los beneficios de sus recursos de flexibilidad y contribuir al éxito general de las operaciones de la red. Esta tesis, a través de soluciones innovadoras y utilizando modelos de inteligencia artificial, aborda la integración de energías renovables, promoviendo una participación justa entre todos los proveedores de respuesta de la demanda. La tesis resulta en última instancia en un sistema de apoyo a la toma de decisiones innovador: MAESTRO, Machine learning Assisted Energy System management Tool for Renewable integration using demand respOnse. MAESTRO está compuesto por un conjunto de modelos diversificados que contribuyen juntos para manejar la complejidad de la gestión de comunidades energéticas con recursos de generación distribuida, proveedores de respuesta de la demanda, sistemas de almacenamiento de energía y vehículos eléctricos. Esta tesis de doctorado comprende un análisis exhaustivo de las técnicas de vanguardia, el diseño y desarrollo del sistema, los resultados experimentales y los hallazgos clave. En esta investigación se publicaron veintiséis artículos científicos, tanto en revistas internacionales como en actas de conferencias. Se lograron contribuciones a proyectos internacionales y proyectos portugueses. [POR] A produção distribuída, nomeadamente as tecnologias baseadas em energias renováveis, emergiram como um componente crucial na transição para mitigar os efeitos das alterações climáticas, proporcionando uma abordagem descentralizada à produção de eletricidade. No entanto, o comportamento volátil da geração distribuída criou desafios na manutenção do equilíbrio e da fiabilidade do sistema. Nesse contexto, surge o conceito de resposta à procura e os programas correspondentes, conferindo proeminência às comunidades energéticas locais. No conceito de resposta à procura, espera-se um empoderamento do consumidor no setor elétrico. Isso tem um impacto significativo nas operações da rede e gera interações complexas devido ao comportamento volátil, preocupações com a privacidade e falta de conhecimento dos consumidores no contexto do mercado energético. Para isso, os agregadores desempenham um papel crucial ao lidar com esses desafios. É fundamental desenvolver ferramentas que permitam aos agregadores ajudar os consumidores a tomar decisões informadas, maximizar os benefícios de seus recursos de flexibilidade e contribuir para o sucesso global das operações da rede. Esta tese de doutoramento, através de soluções inovadoras e recorrendo a modelos de inteligência artificial, aborda a integração de energias renováveis, promovendo uma participação justa entre todos os fornecedores de resposta à procura. A tese resulta, em última instância, num sistema inovador de apoio à tomada de decisões - MAESTRO, Machine learning Assisted Energy System management Tool for Renewable integration using demand respOnse. A ferramenta MAESTRO é composta por um conjunto de modelos diversificados que, em conjunto, contribuem para lidar com a complexidade da gestão de comunidades energéticas com recursos de geração distribuída, fornecedores de resposta à procura, sistemas de armazenamento de energia e veículos elétricos. Esta tese de doutoramento abrange uma análise abrangente de técnicas de ponta, design e desenvolvimento do sistema, resultados experimentais e descobertas-chave. Nesta pesquisa, foram publicados vinte e seis artigos científicos, tanto em revistas internacionais como em atas de conferências. Foram realizadas contribuições para projetos internacionais e projetos portugueses

Cell type identification, differential expression analysis and trajectory inference in single-cell transcriptomics

Single-cell RNA-sequencing (scRNA-seq) is a cutting-edge technology that enables to quantify the transcriptome, the set of expressed RNA transcripts, of a group of cells at the single-cell level. It represents a significant upgrade from bulk RNA-seq, which measures the combined signal of thousands of cells. Measuring gene expression by bulk RNA-seq is an invaluable tool for biomedical researchers who want to understand how cells alter their gene expression due to an illness, differentiation, ternal stimulus, or other events. Similarly, scRNA-seq has become an essential method for biomedical researchers, and it has brought several new applications previously unavailable with bulk RNA-seq. scRNA-seq has the same applications as bulk RNA-seq. However, the single-cell resolution also enables cell annotation based on gene markers of clusters, that is, cell populations that have been identified based on machine learning to be, on average, dissimilar at the transcriptomic level. Researchers can use the cell clusters to detect cell-type-specific gene expression changes between conditions such as case and control groups. Clustering can sometimes even discover entirely new cell types. Besides the cluster-level representation, the single-cell resolution also enables to model cells as a trajectory, representing how the cells are related at the cell level and what is the dynamic differentiation process that the cells undergo in a tissue. This thesis introduces new computational methods for cell type identification and trajectory inference from scRNA-seq data. A new cell type identification method (ILoReg) was proposed, which enables high-resolution clustering of cells into populations with subtle transcriptomic differences. In addition, two new trajectory inference methods were developed: scShaper, which is an accurate and robust method for inferring linear trajectories; and Totem, which is a user-friendly and flexible method for inferring tree-shaped trajectories. In addition, one of the works benchmarked methods for detecting cell-type-specific differential states from scRNA-seq data with multiple subjects per comparison group, requiring tailored methods to confront false discoveries. KEYWORDS: Single-cell RNA sequencing, transcriptome, cell type identification, trajectory inference, differential expressionYksisoluinen RNA-sekvensointi on huipputeknologia, joka mahdollistaa transkriptomin eli ilmentyneiden RNA-transkriptien laskennallisen määrittämisen joukolle soluja yhden solun tarkkuudella, ja sen kehittäminen oli merkittävä askel eteenpäin perinteisestä bulkki-RNA-sekvensoinnista, joka mittaa tuhansien solujen yhteistä signaalia. Bulkki-RNA-sekvensointi on tärkeä työväline biolääketieteen tutkijoille, jotka haluavat ymmärtää miten solut muuttavat geenien ilmentymistä sairauden, erilaistumisen, ulkoisen ärsykkeen tai muun tapahtuman seurauksena. Yksisoluisesta RNA-sekvensoinnista on vastaavasti kehittynyt tärkeä työväline tutkijoille, ja se on tuonut useita uusia sovelluksia. Yksisoluisella RNA-sekvensoinnilla on samat sovellukset kuin bulkki-RNA-sekvensoinnilla, mutta sen lisäksi se mahdollistaa solujen tunnistamisen geenimarkkerien perusteella. Geenimarkkerit etsitään tilastollisin menetelmin solupopulaatioille, joiden on tunnistettu koneoppimisen menetelmin muodostavan transkriptomitasolla keskenään erilaisia joukkoja eli klustereita. Tutkijat voivat hyödyntää soluklustereita tutkimaan geeniekspressioeroja solutyyppien sisällä esimerkiksi sairaiden ja terveiden välillä, ja joskus klusterointi voi jopa tunnistaa uusia solutyyppejä. Yksisolutason mittaukset mahdollistavat myös solujen mallintamisen trajektorina, joka esittää kuinka solut kehittyvät dynaamisesti toisistaan geenien ilmentymistä vaativien prosessien aikana. Tämä väitöskirja esittelee uusia laskennallisia menetelmiä solutyyppien ja trajektorien tunnistamiseen yksisoluisesta RNA-sekvensointidatasta. Väitöskirja esittelee uuden solutyyppitunnistusmenetelmän (ILoReg), joka mahdollistaa hienovaraisia geeniekspressioeroja sisältävien solutyyppien tunnistamisen. Sen lisäksi väitöskirjassa kehitettiin kaksi uutta trajektorin tunnistusmenetelmää: scShaper, joka on tarkka ja robusti menetelmä lineaaristen trajektorien tunnistamiseen, sekä Totem, joka on käyttäjäystävällinen ja joustava menetelmä puumallisten trajektorien tunnistamiseen. Lopuksi väitöskirjassa vertailtiin menetelmiä solutyyppien sisäisten geeniekspressioerojen tunnistamiseen ryhmien välillä, joissa on useita koehenkilöitä tai muita biologisia replikaatteja, mikä vaatii erityisiä menetelmiä väärien positiivisten löydösten vähentämiseen. ASIASANAT: yksisoluinen RNA-sekvensointi, klusterointi, trajektorin tunnistus, geeniekspressi

Energy-efficient routing protocols in heterogeneous wireless sensor networks

Sensor networks feature low-cost sensor devices with wireless network capability, limited transmit power, resource constraints and limited battery energy. The usage of cheap and tiny wireless sensors will allow very large networks to be deployed at a feasible cost to provide a bridge between information systems and the physical world. Such large-scale deployments will require routing protocols that scale to large network sizes in an energy-efficient way. This thesis addresses the design of such network routing methods. A classification of existing routing protocols and the key factors in their design (i.e., hardware, topology, applications) provides the motivation for the new three-tier architecture for heterogeneous networks built upon a generic software framework (GSF). A range of new routing algorithms have hence been developed with the design goals of scalability and energy-efficient performance of network protocols. They are respectively TinyReg - a routing algorithm based on regular-graph theory, TSEP - topological stable election protocol, and GAAC - an evolutionary algorithm based on genetic algorithms and ant colony algorithms. The design principle of our routing algorithms is that shortening the distance between the cluster-heads and the sink in the network, will minimise energy consumption in order to extend the network lifetime, will achieve energy efficiency. Their performance has been evaluated by simulation in an extensive range of scenarios, and compared to existing algorithms. It is shown that the newly proposed algorithms allow long-term continuous data collection in large networks, offering greater network longevity than existing solutions. These results confirm the validity of the GSF as an architectural approach to the deployment of large wireless sensor networks

The effect of reference price regulation on pharmaceutical prices in Finland

This thesis considers the impact of the Finnish reference pricing system (RPS) on pharmaceutical wholesale prices. The policy, introduced in 2009, sets a maximum amount for the public reimbursement of pharmaceuticals assigned to the system. The reform was designed to improve the generic substitution practice that began in 2003. Using rich panel data for the years of 2006–2012, I apply the difference-in-differences method, exploiting the fact that some products never entered reference pricing (RP). I find statistically significant evidence that prices fell by 5.9% in 2009 and 8.7% in 2010, after which the effect disappeared. However, concern with parallel price trends between the treatment and control groups prevents causal interpretation of the results.Tutkielman tiivistelmätiedoissa näkyvä hyväksymisvuosi on 2019.The year of approval showing in the abstract of the thesis is 2019

Mostly Beneficial Clustering: Aggregating Data for Operational Decision Making

With increasingly volatile market conditions and rapid product innovations, operational decision-making for large-scale systems entails solving thousands of problems with limited data. Data aggregation is proposed to combine the data across problems to improve the decisions obtained by solving those problems individually. We propose a novel cluster-based Shrunken-SAA approach that can exploit the cluster structure among problems when implementing the data aggregation approaches. We prove that, as the number of problems grows, leveraging the given cluster structure among problems yields additional benefits over the data aggregation approaches that neglect such structure. When the cluster structure is unknown, we show that unveiling the cluster structure, even at the cost of a few data points, can be beneficial, especially when the distance between clusters of problems is substantial. Our proposed approach can be extended to general cost functions under mild conditions. When the number of problems gets large, the optimality gap of our proposed approach decreases exponentially in the distance between the clusters. We explore the performance of the proposed approach through the application of managing newsvendor systems via numerical experiments. We investigate the impacts of distance metrics between problem instances on the performance of the cluster-based Shrunken-SAA approach with synthetic data. We further validate our proposed approach with real data and highlight the advantages of cluster-based data aggregation, especially in the small-data large-scale regime, compared to the existing approaches

Tracing, Ranking and Pricing DER Flexibility in Active Distribution Networks

This paper presents a framework for analysing the aggregated flexibility of active distribution networks (ADNs) with distributed energy resources (DER). The analysis takes a different perspective than existing studies, which focus on characterising flexibility as the limits of the flexible power provision, i.e., the set of the network feasible operating points in the P-Q space. Instead, this work aims to estimate the contributions of different flexible units to the aggregated flexibility, which is essential for flexible power ranking and pricing. The proposed framework exploits cost-minimising OPF models complemented with cooperative game formulations that are able to capture the combinatorial nature of activating multiple flexible units. Moreover, in contrast to existing studies that imply perfect coordination of units, the proposed framework specifies the actions needed to reach feasible operating points, reflecting the nonlinearities of the network flexibility model. Extensive simulations are performed for different flexibility metrics to demonstrate the applicability of the framework. Depending on the metric selected (capacity, cost, or economic surplus of flexibility), distribution system operators (DSOs) can identify the most critical flexible units or remunerate units for participating in flexibility services provision

Network monitoring and performance assessment: from statistical models to neural networks

Máster en Investigación e Innovación en Tecnologías de la Información y las ComunicacionesIn the last few years, computer networks have been playing a key role in many different fields. Companies have also evolved around the internet, getting advantage of the huge capacity of diffusion. Nevertheless, this also means that computer networks and IT systems have become a critical element for the business. In case of interruption or malfunction of the systems, this could result in devastating economic impact. In this light, it is necessary to provide models to properly evaluate and characterize the computer networks. Focusing on modeling, one has many different alternatives: from classical options based on statistic to recent alternatives based on machine learning and deep learning. In this work, we want to study the different models available for each context, paying attention to the advantage and disadvantages to provide the best solution for each case. To cover the majority of the spectrum, three cases have been studied: time-unaware phenomena, where we look at the bias-variance trade-off, time-dependent phenomena, where we pay attention the trends of the time series, and text processing to process attributes obtained by DPI. For each case, several alternatives have been studied and solutions have been tested both with synthetic data and real-world data, showing the successfulness of the proposa