2,052 research outputs found

    Deep generative models for network data synthesis and monitoring

    Get PDF
    Measurement and monitoring are fundamental tasks in all networks, enabling the down-stream management and optimization of the network. Although networks inherently have abundant amounts of monitoring data, its access and effective measurement is another story. The challenges exist in many aspects. First, the inaccessibility of network monitoring data for external users, and it is hard to provide a high-fidelity dataset without leaking commercial sensitive information. Second, it could be very expensive to carry out effective data collection to cover a large-scale network system, considering the size of network growing, i.e., cell number of radio network and the number of flows in the Internet Service Provider (ISP) network. Third, it is difficult to ensure fidelity and efficiency simultaneously in network monitoring, as the available resources in the network element that can be applied to support the measurement function are too limited to implement sophisticated mechanisms. Finally, understanding and explaining the behavior of the network becomes challenging due to its size and complex structure. Various emerging optimization-based solutions (e.g., compressive sensing) or data-driven solutions (e.g. deep learning) have been proposed for the aforementioned challenges. However, the fidelity and efficiency of existing methods cannot yet meet the current network requirements. The contributions made in this thesis significantly advance the state of the art in the domain of network measurement and monitoring techniques. Overall, we leverage cutting-edge machine learning technology, deep generative modeling, throughout the entire thesis. First, we design and realize APPSHOT , an efficient city-scale network traffic sharing with a conditional generative model, which only requires open-source contextual data during inference (e.g., land use information and population distribution). Second, we develop an efficient drive testing system — GENDT, based on generative model, which combines graph neural networks, conditional generation, and quantified model uncertainty to enhance the efficiency of mobile drive testing. Third, we design and implement DISTILGAN, a high-fidelity, efficient, versatile, and real-time network telemetry system with latent GANs and spectral-temporal networks. Finally, we propose SPOTLIGHT , an accurate, explainable, and efficient anomaly detection system of the Open RAN (Radio Access Network) system. The lessons learned through this research are summarized, and interesting topics are discussed for future work in this domain. All proposed solutions have been evaluated with real-world datasets and applied to support different applications in real systems

    On the Generation of Realistic and Robust Counterfactual Explanations for Algorithmic Recourse

    Get PDF
    This recent widespread deployment of machine learning algorithms presents many new challenges. Machine learning algorithms are usually opaque and can be particularly difficult to interpret. When humans are involved, algorithmic and automated decisions can negatively impact people’s lives. Therefore, end users would like to be insured against potential harm. One popular way to achieve this is to provide end users access to algorithmic recourse, which gives end users negatively affected by algorithmic decisions the opportunity to reverse unfavorable decisions, e.g., from a loan denial to a loan acceptance. In this thesis, we design recourse algorithms to meet various end user needs. First, we propose methods for the generation of realistic recourses. We use generative models to suggest recourses likely to occur under the data distribution. To this end, we shift the recourse action from the input space to the generative model’s latent space, allowing to generate counterfactuals that lie in regions with data support. Second, we observe that small changes applied to the recourses prescribed to end users likely invalidate the suggested recourse after being nosily implemented in practice. Motivated by this observation, we design methods for the generation of robust recourses and for assessing the robustness of recourse algorithms to data deletion requests. Third, the lack of a commonly used code-base for counterfactual explanation and algorithmic recourse algorithms and the vast array of evaluation measures in literature make it difficult to compare the per formance of different algorithms. To solve this problem, we provide an open source benchmarking library that streamlines the evaluation process and can be used for benchmarking, rapidly developing new methods, and setting up new experiments. In summary, our work contributes to a more reliable interaction of end users and machine learned models by covering fundamental aspects of the recourse process and suggests new solutions towards generating realistic and robust counterfactual explanations for algorithmic recourse

    UMSL Bulletin 2023-2024

    Get PDF
    The 2023-2024 Bulletin and Course Catalog for the University of Missouri St. Louis.https://irl.umsl.edu/bulletin/1088/thumbnail.jp

    Sound Event Detection by Exploring Audio Sequence Modelling

    Get PDF
    Everyday sounds in real-world environments are a powerful source of information by which humans can interact with their environments. Humans can infer what is happening around them by listening to everyday sounds. At the same time, it is a challenging task for a computer algorithm in a smart device to automatically recognise, understand, and interpret everyday sounds. Sound event detection (SED) is the process of transcribing an audio recording into sound event tags with onset and offset time values. This involves classification and segmentation of sound events in the given audio recording. SED has numerous applications in everyday life which include security and surveillance, automation, healthcare monitoring, multimedia information retrieval, and assisted living technologies. SED is to everyday sounds what automatic speech recognition (ASR) is to speech and automatic music transcription (AMT) is to music. The fundamental questions in designing a sound recognition system are, which portion of a sound event should the system analyse, and what proportion of a sound event should the system process in order to claim a confident detection of that particular sound event. While the classification of sound events has improved a lot in recent years, it is considered that the temporal-segmentation of sound events has not improved in the same extent. The aim of this thesis is to propose and develop methods to improve the segmentation and classification of everyday sound events in SED models. In particular, this thesis explores the segmentation of sound events by investigating audio sequence encoding-based and audio sequence modelling-based methods, in an effort to improve the overall sound event detection performance. In the first phase of this thesis, efforts are put towards improving sound event detection by explicitly conditioning the audio sequence representations of an SED model using sound activity detection (SAD) and onset detection. To achieve this, we propose multi-task learning-based SED models in which SAD and onset detection are used as auxiliary tasks for the SED task. The next part of this thesis explores self-attention-based audio sequence modelling, which aggregates audio representations based on temporal relations within and between sound events, scored on the basis of the similarity of sound event portions in audio event sequences. We propose SED models that include memory-controlled, adaptive, dynamic, and source separation-induced self-attention variants, with the aim to improve overall sound recognition

    Security Aspects in Web of Data Based on Trust Principles. A brief of Literature Review

    Get PDF
    Within scientific community, there is a certain consensus to define "Big Data" as a global set, through a complex integration that embraces several dimensions from using of research data, Open Data, Linked Data, Social Network Data, etc. These data are scattered in different sources, which suppose a mix that respond to diverse philosophies, great diversity of structures, different denominations, etc. Its management faces great technological and methodological challenges: The discovery and selection of data, its extraction and final processing, preservation, visualization, access possibility, greater or lesser structuring, between other aspects, which allow showing a huge domain of study at the level of analysis and implementation in different knowledge domains. However, given the data availability and its possible opening: What problems do the data opening face? This paper shows a literature review about these security aspects

    LIPIcs, Volume 251, ITCS 2023, Complete Volume

    Get PDF
    LIPIcs, Volume 251, ITCS 2023, Complete Volum

    UMSL Bulletin 2022-2023

    Get PDF
    The 2022-2023 Bulletin and Course Catalog for the University of Missouri St. Louis.https://irl.umsl.edu/bulletin/1087/thumbnail.jp

    Discussion on drivers and proposition of approaches to support the transition of traditional electricity consumers to prosumers

    Get PDF
    In recent years, traditional power systems have undergone a significant transition, mainly related to the massive penetration of Renewable Energy Sources (RES). More specifically, the transformation of residential consumers into prosumers has been challenging to the traditional operation of electricity markets. This transition brings new challenges and opportunities to the power system, leading to new Business Model (BM). One widely discussed change is related to a consumer-centric or prosumer-driven approach, promoting increased participation of small consumers in power systems. The present thesis aims at discussing the recent BMs as enablers of the increasing prosumers’ role in the energy market and power system worldwide, deepening the discussion with a holistic view of the Brazilian context. To do so, it defines the main features of prosumers and their general related regulation as well as possible market designs within power systems. Moreover, the work intends to contribute to the knowledge, identification and understanding of the main regulatory barriers and enablers for the development of those BMs in the Brazilian energy market. In addition, it discusses enabling technologies to properly create the conditions that sustain new prosumer-driven markets. Then, it presents a comprehensive review of existing and innovative BMs and a discussion on their future roles in modern power systems and, in the Brazilian regulatory framework seeking to guide the decisions for the country to develop its political and regulatory environment in the future. Moreover, a set of recommendations for promoting these BMs in the power system worldwide is provided along with policy recommendations to promote prosumers aggregation in the Brazilian energy sector. An important conclusion is that, even though economically possible, not all innovative BMs can spread around the world due to regulatory issues. Seeking to further explore one of the prosumer-driven approaches presented and the challenges imposed by this innovative BM, a study of energy and reserve markets based on the Peer-to-Peer (P2P) structure is carried out. This structure is very promising for the prosumers’ promotion but presents some challenges for the network operation. A critical challenge is to ensure that network constraints are not violated due to energy trades between peers and neither due to the use of reserve capacity. Therefore, two methodologies are proposed. First, is proposed a three-step approach (P2PTDF), using Topological Distribution Factors (TDF) to penalize peers responsible for violations that may occur in the network constraints, ensuring a feasible solution. Second, it is proposed a new integrated prosumers-DSO approach applied in P2P energy and reserve tradings that also ensures the feasibility of both energy and reserve transactions under network constraints. The proposed approach includes the estimation of reserve requirements based on the RES uncertain behavior from historical generation data, which allows identifying RES patterns. The proposed models are assessed through a case study that uses a 14-bus system, under the technical and economic criteria. The results show that the approaches can ensure a feasible network operation.Nos últimos anos, os sistemas tradicionais de energia passaram por uma transição significativa, principalmente relacionada à penetração massiva de fontes de energia renováveis (do inglês, Renewable energy sources-RES). Mais especificamente, a transformação de consumidores residenciais em prosumidores tem desafiado a atual operação do mercado de energia elétrica. Essa transição traz novos desafios e oportunidades para o sistema elétrico, levando a novos modelos de negócios (do inglês, Business Models-BM). Uma mudança amplamente discutida está relacionada a uma abordagem centrada no consumidor ou direcionada ao prossumidor, promovendo maior participação de pequenos consumidores nos sistemas de energia. A presente tese tem como objetivo discutir os recentes BMs como facilitadores do crescente papel dos prosumidores no mercado de energia e no sistema elétrico mundial, aprofundando a discussão com uma visão holística do contexto brasileiro. Para tanto, define as principais características dos prosumidores e sua regulamentação geral relacionada, bem como possíveis designs de mercado dentro dos sistemas de energia. Além disso, o trabalho pretende contribuir para o conhecimento, identificação e compreensão das principais barreiras regulatórias e facilitadoras para o desenvolvimento desses BMs no mercado brasileiro de energia. Assim como, discutir as tecnologias importantes para criar adequadamente as condições que sustentam novos mercados orientados ao consumidor final. Em seguida, apresenta uma revisão abrangente dos BMs existentes e inovadores e uma discussão sobre seus papéis futuros nos sistemas de energia modernos e, no quadro regulatório brasileiro, buscando orientar as decisões para que o país desenvolva seu ambiente político e regulatório no futuro. Além disso, um conjunto de recomendações para promover esses BMs no sistema de energia em todo o mundo é fornecido juntamente com recomendações de políticas para promover a agregação de prosumidores no setor de energia brasileiro. Uma conclusão importante é que, mesmo sendo economicamente possível, nem todos os BMs inovadores podem se espalhar pelo mundo devido a obstáculos regulatórias. Buscando explorar ainda mais uma das abordagens orientadas ao prosumidor apresentadas e os desafios impostos por este BM inovador, é realizado um estudo dos mercados de energia e de reserva com base na estrutura ponto a ponto (do inglês, peer-to-peer-P2P). Esta estrutura é muito promissora para a promoção dos prosumidores mas apresenta alguns desafios para o funcionamento da rede. Um desafio crítico é garantir que as restrições da rede não sejam violadas devido a negociações de energia entre pares e nem devido ao uso da capacidade de reserva. Portanto, duas metodologias são propostas. Primeiramente, é proposta uma abordagem em três passos (P2PTDF), utilizando Fatores de Distribuição Topológica (do inglês, Topological Distribution Factors-TDF ) para penalizar os peers responsáveis por violações que possam ocorrer nas restrições da rede, garantindo uma solução viável. Em segundo lugar, é proposta uma nova abordagem integrada de prosumidores-DSO aplicada em transações P2P de energia e reserva que também garante a viabilidade de transações de energia e reserva sob restrições de rede. A abordagem proposta inclui a estimativa dos requisitos de reserva com base no comportamento incerto da RES a partir de dados históricos de geração, o que permite identificar padrões de RES. Os modelos propostos são avaliados através de um estudo de caso que utiliza um sistema de 14 barras, sob os critérios técnico e econômico. Os resultados mostram que as abordagens podem garantir uma operação de rede viável abrangendo energia e mercados de reserva

    A Comprehensive Survey of Artificial Intelligence Techniques for Talent Analytics

    Full text link
    In today's competitive and fast-evolving business environment, it is a critical time for organizations to rethink how to make talent-related decisions in a quantitative manner. Indeed, the recent development of Big Data and Artificial Intelligence (AI) techniques have revolutionized human resource management. The availability of large-scale talent and management-related data provides unparalleled opportunities for business leaders to comprehend organizational behaviors and gain tangible knowledge from a data science perspective, which in turn delivers intelligence for real-time decision-making and effective talent management at work for their organizations. In the last decade, talent analytics has emerged as a promising field in applied data science for human resource management, garnering significant attention from AI communities and inspiring numerous research efforts. To this end, we present an up-to-date and comprehensive survey on AI technologies used for talent analytics in the field of human resource management. Specifically, we first provide the background knowledge of talent analytics and categorize various pertinent data. Subsequently, we offer a comprehensive taxonomy of relevant research efforts, categorized based on three distinct application-driven scenarios: talent management, organization management, and labor market analysis. In conclusion, we summarize the open challenges and potential prospects for future research directions in the domain of AI-driven talent analytics.Comment: 30 pages, 15 figure
    corecore