1,359 research outputs found

    LIPIcs, Volume 251, ITCS 2023, Complete Volume

    Get PDF
    LIPIcs, Volume 251, ITCS 2023, Complete Volum

    Reinforcement learning in large state action spaces

    Get PDF
    Reinforcement learning (RL) is a promising framework for training intelligent agents which learn to optimize long term utility by directly interacting with the environment. Creating RL methods which scale to large state-action spaces is a critical problem towards ensuring real world deployment of RL systems. However, several challenges limit the applicability of RL to large scale settings. These include difficulties with exploration, low sample efficiency, computational intractability, task constraints like decentralization and lack of guarantees about important properties like performance, generalization and robustness in potentially unseen scenarios. This thesis is motivated towards bridging the aforementioned gap. We propose several principled algorithms and frameworks for studying and addressing the above challenges RL. The proposed methods cover a wide range of RL settings (single and multi-agent systems (MAS) with all the variations in the latter, prediction and control, model-based and model-free methods, value-based and policy-based methods). In this work we propose the first results on several different problems: e.g. tensorization of the Bellman equation which allows exponential sample efficiency gains (Chapter 4), provable suboptimality arising from structural constraints in MAS(Chapter 3), combinatorial generalization results in cooperative MAS(Chapter 5), generalization results on observation shifts(Chapter 7), learning deterministic policies in a probabilistic RL framework(Chapter 6). Our algorithms exhibit provably enhanced performance and sample efficiency along with better scalability. Additionally, we also shed light on generalization aspects of the agents under different frameworks. These properties have been been driven by the use of several advanced tools (e.g. statistical machine learning, state abstraction, variational inference, tensor theory). In summary, the contributions in this thesis significantly advance progress towards making RL agents ready for large scale, real world applications

    Tensor-variate machine learning on graphs

    Get PDF
    Traditional machine learning algorithms are facing significant challenges as the world enters the era of big data, with a dramatic expansion in volume and range of applications and an increase in the variety of data sources. The large- and multi-dimensional nature of data often increases the computational costs associated with their processing and raises the risks of model over-fitting - a phenomenon known as the curse of dimensionality. To this end, tensors have become a subject of great interest in the data analytics community, owing to their remarkable ability to super-compress high-dimensional data into a low-rank format, while retaining the original data structure and interpretability. This leads to a significant reduction in computational costs, from an exponential complexity to a linear one in the data dimensions. An additional challenge when processing modern big data is that they often reside on irregular domains and exhibit relational structures, which violates the regular grid assumptions of traditional machine learning models. To this end, there has been an increasing amount of research in generalizing traditional learning algorithms to graph data. This allows for the processing of graph signals while accounting for the underlying relational structure, such as user interactions in social networks, vehicle flows in traffic networks, transactions in supply chains, chemical bonds in proteins, and trading data in financial networks, to name a few. Although promising results have been achieved in these fields, there is a void in literature when it comes to the conjoint treatment of tensors and graphs for data analytics. Solutions in this area are increasingly urgent, as modern big data is both large-dimensional and irregular in structure. To this end, the goal of this thesis is to explore machine learning methods that can fully exploit the advantages of both tensors and graphs. In particular, the following approaches are introduced: (i) Graph-regularized tensor regression framework for modelling high-dimensional data while accounting for the underlying graph structure; (ii) Tensor-algebraic approach for computing efficient convolution on graphs; (iii) Graph tensor network framework for designing neural learning systems which is both general enough to describe most existing neural network architectures and flexible enough to model large-dimensional data on any and many irregular domains. The considered frameworks were employed in several real-world applications, including air quality forecasting, protein classification, and financial modelling. Experimental results validate the advantages of the proposed methods, which achieved better or comparable performance against state-of-the-art models. Additionally, these methods benefit from increased interpretability and reduced computational costs, which are crucial for tackling the challenges posed by the era of big data.Open Acces

    Decoding the Real World: Tackling Virtual Ethnographic Challenges through Data-Driven Methods

    Get PDF

    TRANSEUNTIS MUNDI, A NOMADIC ARTISTIC PRACTICE

    Get PDF
    In this practice-led Ph.D. research, I investigate how an artistic practice can respond to the migration phenomena performed by human beings across the planet over millennia ¬– what I refer to as the millennial global human journey. Based on the idea of mobility, I chose to frame this research in the articulation of concepts deriving from the prefix trans: transculture, transhumance and transmediality. This research contributes to studies in art composition by developing the processes and concept of transmedial composition, mainly contributing to the field of New Media Art. This investigation resulted in the work Transeuntis Mundi (TM) Project – a nomadic artistic practice that encompasses: the TM Derive and manual, the TM Archive, the TM VR work Derive 01 and two forms for its notation. Transeuntis mundi (TM), from the Latin language, means the ‘passersby of the world’ and metaphorically personify in this work the millennial migrants and their global journeys. Based on proposals from the Realism art movement and the walking-based methodologies of Walkscapes and DĂ©rive, the TM Derive was created as a nomadic methodology of composition in response to the ideas of migration and ancestry. It is framed by the minimal stories ¬– the form of narrative of this work, captured from field recordings with 3D technology of everyday life worldwide. This material formed the TM Archive, presented in the TM VR work. The TM VR work Transeuntis Mundi Derive 01 is an immersive and interactive performative experience for virtual reality, that artistically brings together stories, sounds, images, people, and places worldwide, ÂŹas a metaphor of the millennial global human migration. This work happens as a VR application using 3D technology with 360Âș image and ambisonic sound, in order to promote an engaged experience through the immersion and interactivity of the participant. This thesis presents and contextualizes these creations: the scope, references, concepts, origin, collaborations, methodology, technologies, and results of this work. It is informed and accompanied by reflexive and critical writing, including an articulation with references of works across different artistic media and fields.UNIRIO Federal University of the State of Rio de Janeir

    Sampling scale sensitivities in surface ocean pCO2 reconstructions in the Southern Ocean

    Get PDF
    The Southern Ocean plays a pre-eminent role in the global carbon-climate system. Model studies show that since the start of the preindustrial era, the region has absorbed about 75% of excess heat and 50% of the oceanic uptake and storage (42±5 PgC) of anthropogenic CO2 emissions. However, due to the spatial and seasonal sparseness of the Southern Ocean CO2 observations (biased toward summer), this role is poorly understood. The seasonal sampling biases have hampered observation-based reconstructions of partial pressure of CO2 at the surface ocean (pCO2) using machine learning (ML) and contributed to the convergence of the root mean squared errors (RMSEs) of ML methods to a common limit known in the literature as the “wall”. The hypothesis here is that addressing the critical missing sampling scale will get the community reconstructions of pCO2 “over the wall”. In this study, I explore the sensitivity of pCO2 reconstructions to these observational scale gaps. Using a scale-sensitive sampling strategy means adopting a sampling strategy which addresses these observational limitations including intra-seasonal as well as seasonal sampling aliases in high eddy kinetic energy and mesoscale-intensive regions. In increasing CO2 sampling efforts in the Southern Ocean using autonomous sampling platforms such as floats, Wave Gliders and Saildrones, the community has tried to answer this problem, but the effectiveness of these efforts has not yet been tested. This study aims to do this evaluation and advance our understanding of the sampling scale sensitivities of surface ocean pCO2 reconstructions from machine-learning techniques and contribute – through a scale-sensitive sampling strategy of observing platforms in the Southern Ocean – to breaking through the proverbial “wall”. This aim was achieved through a series of observing system simulation experiments (OSSEs) applied to a forced mesoscale-resolving (±10km) ocean NEMO-PISCES physics-biogeochemistry model with daily output. In addition to underway ships, the sampling scales of the autonomous sampling platforms such as Floats, WaveGliders and Saildrones, on pCO2 reconstructions were investigated in this series of OSSEs. The primary results showed that two sampling scales, which Saildrones are able to address, are required to improve the RMSE scores of machine-learning techniques and then reduce uncertainties and biases in pCO2 reconstructions. The two sampling scales include (1) the seasonal cycle of the meridional gradients and (2) the intra-seasonal variability. Based on the impacts of these two sampling scales on the RMSE scores and biases, it wasfound that resolving the seasonal cycle of the meridional gradient is the first-order requirement while resolving the intra-seasonal variability is the second. Applying the second-order requirement in the whole Southern Ocean to explore the sensitivity of the clustering choice to the two-step pCO2 reconstruction (clustering- regression). It was found that using an ensemble of clustering methods in this two-step reconstruction performs far much better than using a clustering method. Using these findings, I proposed an observational strategy that is viable and strengthens the limitations in existing underway SOCAT ship- and SOCCOM float-based reconstructions of surface ocean pCO2. More specifically, I proposed a hybrid scale-sensitive sampling strategy for the whole Southern Ocean by integrating underway ships with Saildrones on winter lines. The analysis of these multiple OSSEs indicates that improving the pCO2 reconstructions requires scalesensitive data to supplement the underway ship-based observations gridded in the SOCAT product. It was also found that scale-sensitive data consisting of high-resolution observations ( 1 day) extending over the seasonal cycle and capturing the pCO2 meridional gradients results in breaking through the proverbial “wall”. These findings will contribute to an accurate mean annual global carbon budget which is critical for the trend of the ocean sink feedback on global warming as well as ocean acidification

    LIPIcs, Volume 261, ICALP 2023, Complete Volume

    Get PDF
    LIPIcs, Volume 261, ICALP 2023, Complete Volum

    Processing massive graphs under limited visibility

    Get PDF
    Graphs are one of the most important and widely used combinatorial structures in mathematics. Their ability to model many real world scenarios which involve a large network of related entities make them useful across disciplines. They are useful as an abstraction in the analysis of networked structures such as the Internet, social networks, road networks, biological networks and many more. The graphs arising out of many real world phenomenon can be very large and they keep evolving over time. For example, Facebook reported over 2:9 billion monthly active users in 2022. Another very large and dynamic network is the human brain consisting of around 1011 nodes and many more edges. These large and evolving graphs present new challenges for algorithm designers. Traditional graph algorithms designed to work with centralised and sequential computing models are rendered useless due to their prohibitively high resource usage. In fact one needs huge amounts of resources just to read the entire graph. A number of new theoretical models have been devised over the years to keep up with the trends in the modern computing systems capable of handing massive input datasets. Some of these models such as streaming model and the query model allow the algorithm to view the graph piecemeal. In some cases, the model allows the graph to be processed by a set of interconnected computing elements such as in distributed computing. In this thesis we address some graph problems in these non-centralised, non-sequential models of computing with a limited access to the input graph. Specifically, we address three different graph problems, each in a different computing model. The first problem we look at is the computation of approximate shortest paths in dynamic streams. The second problem deals with finding kings in tournament graphs, given query access to the arcs of the tournament. The third and the final problem we investigate is a local test criteria for testing the expansion of a graph in the distributed CONGEST model
    • 

    corecore