10,852 research outputs found

    Reinforcement learning in large state action spaces

    Get PDF
    Reinforcement learning (RL) is a promising framework for training intelligent agents which learn to optimize long term utility by directly interacting with the environment. Creating RL methods which scale to large state-action spaces is a critical problem towards ensuring real world deployment of RL systems. However, several challenges limit the applicability of RL to large scale settings. These include difficulties with exploration, low sample efficiency, computational intractability, task constraints like decentralization and lack of guarantees about important properties like performance, generalization and robustness in potentially unseen scenarios. This thesis is motivated towards bridging the aforementioned gap. We propose several principled algorithms and frameworks for studying and addressing the above challenges RL. The proposed methods cover a wide range of RL settings (single and multi-agent systems (MAS) with all the variations in the latter, prediction and control, model-based and model-free methods, value-based and policy-based methods). In this work we propose the first results on several different problems: e.g. tensorization of the Bellman equation which allows exponential sample efficiency gains (Chapter 4), provable suboptimality arising from structural constraints in MAS(Chapter 3), combinatorial generalization results in cooperative MAS(Chapter 5), generalization results on observation shifts(Chapter 7), learning deterministic policies in a probabilistic RL framework(Chapter 6). Our algorithms exhibit provably enhanced performance and sample efficiency along with better scalability. Additionally, we also shed light on generalization aspects of the agents under different frameworks. These properties have been been driven by the use of several advanced tools (e.g. statistical machine learning, state abstraction, variational inference, tensor theory). In summary, the contributions in this thesis significantly advance progress towards making RL agents ready for large scale, real world applications

    Coworking through the Pandemic: Flexibly Yours

    Get PDF
    Coworking can be defined as a paid for service (usually) providing shared workspace and amenities to users. When the pandemic hit, owing to the business model’s in-person foundations of physical proximity and shared amenities, the coworking industry was expected to be seriously impacted. Yet fast forward, and as the pandemic has played out, coworking businesses are uniquely positioned in this uncertain and changing workscape. This dissertation presents one of the first academic explorations into how independent coworking businesses fared in the initial year of the pandemic. Specifically, the research explores the following questions: 1. How did independent coworking businesses manage and adapt to the pandemic? 2. What is virtual coworking and what are the experiences of workers in these virtual coworking spaces? 3. How does coworking flexibility affect social support and connection? Using a critically interpretive poststructural approach, this ethnography included virtual fieldwork and interviews. Sixty hours of virtual participant observation and 30 loosely structured interviews were conducted with coworking stakeholders (i.e., owner-operators, managers, and users) over videoconferencing platforms. Secondary data included written fieldnotes and coworking documents. Results capture the strategies used by coworking business owner-operators and managers to sustain their businesses and the attendant relationships with coworking users, irrespective of whether or not a physical location could be provided under pandemic lockdowns. Given the expansion of coworking businesses into virtual service offerings, a key contribution of my research is the finding that co-location in a physical coworking space is not necessary to cultivate vibes and a sense of community. By removing the physical infrastructure of coworking, the virtual coworking product in which I participated points to both a reinforcement of and an emphasis on the centrality of social connection, support, and community. By de-centering the priority of a physical co-location, I conceptualize coworking businesses as commodified support infrastructures—affective atmospheres produced through the entanglement of human bodies, other living things, objects, and technologies in a space. In viewing coworking businesses as fluid affective atmospheres of support, my research adds to the emerging coworking scholarship that attends to the atmospheric qualities of coworking, the role of affective labour, and the possibilities of encounters and interactions as bodies, objects, and technologies interconnect. My results reinforce the deep ambivalence of coworking, capturing tensions between productivity and sociality, and a blurring of boundaries between professional and private, and work and leisure. The analysis also suggests that the inherent flexibility, informality, turnover, and autonomy in coworking practices can make creating stable social connections and support difficult. Finally, the COVID-19 crisis brought to light how coworking lies primarily outside the scope of current employment legislation, which includes occupational health and safety, employment standards, and workers’ compensation. In the absence of well-defined policy directions, coworking business owner-operators and managers made individualized decisions, thereby ultimately downloading further risk and responsibility onto their coworking users

    Integration of heterogeneous data sources and automated reasoning in healthcare and domotic IoT systems

    Get PDF
    In recent years, IoT technology has radically transformed many crucial industrial and service sectors such as healthcare. The multi-facets heterogeneity of the devices and the collected information provides important opportunities to develop innovative systems and services. However, the ubiquitous presence of data silos and the poor semantic interoperability in the IoT landscape constitute a significant obstacle in the pursuit of this goal. Moreover, achieving actionable knowledge from the collected data requires IoT information sources to be analysed using appropriate artificial intelligence techniques such as automated reasoning. In this thesis work, Semantic Web technologies have been investigated as an approach to address both the data integration and reasoning aspect in modern IoT systems. In particular, the contributions presented in this thesis are the following: (1) the IoT Fitness Ontology, an OWL ontology that has been developed in order to overcome the issue of data silos and enable semantic interoperability in the IoT fitness domain; (2) a Linked Open Data web portal for collecting and sharing IoT health datasets with the research community; (3) a novel methodology for embedding knowledge in rule-defined IoT smart home scenarios; and (4) a knowledge-based IoT home automation system that supports a seamless integration of heterogeneous devices and data sources

    Stray Flux Monitoring and Multi-Sensor Fusion Condition Monitoring for Squirrel Cage Induction Machines

    Get PDF
    This research work investigates the ability of external magnetic flux-based condition monitoring to detect rotor-related faults and incipient stage bearing faults in squirrel-cage induction machines (SCIMs). This work also discusses the multisensory synergy of the external magnetic flux measurement with other measurements. To investigate the stray flux-based monitoring technique, this dissertation presents a theoretical analysis of the characteristic components in the stray flux spectrum of SCIMs as well as experimental validations. A wavelet packet decomposition (WPD) denoising method is proposed for flux-based incipient bearing fault detection. Additionally, a sensor fusion method to efficiently utilize the information from heterogeneous sensor measurements (external magnetic flux and stator current) to achieve higher rotor-related fault detection sensitivity and a higher fault type recognition rate is presented. Instead of using all the characteristic components directly, the proposed fusion method groups the features of several rotor abnormalities and then draws a conclusion on machine health status based on the abnormalities that are present in the machine. Finally, a novel sensor fusion-based rotor vibration observer method is proposed for incipient bearing fault detection. The observer can reject the electrical disturbances from the supply side. Meanwhile, the proposed observer is less affected by the mechanical noise from lousy environment than using vibration-based monitoring.Ph.D

    On the Hierarchy of Distributed Majority Protocols

    Get PDF

    Resource Management in Mobile Edge Computing for Compute-intensive Application

    Full text link
    With current and future mobile applications (e.g., healthcare, connected vehicles, and smart grids) becoming increasingly compute-intensive for many mission-critical use cases, the energy and computing capacities of embedded mobile devices are proving to be insufficient to handle all in-device computation. To address the energy and computing shortages of mobile devices, mobile edge computing (MEC) has emerged as a major distributed computing paradigm. Compared to traditional cloud-based computing, MEC integrates network control, distributed computing, and storage to customizable, fast, reliable, and secure edge services that are closer to the user and data sites. However, the diversity of applications and a variety of user specified requirements (viz., latency, scalability, availability, and reliability) add additional complications to the system and application optimization problems in terms of resource management. In this thesis dissertation, we aim to develop customized and intelligent placement and provisioning strategies that are needed to handle edge resource management problems for different challenging use cases: i) Firstly, we propose an energy-efficient framework to address the resource allocation problem of generic compute-intensive applications, such as Directed Acyclic Graph (DAG) based applications. We design partial task offloading and server selection strategies with the purpose of minimizing the transmission cost. Our experiment and simulation results indicate that partial task offloading provides considerable energy savings, especially for resource-constrained edge systems. ii) Secondly, to address the dynamism edge environments, we propose solutions that integrate Dynamic Spectrum Access (DSA) and Cooperative Spectrum Sensing (CSS) with fine-grained task offloading schemes. Similarly, we show the high efficiency of the proposed strategy in capturing dynamic channel states and enforcing intelligent channel sensing and task offloading decisions. iii) Finally, application-specific long-term optimization frameworks are proposed for two representative applications: a) multi-view 3D reconstruction and b) Deep Neural Network (DNN) inference. Here, in order to eliminate redundant and unnecessary reconstruction processing, we introduce key-frame and resolution selection incorporated with task assignment, quality prediction, and pipeline parallelization. The proposed framework is able to provide a flexible balance between reconstruction time and quality satisfaction. As for DNN inference, a joint resource allocation and DNN partitioning framework is proposed. The outcomes of this research seek to benefit the future distributed computing, smart applications, and data-intensive science communities to build effective, efficient, and robust MEC environments

    It is too hot in here! A performance, energy and heat aware scheduler for Asymmetric multiprocessing processors in embedded systems.

    Get PDF
    Modern architecture present in self-power devices such as mobiles or tablet computers proposes the use of asymmetric processors that allow either energy-efficient or performant computation on the same SoC. For energy efficiency and performance consideration, the asymmetry resides in differences in CPU micro-architecture design and results in diverging raw computing capability. Other components such as the processor memory subsystem also show differences resulting in different memory transaction timing. Moreover, based on a bus-snoop protocol, cache coherency between processors comes with a peculiarity in memory latency depending on the processors operating frequencies. All these differences come with challenging decisions on both application schedulability and processor operating frequencies. In addition, because of the small form factor of such embedded systems, these devices generally cannot afford active cooling systems. Therefore thermal mitigation relies on dynamic software solutions. Current operating systems for embedded systems such as Linux or Android do not consider all these particularities. As such, they often fail to satisfy user expectations of a powerful device with long battery life. To remedy this situation, this thesis proposes a unified approach to deliver high-performance and energy-efficiency computation in each of its flavours, considering the memory subsystem and all computation units available in the system. Performance is maximized even when the device is under heavy thermal constraints. The proposed unified solution is based on accurate models targeting both performance and thermal behaviour and resides at the operating systems kernel level to manage all running applications in a global manner. Particularly, the performance model considers both the computation part and also the memory subsystem of symmetric or asymmetric processors present in embedded devices. The thermal model relies on the accurate physical thermal properties of the device. Using these models, application schedulability and processor frequency scaling decisions to either maximize performance or energy efficiency within a thermal budget are extensively studied. To cover a large range of application behaviour, both models are built and designed using a generative workload that considers fine-grain details of the underlying microarchitecture of the SoC. Therefore, this approach can be derived and applied to multiple devices with little effort. Extended evaluation on real-world benchmarks for high performance and general computing, as well as common applications targeting the mobile and tablet market, show the accuracy and completeness of models used in this unified approach to deliver high performance and energy efficiency under high thermal constraints for embedded devices
    • …
    corecore