12,291 research outputs found

    Exploiting Symmetry and Heuristic Demonstrations in Off-policy Reinforcement Learning for Robotic Manipulation

    Full text link
    Reinforcement learning demonstrates significant potential in automatically building control policies in numerous domains, but shows low efficiency when applied to robot manipulation tasks due to the curse of dimensionality. To facilitate the learning of such tasks, prior knowledge or heuristics that incorporate inherent simplification can effectively improve the learning performance. This paper aims to define and incorporate the natural symmetry present in physical robotic environments. Then, sample-efficient policies are trained by exploiting the expert demonstrations in symmetrical environments through an amalgamation of reinforcement and behavior cloning, which gives the off-policy learning process a diverse yet compact initiation. Furthermore, it presents a rigorous framework for a recent concept and explores its scope for robot manipulation tasks. The proposed method is validated via two point-to-point reaching tasks of an industrial arm, with and without an obstacle, in a simulation experiment study. A PID controller, which tracks the linear joint-space trajectories with hard-coded temporal logic to produce interim midpoints, is used to generate demonstrations in the study. The results of the study present the effect of the number of demonstrations and quantify the magnitude of behavior cloning to exemplify the possible improvement of model-free reinforcement learning in common manipulation tasks. A comparison study between the proposed method and a traditional off-policy reinforcement learning algorithm indicates its advantage in learning performance and potential value for applications

    Countermeasures for the majority attack in blockchain distributed systems

    Get PDF
    La tecnología Blockchain es considerada como uno de los paradigmas informáticos más importantes posterior al Internet; en función a sus características únicas que la hacen ideal para registrar, verificar y administrar información de diferentes transacciones. A pesar de esto, Blockchain se enfrenta a diferentes problemas de seguridad, siendo el ataque del 51% o ataque mayoritario uno de los más importantes. Este consiste en que uno o más mineros tomen el control de al menos el 51% del Hash extraído o del cómputo en una red; de modo que un minero puede manipular y modificar arbitrariamente la información registrada en esta tecnología. Este trabajo se enfocó en diseñar e implementar estrategias de detección y mitigación de ataques mayoritarios (51% de ataque) en un sistema distribuido Blockchain, a partir de la caracterización del comportamiento de los mineros. Para lograr esto, se analizó y evaluó el Hash Rate / Share de los mineros de Bitcoin y Crypto Ethereum, seguido del diseño e implementación de un protocolo de consenso para controlar el poder de cómputo de los mineros. Posteriormente, se realizó la exploración y evaluación de modelos de Machine Learning para detectar software malicioso de tipo Cryptojacking.DoctoradoDoctor en Ingeniería de Sistemas y Computació

    Learning disentangled speech representations

    Get PDF
    A variety of informational factors are contained within the speech signal and a single short recording of speech reveals much more than the spoken words. The best method to extract and represent informational factors from the speech signal ultimately depends on which informational factors are desired and how they will be used. In addition, sometimes methods will capture more than one informational factor at the same time such as speaker identity, spoken content, and speaker prosody. The goal of this dissertation is to explore different ways to deconstruct the speech signal into abstract representations that can be learned and later reused in various speech technology tasks. This task of deconstructing, also known as disentanglement, is a form of distributed representation learning. As a general approach to disentanglement, there are some guiding principles that elaborate what a learned representation should contain as well as how it should function. In particular, learned representations should contain all of the requisite information in a more compact manner, be interpretable, remove nuisance factors of irrelevant information, be useful in downstream tasks, and independent of the task at hand. The learned representations should also be able to answer counter-factual questions. In some cases, learned speech representations can be re-assembled in different ways according to the requirements of downstream applications. For example, in a voice conversion task, the speech content is retained while the speaker identity is changed. And in a content-privacy task, some targeted content may be concealed without affecting how surrounding words sound. While there is no single-best method to disentangle all types of factors, some end-to-end approaches demonstrate a promising degree of generalization to diverse speech tasks. This thesis explores a variety of use-cases for disentangled representations including phone recognition, speaker diarization, linguistic code-switching, voice conversion, and content-based privacy masking. Speech representations can also be utilised for automatically assessing the quality and authenticity of speech, such as automatic MOS ratings or detecting deep fakes. The meaning of the term "disentanglement" is not well defined in previous work, and it has acquired several meanings depending on the domain (e.g. image vs. speech). Sometimes the term "disentanglement" is used interchangeably with the term "factorization". This thesis proposes that disentanglement of speech is distinct, and offers a viewpoint of disentanglement that can be considered both theoretically and practically

    Exploring the Structure of Scattering Amplitudes in Quantum Field Theory: Scattering Equations, On-Shell Diagrams and Ambitwistor String Models in Gauge Theory and Gravity

    Get PDF
    In this thesis I analyse the structure of scattering amplitudes in super-symmetric gauge and gravitational theories in four dimensional spacetime, starting with a detailed review of background material accessible to a non-expert. I then analyse the 4D scattering equations, developing the theory of how they can be used to express scattering amplitudes at tree level. I go on to explain how the equations can be solved numerically using a Monte Carlo algorithm, and introduce my Mathematica package treeamps4dJAF which performs these calculations. Next I analyse the relation between the 4D scattering equations and on-shell diagrams in N = 4 super Yang-Mills, which provides a new perspective on the tree level amplitudes of the theory. I apply a similar analysis to N = 8 supergravity, developing the theory of on-shell diagrams to derive new Grassmannian integral formulae for the amplitudes of the theory. In both theories I derive a new worldsheet expression for the 4 point one loop amplitude supported on 4D scattering equations. Finally I use 4D ambitwistor string theory to analyse scattering amplitudes in N = 4 conformal supergravity, deriving new worldsheet formulae for both plane wave and non-plane wave amplitudes supported on 4D scattering equations. I introduce a new prescription to calculate the derivatives of on-shell variables with respect to momenta, and I use this to show that certain non-plane wave amplitudes can be calculated as momentum derivatives of amplitudes with plane wave states

    Quantifying the Indirect Effect of Wolves on Aspen in Northern Yellowstone National Park: Evidence for a Trophic Cascade?

    Get PDF
    Yellowstone National Park is renowned for its incredible wildlife, and perhaps the most famous of these species is the gray wolf, which was reintroduced to the Park in the mid-1990s. After reintroduction, it was highly publicized by scientists, journalists, and environmentalists that the wolf both decreased elk density and changed elk behavior in a way that reduced elk effects on plants, a process known as a “trophic cascade.” Aspen, which is eaten by elk in winter, is one species at the forefront of Yellowstone trophic cascade research because it has been in decline across the Park for over a century. However, due to the challenges of measuring trophic cascades, there is continued uncertainty regarding the effects of wolves on aspen in northern Yellowstone. Thus, the purpose of my dissertation was to provide a comprehensive test of a trophic cascade in this system. Specifically, I used 20 years of data on aspen, elk, and wolves in Yellowstone to: 1) clarify annual trends in browsing and height of young aspen (a proxy for regeneration) after wolf reintroduction, 2) assess the influence of wolves scaring elk on aspen (“trait-mediated indirect effects”), and 3) evaluate the effect of wolves killing elk on aspen (“density-mediated indirect effects”). My research suggests that wolves indirectly contributed to increased aspen over story recruitment following their reintroduction by helping to reduce the elk population size, but elk response to the risk of wolf predation did not reduce elk foraging in a way that measurably increased aspen recruitment. Additionally, hunter harvest of elk north of the park was twice as important as wolf predation in causing increased aspen recruitment. However, despite wolves and hunters limiting elk abundance, it is still uncommon for young aspen to grow past peak browsing height (120-cm), indicating that many stands remain vulnerable to elk herbivory nearly 30 years after wolf reintroduction. These results highlight that the strength and mechanism of predator effects on plant communities are context-specific. Thus, using predator reintroduction as a tool for ecosystem restoration without considering the many factors that shape trophic cascades may result in different management and conservation outcomes than intended

    Towards a non-equilibrium thermodynamic theory of ecosystem assembly and development

    Get PDF
    Non-equilibrium thermodynamics has had a significant historic influence on the development of theoretical ecology, even informing the very concept of an ecosystem. Much of this influence has manifested as proposed extremal principles. These principles hold that systems will tend to maximise certain thermodynamic quantities, subject to the other constraints they operate under. A particularly notable extremal principle is the maximum entropy production principle (MaxEPP); that systems maximise their rate of entropy production. However, these principles are not robustly based in physical theory, and suffer from treating complex ecosystems in an extremely coarse manner. To address this gap, this thesis derives a limited but physically justified extremal principle, as well as carrying out a detailed investigation of the impact of non-equilibrium thermodynamic constraints on the assembly of microbial communities. The extremal principle we obtain pertains to the switching between states in simple bistable systems, with switching paths that generate more entropy being favoured. Our detailed investigation into microbial communities involved developing a novel thermodynamic microbial community model, using which we found the rate of ecosystem development to be set by the availability of free-energy. Further investigation was carried out using this model, demonstrating the way that trade-offs emerging from fundamental thermodynamic constraints impact the dynamics of assembling microbial communities. Taken together our results demonstrate that theory can be developed from non-equilibrium thermodynamics, that is both ecologically relevant and physically well grounded. We find that broad extremal principles are unlikely to be obtained, absent significant advances in the field of stochastic thermodynamics, limiting their applicability to ecology. However, we find that detailed consideration of the non-equilibrium thermodynamic mechanisms that impact microbial communities can broaden our understanding of their assembly and functioning.Open Acces

    A suite of quantum algorithms for the shortestvector problem

    Get PDF
    Crytography has come to be an essential part of the cybersecurity infrastructure that provides a safe environment for communications in an increasingly connected world. The advent of quantum computing poses a threat to the foundations of the current widely-used cryptographic model, due to the breaking of most of the cryptographic algorithms used to provide confidentiality, authenticity, and more. Consequently a new set of cryptographic protocols have been designed to be secure against quantum computers, and are collectively known as post-quantum cryptography (PQC). A forerunner among PQC is lattice-based cryptography, whose security relies upon the hardness of a number of closely related mathematical problems, one of which is known as the shortest vector problem (SVP). In this thesis I describe a suite of quantum algorithms that utilize the energy minimization principle to attack the shortest vector problem. The algorithms outlined span the gate-model and continuous time quantum computing, and explore methods of parameter optimization via variational methods, which are thought to be effective on near-term quantum computers. The performance of the algorithms are analyzed numerically, analytically, and on quantum hardware where possible. I explain how the results obtained in the pursuit of solving SVP apply more broadly to quantum algorithms seeking to solve general real-world problems; minimize the effect of noise on imperfect hardware; and improve efficiency of parameter optimization.Open Acces

    Coverage measurements of NB-IoT technology

    Get PDF
    Abstract. The narrowband internet of things (NB-IoT) is a cellular radio access technology that provides seamless connectivity to wireless IoT devices with low latency, low power consumption, and long-range coverage. For long-range coverage, NB-IoT offers a coverage enhancement (CE) mechanism that is achieved by repeating the transmission of signals. Good network coverage is essential to reduce the battery usage and power consumption of IoT devices, while poor network coverage increases the number of repetitions in transmission, which causes high power consumption of IoT devices. The primary objective of this work is to determine the network coverage of NB-IoT technology under the University of Oulu’s 5G test network (5GTN) base station. In this thesis work, measurement results on key performance indicators such as reference signal received power (RSRP), reference signal received quality (RSRQ), received signal strength indicator (RSSI), and signal to noise plus interference (SINR) have been reported. The goal of the measurement is to find out the NB-IoT signal strength at different locations, which are served by the 5GTN cells configured with different parameters, e.g., Tx power levels, antenna tilt angles. The signal strength of NB-IoT technology has been measured at different places under the 5GTN base station in Oulu, Finland. Drive tests have been conducted to measure the signal strength of NB-IoT technology by using the Quectel BG96 module, Qualcomm kDC-5737 dongle and Keysight Nemo Outdoor software. The results have shown the values of RSRP, RSRQ, RSSI, and SINR at different locations within several kilometres of the 5GTN base stations. These values indicate the performance of the network and are used to assess the performance of network services to the end-users. In this work, the overall performance of the network has been checked to verify if network performance meets good signal levels and good network coverage. Relevant details of the NB-IoT technology, the theory behind the signal coverage and comparisons with the measurement results have also been discussed to check the relevance of the measurement results

    How to Be a God

    Get PDF
    When it comes to questions concerning the nature of Reality, Philosophers and Theologians have the answers. Philosophers have the answers that can’t be proven right. Theologians have the answers that can’t be proven wrong. Today’s designers of Massively-Multiplayer Online Role-Playing Games create realities for a living. They can’t spend centuries mulling over the issues: they have to face them head-on. Their practical experiences can indicate which theoretical proposals actually work in practice. That’s today’s designers. Tomorrow’s will have a whole new set of questions to answer. The designers of virtual worlds are the literal gods of those realities. Suppose Artificial Intelligence comes through and allows us to create non-player characters as smart as us. What are our responsibilities as gods? How should we, as gods, conduct ourselves? How should we be gods

    A productive response to legacy system petrification

    Get PDF
    Requirements change. The requirements of a legacy information system change, often in unanticipated ways, and at a more rapid pace than the rate at which the information system itself can be evolved to support them. The capabilities of a legacy system progressively fall further and further behind their evolving requirements, in a degrading process termed petrification. As systems petrify, they deliver diminishing business value, hamper business effectiveness, and drain organisational resources. To address legacy systems, the first challenge is to understand how to shed their resistance to tracking requirements change. The second challenge is to ensure that a newly adaptable system never again petrifies into a change resistant legacy system. This thesis addresses both challenges. The approach outlined herein is underpinned by an agile migration process - termed Productive Migration - that homes in upon the specific causes of petrification within each particular legacy system and provides guidance upon how to address them. That guidance comes in part from a personalised catalogue of petrifying patterns, which capture recurring themes underlying petrification. These steer us to the problems actually present in a given legacy system, and lead us to suitable antidote productive patterns via which we can deal with those problems one by one. To prevent newly adaptable systems from again degrading into legacy systems, we appeal to a follow-on process, termed Productive Evolution, which embraces and keeps pace with change rather than resisting and falling behind it. Productive Evolution teaches us to be vigilant against signs of system petrification and helps us to nip them in the bud. The aim is to nurture systems that remain supportive of the business, that are adaptable in step with ongoing requirements change, and that continue to retain their value as significant business assets
    corecore