8,548 research outputs found

    Modular lifelong machine learning

    Get PDF
    Deep learning has drastically improved the state-of-the-art in many important fields, including computer vision and natural language processing (LeCun et al., 2015). However, it is expensive to train a deep neural network on a machine learning problem. The overall training cost further increases when one wants to solve additional problems. Lifelong machine learning (LML) develops algorithms that aim to efficiently learn to solve a sequence of problems, which become available one at a time. New problems are solved with less resources by transferring previously learned knowledge. At the same time, an LML algorithm needs to retain good performance on all encountered problems, thus avoiding catastrophic forgetting. Current approaches do not possess all the desired properties of an LML algorithm. First, they primarily focus on preventing catastrophic forgetting (Diaz-Rodriguez et al., 2018; Delange et al., 2021). As a result, they neglect some knowledge transfer properties. Furthermore, they assume that all problems in a sequence share the same input space. Finally, scaling these methods to a large sequence of problems remains a challenge. Modular approaches to deep learning decompose a deep neural network into sub-networks, referred to as modules. Each module can then be trained to perform an atomic transformation, specialised in processing a distinct subset of inputs. This modular approach to storing knowledge makes it easy to only reuse the subset of modules which are useful for the task at hand. This thesis introduces a line of research which demonstrates the merits of a modular approach to lifelong machine learning, and its ability to address the aforementioned shortcomings of other methods. Compared to previous work, we show that a modular approach can be used to achieve more LML properties than previously demonstrated. Furthermore, we develop tools which allow modular LML algorithms to scale in order to retain said properties on longer sequences of problems. First, we introduce HOUDINI, a neurosymbolic framework for modular LML. HOUDINI represents modular deep neural networks as functional programs and accumulates a library of pre-trained modules over a sequence of problems. Given a new problem, we use program synthesis to select a suitable neural architecture, as well as a high-performing combination of pre-trained and new modules. We show that our approach has most of the properties desired from an LML algorithm. Notably, it can perform forward transfer, avoid negative transfer and prevent catastrophic forgetting, even across problems with disparate input domains and problems which require different neural architectures. Second, we produce a modular LML algorithm which retains the properties of HOUDINI but can also scale to longer sequences of problems. To this end, we fix the choice of a neural architecture and introduce a probabilistic search framework, PICLE, for searching through different module combinations. To apply PICLE, we introduce two probabilistic models over neural modules which allows us to efficiently identify promising module combinations. Third, we phrase the search over module combinations in modular LML as black-box optimisation, which allows one to make use of methods from the setting of hyperparameter optimisation (HPO). We then develop a new HPO method which marries a multi-fidelity approach with model-based optimisation. We demonstrate that this leads to improvement in anytime performance in the HPO setting and discuss how this can in turn be used to augment modular LML methods. Overall, this thesis identifies a number of important LML properties, which have not all been attained in past methods, and presents an LML algorithm which can achieve all of them, apart from backward transfer

    Representation Learning With Hidden Unit Clustering For Low Resource Speech Applications

    Full text link
    The representation learning of speech, without textual resources, is an area of significant interest for many low resource speech applications. In this paper, we describe an approach to self-supervised representation learning from raw audio using a hidden unit clustering (HUC) framework. The input to the model consists of audio samples that are windowed and processed with 1-D convolutional layers. The learned "time-frequency" representations from the convolutional neural network (CNN) module are further processed with long short term memory (LSTM) layers which generate a contextual vector representation for every windowed segment. The HUC framework, allowing the categorization of the representations into a small number of phoneme-like units, is used to train the model for learning semantically rich speech representations. The targets consist of phoneme-like pseudo labels for each audio segment and these are generated with an iterative k-means algorithm. We explore techniques that improve the speaker invariance of the learned representations and illustrate the effectiveness of the proposed approach on two settings, i) completely unsupervised speech applications on the sub-tasks described as part of the ZeroSpeech 2021 challenge and ii) semi-supervised automatic speech recognition (ASR) applications on the TIMIT dataset and on the GramVaani challenge Hindi dataset. In these experiments, we achieve state-of-art results for various ZeroSpeech tasks. Further, on the ASR experiments, the HUC representations are shown to improve significantly over other established benchmarks based on Wav2vec, HuBERT and Best-RQ

    Benefits and Risks Measurement Model : In different managerial positions during RPA implementation

    Get PDF
    The emergent technologies have always been used for ameliorating business processes. Automation solutions and use of AI tools have increased the creation of more efficient and reliable processes. One of these automation solutions is robotic process automation that is used to automate software-based processes. Use of software systems have previously required human attention to work on different graphic user interfaces to read, fill, and combine data among the systems. Robotic process automation solutions offer a new way of working by automating repetitive manual tasks and creating more time for human work force to be creative. Implementation of modern technologies has its benefits and risks on various levels of the organisation. Change and adoption management is crucial to be up to date so the value from the implementation process can be captured and risks avoided in every position. Various measurement metrics and tools can help the management to monitor the ongoing process and to evaluate the outcome of the implementation. The aim of this thesis is to categorise the perceived benefits and risks of robotic process automation and the measurement metrics and tools to monitor them in different managerial positions during the implementation. The methods used for creating this categorised model are literature review as secondary data to create a theoretical model and a survey to industry experts to gather primary data to agree or disagree with the created model. The secondary data was researched to gather knowledge about different known benefits and risks models while trying to position their categorisation into project manager, developer, and customer service agent positions. The primary data gathered from the experts on the same managerial positions was used to strengthen the theoretical model. As conclusion, the final model represents the perceived benefits and risks under the managerial positions and measurement metrics and tools in general. The results show that the benefits and risks of robotic automation process implementation can be categorised under managerial positions to help the management to ensure the full value capture while ameliorating the business processes with the automation. Taking the categorised benefit dimensions and risk concerns into account during the implementation’s change management can help the organisations to be ready for the more advanced artificial intelligence solutions as the robotic process automation is referred as a steppingstone towards the forthcoming technological revolution

    Tourism and heritage in the Chornobyl Exclusion Zone

    Get PDF
    Tourism and Heritage in the Chornobyl Exclusion Zone (CEZ) uses an ethnographic lens to explore the dissonances associated with the commodification of Chornobyl's heritage. The book considers the role of the guides as experience brokers, focusing on the synergy between tourists and guides in the performance of heritage interpretation. Banaszkiewicz proposes to perceive tour guides as important actors in the bottom-up construction of heritage discourse contributing to more inclusive and participatory approach to heritage management. Demonstrating that the CEZ has been going through a dynamic transformation into a mass tourism attraction, the book offers a critical reflection on heritagisation as a meaning-making process in which the resources of the past are interpreted, negotiated, and recognised as a valuable legacy. Applying the concepts of dissonant heritage to describe the heterogeneous character of the CEZ, the book broadens the interpretative scope of dark tourism which takes on a new dimension in the context of the war in Ukraine. Tourism and Heritage in the Chornobyl Exclusion Zone argues that post-disaster sites such as Chornobyl can teach us a great deal about the importance of preserving cultural and natural heritage for future generations. The book will be of interest to academics and students who are engaged in the study of heritage, tourism, memory, disasters and Eastern Europe

    Feature Papers in Compounds

    Get PDF
    This book represents a collection of contributions in the field of the synthesis and characterization of chemical compounds, natural products, chemical reactivity, and computational chemistry. Among its contents, the reader will find high-quality, peer-reviewed research and review articles that were published in the open access journal Compounds by members of the Editorial Board and the authors invited by the Editorial Office and Editor-in-Chief

    Intelligent computing : the latest advances, challenges and future

    Get PDF
    Computing is a critical driving force in the development of human civilization. In recent years, we have witnessed the emergence of intelligent computing, a new computing paradigm that is reshaping traditional computing and promoting digital revolution in the era of big data, artificial intelligence and internet-of-things with new computing theories, architectures, methods, systems, and applications. Intelligent computing has greatly broadened the scope of computing, extending it from traditional computing on data to increasingly diverse computing paradigms such as perceptual intelligence, cognitive intelligence, autonomous intelligence, and human computer fusion intelligence. Intelligence and computing have undergone paths of different evolution and development for a long time but have become increasingly intertwined in recent years: intelligent computing is not only intelligence-oriented but also intelligence-driven. Such cross-fertilization has prompted the emergence and rapid advancement of intelligent computing

    Computational Geometry Contributions Applied to Additive Manufacturing

    Get PDF
    This Doctoral Thesis develops novel articulations of Computation Geometry for applications on Additive Manufacturing, as follows: (1) Shape Optimization in Lattice Structures. Implementation and sensitivity analysis of the SIMP (Solid Isotropic Material with Penalization) topology optimization strategy. Implementation of a method to transform density maps, resulting from topology optimization, into surface lattice structures. Procedure to integrate material homogenization and Design of Experiments (DOE) to estimate the stress/strain response of large surface lattice domains. (2) Simulation of Laser Metal Deposition. Finite Element Method implementation of a 2D nonlinear thermal model of the Laser Metal Deposition (LMD) process considering temperaturedependent material properties, phase change and radiation. Finite Element Method implementation of a 2D linear transient thermal model for a metal substrate that is heated by the action of a laser. (3) Process Planning for Laser Metal Deposition. Implementation of a 2.5D path planning method for Laser Metal Deposition. Conceptualization of a workflow for the synthesis of the Reeb Graph for a solid region in ℝ" denoted by its Boundary Representation (B-Rep). Implementation of a voxel-based geometric simulator for LMD process. Conceptualization, implementation, and validation of a tool for the minimization of the material over-deposition at corners in LMD. Implementation of a 3D (non-planar) slicing and path planning method for the LMD-manufacturing of overhanging features in revolute workpieces. The aforementioned contributions have been screened by the international scientific community via Journal and Conference submissions and publications

    Learning disentangled speech representations

    Get PDF
    A variety of informational factors are contained within the speech signal and a single short recording of speech reveals much more than the spoken words. The best method to extract and represent informational factors from the speech signal ultimately depends on which informational factors are desired and how they will be used. In addition, sometimes methods will capture more than one informational factor at the same time such as speaker identity, spoken content, and speaker prosody. The goal of this dissertation is to explore different ways to deconstruct the speech signal into abstract representations that can be learned and later reused in various speech technology tasks. This task of deconstructing, also known as disentanglement, is a form of distributed representation learning. As a general approach to disentanglement, there are some guiding principles that elaborate what a learned representation should contain as well as how it should function. In particular, learned representations should contain all of the requisite information in a more compact manner, be interpretable, remove nuisance factors of irrelevant information, be useful in downstream tasks, and independent of the task at hand. The learned representations should also be able to answer counter-factual questions. In some cases, learned speech representations can be re-assembled in different ways according to the requirements of downstream applications. For example, in a voice conversion task, the speech content is retained while the speaker identity is changed. And in a content-privacy task, some targeted content may be concealed without affecting how surrounding words sound. While there is no single-best method to disentangle all types of factors, some end-to-end approaches demonstrate a promising degree of generalization to diverse speech tasks. This thesis explores a variety of use-cases for disentangled representations including phone recognition, speaker diarization, linguistic code-switching, voice conversion, and content-based privacy masking. Speech representations can also be utilised for automatically assessing the quality and authenticity of speech, such as automatic MOS ratings or detecting deep fakes. The meaning of the term "disentanglement" is not well defined in previous work, and it has acquired several meanings depending on the domain (e.g. image vs. speech). Sometimes the term "disentanglement" is used interchangeably with the term "factorization". This thesis proposes that disentanglement of speech is distinct, and offers a viewpoint of disentanglement that can be considered both theoretically and practically

    Modeling, Simulation and Data Processing for Additive Manufacturing

    Get PDF
    Additive manufacturing (AM) or, more commonly, 3D printing is one of the fundamental elements of Industry 4.0. and the fourth industrial revolution. It has shown its potential example in the medical, automotive, aerospace, and spare part sectors. Personal manufacturing, complex and optimized parts, short series manufacturing and local on-demand manufacturing are some of the current benefits. Businesses based on AM have experienced double-digit growth in recent years. Accordingly, we have witnessed considerable efforts in developing processes and materials in terms of speed, costs, and availability. These open up new applications and business case possibilities all the time, which were not previously in existence. Most research has focused on material and AM process development or effort to utilize existing materials and processes for industrial applications. However, improving the understanding and simulation of materials and AM process and understanding the effect of different steps in the AM workflow can increase the performance even more. The best way of benefit of AM is to understand all the steps related to that—from the design and simulation to additive manufacturing and post-processing ending the actual application.The objective of this Special Issue was to provide a forum for researchers and practitioners to exchange their latest achievements and identify critical issues and challenges for future investigations on “Modeling, Simulation and Data Processing for Additive Manufacturing”. The Special Issue consists of 10 original full-length articles on the topic
    • 

    corecore