Learning the Structure of Deep Sparse Graphical Models

Deep belief networks are a powerful way to model complex probability distributions. However, learning the structure of a belief network, particularly one with hidden units, is difficult. The Indian buffet process has been used as a nonparametric Bayesian prior on the directed structure of a belief network with a single infinitely wide hidden layer. In this paper, we introduce the cascading Indian buffet process (CIBP), which provides a nonparametric prior on the structure of a layered, directed belief network that is unbounded in both depth and width, yet allows tractable inference. We use the CIBP prior with the nonlinear Gaussian belief network so each unit can additionally vary its behavior between discrete and continuous representations. We provide Markov chain Monte Carlo algorithms for inference in these belief networks and explore the structures learned on several image data sets.Comment: 20 pages, 6 figures, AISTATS 2010, Revise

Understading Black Boxes: Knowledge Induction From Models

Due to regurations and laws prohibiting uses of private data on customers and their transactions in customer data base, most customer data sets are not easily accessable even in the same organizations. A solutio for this reguatory problems can be providing statistical summary of the data or models induced from the dat, instead of providing raw data sets. The models, however, have limited information on the original raw data set. This study explores possible solutions for these problems. The study uses prediction models from data on credit information of customers provided by a local bank in Seoul, S. Korea. This study suggests approaches in figuring what is inside of the non-rules based models such as regression models or neural network models. The study proposes several rule accumulation algorithms such as (RAA) and a GA-based rule refinement algorithm (GA-RRA) as possible solutions for the problems. The experiments show the performance of the random dataset, RAA, elimination of redundant rules (ERR), and GA-RRA

Інтелектуальна медична система на основі нейронних мереж RESNET

Робота публікується згідно наказу ректора від 29.12.2020 р. №580/од "Про розміщення кваліфікаційних робіт вищої освіти в репозиторії НАУ". Керівник дипломної роботи: д.т.н., проф., завідувач кафедри авіаційних комп’ютерно-інтегрованих комплексів, Синєглазов Віктор МихайловичЗ технічним розвитком засобів автоматизації загострюється потреба у створенні сучасного медично - діагностичного обладнання, яке для лікаря є необхідним вимірювально-інформаційним інструментом отримання інформації про захворювання обстежуваного пацієнта з метою підтримки постановки діагнозу. Використовувані в цій сфері стандартні системи надають лікарю тільки первинну інформацію у вигляді фрагментів даних про досліджуваний орган, що є основою постановки діагностичного висновку. Його формування здійснюється лікарем суб'єктивно (шляхом різних методів когнітивної психології: сприйняття, уявлення, пізнання, розуміння, пояснення, формування рішення і т.п.) в рамках рекомендованих діагностичних мінімумів виявлення того чи іншого захворювання. Спільно з цим, лікар постійно виконує операції управління при жорстких обмеженнях часу на обстеження, це зв’язано з його специфіками, що створює додатковий дискомфорт пацієнту. Все це, а також наявність таких суб'єктивних факторів, як обсяг, інтенсивність, стійкість уваги лікаря, знижують якість проведення обстеження, збільшують його терміни, підвищують ймовірність формування помилкового висновку. Одним з напрямків підвищення ефективності діагностичного обстеження є включення до складу інструментальних засобів окремої комп'ютерної системи підтримки прийняття рішень, що дозволяє реалізувати частину операцій з серії психологічних перетворень, які виконуються лікарем. До них відносяться: виявлення діагностичних ознак захворювання і історії хвороби, облік даних попередніх обстежень, формування висновків за сукупністю знайдених патологій і ознак захворювань у вигляді діагнозу і ін. Автоматизація виконання цих операцій дозволить істотно знизити вплив на якість діагностичних висновків таких людських (суб'єктивних) факторів, як зменшення обсягу уваги, інтенсивності, стійкості тощо, так як ряд операцій з підтримки діагностичних висновків будуть передані машині. Проведений аналіз відомих методик і апаратури показав наступні особливості - як фактори, що впливають на достовірність діагностичного висновку

Uncertainty analysis in product service system: Bayesian network modelling for availability contract

There is an emerging trend of manufacturing companies offering combined products and services to customers as integrated solutions. Availability contracts are an apt instance of such offerings, where product use is guaranteed to customer and is enforced by incentive-penalty schemes. Uncertainties in such an industry setting, where all stakeholders are striving to achieve their respective performance goals and at the same time collaborating intensively, is increased. Understanding through-life uncertainties and their impact on cost is critical to ensure sustainability and profitability of the industries offering such solutions. In an effort to address this challenge, the aim of this research study is to provide an approach for the analysis of uncertainties in Product Service System (PSS) delivered in business-to-business application by specifying a procedure to identify, characterise and model uncertainties with an emphasis to provide decision support and prioritisation of key uncertainties affecting the performance outcomes. The thesis presents a literature review in research areas which are at the interface of topics such as uncertainty, PSS and availability contracts. From this seven requirements that are vital to enhance the understanding and quantification of uncertainties in Product Service System are drawn. These requirements are synthesised into a conceptual uncertainty framework. The framework prescribes four elements, which include identifying a set of uncertainties, discerning the relationships between uncertainties, tools and techniques to treat uncertainties and finally, results that could ease uncertainty management and analysis efforts. The conceptual uncertainty framework was applied to an industry case study in availability contracts, where each of the four elements was realised. This application phase of the research included the identification of uncertainties in PSS, development of a multi-layer uncertainty classification, deriving the structure of Bayesian Network and finally, evaluation and validation of the Bayesian Network. The findings suggest that understanding uncertainties from a system perspective is essential to capture the network aspect of PSS. This network comprises of several stakeholders, where there is increased flux of information and material flows and this could be effectively represented using Bayesian Networks

Context-aware Personal Learning Environment

Research is now shifting away from Virtual Learning Environments (VLEs) and towards the use of the Personal Learning Environment (PLE). A review of a number of PLE architectures are presented in the literature, and while they convey well the concept of a PLE, nevertheless they could best be described as high-level architectures, (sometimes referred to as frameworks in the literature), which focus mainly the functionality of PLEs. In particular, there is little published which gives a detailed designed of a PLE architecture. Moreover, the published work focuses largely on the support for lifelong learning and formal / informal learning; these are two of the main limitations of VLEs. However, this study argues that unexplored potential remains, as there is scope for PLEs to cover more areas. To the best of our knowledge, none of the existing PLE architectures have context-aware systems embedded within their architecture. There is no intelligence in these architectures to filter the e-resources and to predict the user need. In addition, the current PLE architectures are not dynamic; it cannot adopt the user current situation. The user of the current PLE architectures receives too much e-resource. The architecture proposed in this research incorporates a context-aware engine. Thus there is intelligence built into the architecture and thus the PLE system is automatically responsive to the context information. There are three types of sensors in any context-aware system (physical, virtual and logical), and these are the elements of the system that gather the context information. In this research, the emphasis will be on virtual sensors which gather the information from virtual space; virtual space here includes any systems which produce information as a set of results. Thus, the context-aware architecture and the implementation of the context-aware engine are major contributions of the work

Algoritmos de aprendizagem adaptativos para classificadores de redes Bayesianas

Doutoramento em MatemáticaNesta tese consideramos o desenvolvimento de algoritmos adaptativos para classificadores de redes Bayesianas (BNCs) num cenário on-line. Neste cenário os dados são apresentados sequencialmente. O modelo de decisão primeiro faz uma predição e logo este é actualizado com os novos dados. Um cenário on-line de aprendizagem corresponde ao cenário “prequencial” proposto por Dawid. Um algoritmo de aprendizagem num cenário prequencial é eficiente se este melhorar o seu desempenho dedutivo e, ao mesmo tempo, reduzir o custo da adaptação. Por outro lado, em muitas aplicações pode ser difícil melhorar o desempenho e adaptar-se a fluxos de dados que apresentam mudança de conceito. Neste caso, os algoritmos de aprendizagem devem ser dotados com estratégias de controlo e adaptação que garantem o ajuste rápido a estas mudanças. Todos os algoritmos adaptativos foram integrados num modelo conceptual de aprendizagem adaptativo e prequencial para classificação supervisada designado AdPreqFr4SL, o qual tem como objectivo primordial atingir um equilíbrio óptimo entre custo-qualidade e controlar a mudança de conceito. O equilíbrio entre custo-qualidade é abordado através do controlo do viés (bias) e da adaptação do modelo. Em vez de escolher uma única classe de BNCs durante todo o processo, propomo-nos utilizar a classe de classificadores Bayesianos k-dependentes (k-DBCs) e começar com o seu modelo mais simples: o classificador Naïve Bayes (NB) (quando o número máximo de dependências permissíveis entre os atributos, k, é 0). Podemos melhorar o desempenho do NB se reduzirmos o bias produto das restrições de independência. Com este fim, propomo-nos incrementar k gradualmente de forma a que em cada etapa de aprendizagem sejam seleccionados modelos de k-DBCs com uma complexidade crescente que melhor se vai ajustando ao actual montante de dados. Assim podemos evitar os problemas causados por demasiado viés (underfitting) ou demasiada variância (overfiting). Por outro lado, a adaptação da estrutura de um BNC com novos dados implica um custo computacional elevado. Propomo-nos reduzir nos custos da adaptação se, sempre que possível, usarmos os novos dados para adaptar os parâmetros. A estrutura é adaptada só em momentos esporádicos, quando é detectado que a sua adaptação é vital para atingir uma melhoria no desempenho. Para controlar a mudança de conceito, incluímos um método baseado no Controlo de Qualidade Estatístico que tem mostrado ser efectivo na detecção destas mudanças. Avaliamos os algoritmos adaptativos usando a classe de classificadores k-DBC em diferentes problemas artificiais e reais e mostramos as vantagens da sua implementação quando comparado com as versões no adaptativas.This thesis mainly addresses the development of adaptive learning algorithms for Bayesian network classifiers (BNCs) in an on-line leaning scenario. In this scenario data arrives at the learning system sequentially. The actual predictive model must first make a prediction and then update the current model with new data. This scenario corresponds to the Dawid’s prequential approach for statistical validation of models. An efficient adaptive algorithm in a prequential learning framework must be able, above all, to improve its predictive accuracy over time while reducing the cost of adaptation. However, in many real-world situations it may be difficult to improve and adapt to existing changing environments, a problem known as concept drift. In changing environments, learning algorithms should be provided with some control and adaptive mechanisms that effort to adjust quickly to these changes. We have integrated all the adaptive algorithms into an adaptive prequential framework for supervised learning called AdPreqFr4SL, which attempts to handle the cost-performance trade-off and also to cope with concept drift. The cost-quality trade-off is approached through bias management and adaptation control. The rationale is as follows. Instead of selecting a particular class of BNCs and using it during all the learning process, we use the class of k-Dependence Bayesian classifiers and start with the simple Naïve Bayes (by setting the maximum number of allowable attribute dependence k to 0). We can then improve the performance of Naïve Bayes over time if we trade-off the bias reduction which leads to the addition of new attribute dependencies with the variance reduction by accurately estimating the parameters. However, as the learning process advances we should place more focus on bias management. We reduce the bias resulting from the independence assumption by gradually adding dependencies between the attributes over time. To this end, we gradually increase k so that at each learning step we can use a class-model of k-DBCs that better suits the available data. Thus, we can avoid the problems caused by either too much bias (underfitting) or too much variance (overfitting). On the other hand, updating the structure of BNCs with new data is a very costly task. Hence some adaptation control is desirable to decide whether it is inevitable to adapt the structure. We reduce the cost of updating by using new data to primarily adapt the parameters. Only when it is detected that the use of the current structure no longer guarantees the desirable improvement in the performance, do we adapt the structure. To handle concept drift, our framework includes a method based on Statistical Quality Control, which has been demonstrated to be efficient for recognizing concept changes. We experimentally evaluated the AdPreqFr4SL on artificial domains and benchmark problems and show its advantages in comparison against its nonadaptive versions

Theory refinement of bayesian networks with hidden variables

Copyright b