Multi-Label Classification Based on the Improved Probabilistic Neural Network

This paper aims to overcome the defects of the existing multi-label classification methods, such as the insufficient use of label correlation and class information. For this purpose, an improved probabilistic neural network for multi-label classification (ML-IPNN) was developed through the following steps. Firstly, the traditional PNN was structurally improved to fit in with multi-label data. Then secondly, a weight matrix was introduced to represent the label correlation and synthetize the information between classes, and the ML-IPNN was trained with the backpropagation mechanism. Finally, the classification results of the ML-IPNN on three common datasets were compared with those of the seven most popular multi-label classification algorithms. The results show that the ML-IPNN outperformed all contrastive algorithms. The research findings brought new light on multi-label classification and the application of artificial neural networks (ANNs)

동적 멀티모달 데이터 학습을 위한 심층 하이퍼네트워크

학위논문 (박사)-- 서울대학교 대학원 : 전기·컴퓨터공학부, 2015. 2. 장병탁.Recent advancements in information communication technology has led the explosive increase of data. Dissimilar to traditional data which are structured and unimodal, in particular, the characteristics of recent data generated from dynamic environments are summarized as high-dimensionality, multimodality, and structurelessness as well as huge-scale size. The learning from non-stationary multimodal data is essential for solving many difficult problems in artificial intelligence. However, despite many successful reports, existing machine learning methods have mainly focused on solving practical problems represented by large-scaled but static databases, such as image classification, tagging, and retrieval. Hypernetworks are a probabilistic graphical model representing empirical distribution, using a hypergraph structure that is a large collection of many hyperedges encoding the associations among variables. This representation allows the model to be suitable for characterizing the complex relationships between features with a population of building blocks. However, since a hypernetwork is represented by a huge combinatorial feature space, the model requires a large number of hyperedges for handling the multimodal large-scale data and thus faces the scalability problem. In this dissertation, we propose a deep architecture of hypernetworks for dealing with the scalability issue for learning from multimodal data with non-stationary properties such as videos, i.e., deep hypernetworks. Deep hypernetworks handle the issues through the abstraction at multiple levels using a hierarchy of multiple hypergraphs. We use a stochastic method based on Monte-Carlo simulation, a graph MC, for efficiently constructing hypergraphs representing the empirical distribution of the observed data. The structure of a deep hypernetwork continuously changes as the learning proceeds, and this flexibility is contrasted to other deep learning models. The proposed model incrementally learns from the data, thus handling the nonstationary properties such as concept drift. The abstract representations in the learned models play roles of multimodal knowledge on data, which are used for the content-aware crossmodal transformation including vision-language conversion. We view the vision-language conversion as a machine translation, and thus formulate the vision-language translation in terms of the statistical machine translation. Since the knowledge on the video stories are used for translation, we call this story-aware vision-language translation. We evaluate deep hypernetworks on large-scale vision-language multimodal data including benmarking datasets and cartoon video series. The experimental results show the deep hypernetworks effectively represent visual-linguistic information abstracted at multiple levels of the data contents as well as the associations between vision and language. We explain how the introduction of a hierarchy deals with the scalability and non-stationary properties. In addition, we present the story-aware vision-language translation on cartoon videos by generating scene images from sentences and descriptive subtitles from scene images. Furthermore, we discuss the meaning of our model for lifelong learning and the improvement direction for achieving human-level artificial intelligence.1 Introduction 1.1 Background and Motivation 1.2 Problems to be Addressed 1.3 The Proposed Approach and its Contribution 1.4 Organization of the Dissertation 2 RelatedWork 2.1 Multimodal Leanring 2.2 Models for Learning from Multimodal Data 2.2.1 Topic Model-Based Multimodal Leanring 2.2.2 Deep Network-based Multimodal Leanring 2.3 Higher-Order Graphical Models 2.3.1 Hypernetwork Models 2.3.2 Bayesian Evolutionary Learning of Hypernetworks 3 Multimodal Hypernetworks for Text-to-Image Retrievals 3.1 Overview 3.2 Hypernetworks for Multimodal Associations 3.2.1 Multimodal Hypernetworks 3.2.2 Incremental Learning of Multimodal Hypernetworks 3.3 Text-to-Image Crossmodal Inference 3.3.1 Representatation of Textual-Visual Data 3.3.2 Text-to-Image Query Expansion 3.4 Text-to-Image Retrieval via Multimodal Hypernetworks 3.4.1 Data and Experimental Settings 3.4.2 Text-to-Image Retrieval Performance 3.4.3 Incremental Learning for Text-to-Image Retrieval 3.5 Summary 4 Deep Hypernetworks for Multimodal Cocnept Learning from Cartoon Videos 4.1 Overview 4.2 Visual-Linguistic Concept Representation of Catoon Videos 4.3 Deep Hypernetworks for Modeling Visual-Linguistic Concepts 4.3.1 Sparse Population Coding 4.3.2 Deep Hypernetworks for Concept Hierarchies 4.3.3 Implication of Deep Hypernetworks on Cognitive Modeling 4.4 Learning of Deep Hypernetworks 4.4.1 Problem Space of Deep Hypernetworks 4.4.2 Graph Monte-Carlo Simulation 4.4.3 Learning of Concept Layers 4.4.4 Incremental Concept Construction 4.5 Incremental Concept Construction from Catoon Videos 4.5.1 Data Description and Parameter Setup 4.5.2 Concept Representation and Development 4.5.3 Character Classification via Concept Learning 4.5.4 Vision-Language Conversion via Concept Learning 4.6 Summary 5 Story-awareVision-LanguageTranslation usingDeepConcept Hiearachies 5.1 Overview 5.2 Vision-Language Conversion as a Machine Translation 5.2.1 Statistical Machine Translation 5.2.2 Vision-Language Translation 5.3 Story-aware Vision-Language Translation using Deep Concept Hierarchies 5.3.1 Story-aware Vision-Language Translation 5.3.2 Vision-to-Language Translation 5.3.3 Language-to-Vision Translation 5.4 Story-aware Vision-Language Translation on Catoon Videos 5.4.1 Data and Experimental Setting 5.4.2 Scene-to-Sentence Generation 5.4.3 Sentence-to-Scene Generation 5.4.4 Visual-Linguistic Story Summarization of Cartoon Videos 5.5 Summary 6 Concluding Remarks 6.1 Summary of the Dissertation 6.2 Directions for Further Research Bibliography 한글초록Docto

Automatic machine learning:methods, systems, challenges

This open access book presents the first comprehensive overview of general methods in Automatic Machine Learning (AutoML), collects descriptions of existing systems based on these methods, and discusses the first international challenge of AutoML systems. The book serves as a point of entry into this quickly-developing field for researchers and advanced students alike, as well as providing a reference for practitioners aiming to use AutoML in their work. The recent success of commercial ML applications and the rapid growth of the field has created a high demand for off-the-shelf ML methods that can be used easily and without expert knowledge. Many of the recent machine learning successes crucially rely on human experts, who select appropriate ML architectures (deep learning architectures or more traditional ML workflows) and their hyperparameters; however the field of AutoML targets a progressive automation of machine learning, based on principles from optimization and machine learning itself

Collected Papers (on various scientific topics), Volume XIII

This thirteenth volume of Collected Papers is an eclectic tome of 88 papers in various fields of sciences, such as astronomy, biology, calculus, economics, education and administration, game theory, geometry, graph theory, information fusion, decision making, instantaneous physics, quantum physics, neutrosophic logic and set, non-Euclidean geometry, number theory, paradoxes, philosophy of science, scientific research methods, statistics, and others, structured in 17 chapters (Neutrosophic Theory and Applications; Neutrosophic Algebra; Fuzzy Soft Sets; Neutrosophic Sets; Hypersoft Sets; Neutrosophic Semigroups; Neutrosophic Graphs; Superhypergraphs; Plithogeny; Information Fusion; Statistics; Decision Making; Extenics; Instantaneous Physics; Paradoxism; Mathematica; Miscellanea), comprising 965 pages, published between 2005-2022 in different scientific journals, by the author alone or in collaboration with the following 110 co-authors (alphabetically ordered) from 26 countries: Abduallah Gamal, Sania Afzal, Firoz Ahmad, Muhammad Akram, Sheriful Alam, Ali Hamza, Ali H. M. Al-Obaidi, Madeleine Al-Tahan, Assia Bakali, Atiqe Ur Rahman, Sukanto Bhattacharya, Bilal Hadjadji, Robert N. Boyd, Willem K.M. Brauers, Umit Cali, Youcef Chibani, Victor Christianto, Chunxin Bo, Shyamal Dalapati, Mario Dalcín, Arup Kumar Das, Elham Davneshvar, Bijan Davvaz, Irfan Deli, Muhammet Deveci, Mamouni Dhar, R. Dhavaseelan, Balasubramanian Elavarasan, Sara Farooq, Haipeng Wang, Ugur Halden, Le Hoang Son, Hongnian Yu, Qays Hatem Imran, Mayas Ismail, Saeid Jafari, Jun Ye, Ilanthenral Kandasamy, W.B. Vasantha Kandasamy, Darjan Karabašević, Abdullah Kargın, Vasilios N. Katsikis, Nour Eldeen M. Khalifa, Madad Khan, M. Khoshnevisan, Tapan Kumar Roy, Pinaki Majumdar, Sreepurna Malakar, Masoud Ghods, Minghao Hu, Mingming Chen, Mohamed Abdel-Basset, Mohamed Talea, Mohammad Hamidi, Mohamed Loey, Mihnea Alexandru Moisescu, Muhammad Ihsan, Muhammad Saeed, Muhammad Shabir, Mumtaz Ali, Muzzamal Sitara, Nassim Abbas, Munazza Naz, Giorgio Nordo, Mani Parimala, Ion Pătrașcu, Gabrijela Popović, K. Porselvi, Surapati Pramanik, D. Preethi, Qiang Guo, Riad K. Al-Hamido, Zahra Rostami, Said Broumi, Saima Anis, Muzafer Saračević, Ganeshsree Selvachandran, Selvaraj Ganesan, Shammya Shananda Saha, Marayanagaraj Shanmugapriya, Songtao Shao, Sori Tjandrah Simbolon, Florentin Smarandache, Predrag S. Stanimirović, Dragiša Stanujkić, Raman Sundareswaran, Mehmet Șahin, Ovidiu-Ilie Șandru, Abdulkadir Șengür, Mohamed Talea, Ferhat Taș, Selçuk Topal, Alptekin Ulutaș, Ramalingam Udhayakumar, Yunita Umniyati, J. Vimala, Luige Vlădăreanu, Ştefan Vlăduţescu, Yaman Akbulut, Yanhui Guo, Yong Deng, You He, Young Bae Jun, Wangtao Yuan, Rong Xia, Xiaohong Zhang, Edmundas Kazimieras Zavadskas, Zayen Azzouz Omar, Xiaohong Zhang, Zhirou Ma.‬‬‬‬‬‬‬