Embed2Sym - scalable neuro-symbolic reasoning via clustered embeddings

Neuro-symbolic reasoning approaches proposed in recent years combine a neural perception component with a symbolic reasoning component to solve a downstream task. By doing so, these approaches can provide neural networks with symbolic reasoning capabilities, improve their interpretability and enable generalization beyond the training task. However, this often comes at the cost of poor training time, with potential scalability issues. In this paper, we propose a scalable neuro-symbolic approach, called Embed2Sym. We complement a two-stage (perception and reasoning) neural network architecture designed to solve a downstream task end-to-end with a symbolic optimisation method for extracting learned latent concepts. Specifically, the trained perception network generates clusters in embedding space that are identified and labelled using symbolic knowledge and a symbolic solver. With the latent concepts identified, a neuro-symbolic model is constructed by combining the perception network with the symbolic knowledge of the downstream task, resulting in a model that is interpretable and transferable. Our evaluation shows that Embed2Sym outperforms state-of-the-art neuro-symbolic systems on benchmark tasks in terms of training time by several orders of magnitude while providing similar if not better accuracy

An energy-based model for neuro-symbolic reasoning on knowledge graphs

Machine learning on graph-structured data has recently become a major topic in industry and research, finding many exciting applications such as recommender systems and automated theorem proving. We propose an energy-based graph embedding algorithm to characterize industrial automation systems, integrating knowledge from different domains like industrial automation, communications and cybersecurity. By combining knowledge from multiple domains, the learned model is capable of making context-aware predictions regarding novel system events and can be used to evaluate the severity of anomalies that might be indicative of, e.g., cybersecurity breaches. The presented model is mappable to a biologically-inspired neural architecture, serving as a first bridge between graph embedding methods and neuromorphic computing - uncovering a promising edge application for this upcoming technology.Comment: Accepted for publication at the 20th IEEE International Conference on Machine Learning and Applications (ICMLA 2021

Neuro-symbolic Reasoning System for Modeling Complex Behaviours.

A neuro-symbolic reasoning strategy for modelling a complex system is presented in which the aim is to forecast, in real time, the physical parameter values of a dynamic environment: the ocean. In situations in which the rules that determine a system are unknown the prediction of the parameter values that determine the characteristic behaviour of the system can be a problematic task. In such a situation it has been found that a case-based reasoning system, in combination with an artifical neural network, can provide a more effective means of performing such predictions than other connectionist or symbolic techniques. The case-based reasoning system incorporates a radial basis function artificial neural network for the case adaptation. The results obtained from experiments, in which the system operated in real time in the oceanographic environment, are presented

The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervision

We propose the Neuro-Symbolic Concept Learner (NS-CL), a model that learns visual concepts, words, and semantic parsing of sentences without explicit supervision on any of them; instead, our model learns by simply looking at images and reading paired questions and answers. Our model builds an object-based scene representation and translates sentences into executable, symbolic programs. To bridge the learning of two modules, we use a neuro-symbolic reasoning module that executes these programs on the latent scene representation. Analogical to human concept learning, the perception module learns visual concepts based on the language description of the object being referred to. Meanwhile, the learned visual concepts facilitate learning new words and parsing new sentences. We use curriculum learning to guide the searching over the large compositional space of images and language. Extensive experiments demonstrate the accuracy and efficiency of our model on learning visual concepts, word representations, and semantic parsing of sentences. Further, our method allows easy generalization to new object attributes, compositions, language concepts, scenes and questions, and even new program domains. It also empowers applications including visual question answering and bidirectional image-text retrieval.Comment: ICLR 2019 (Oral). Project page: http://nscl.csail.mit.edu

Neuro Symbolic Reasoning for Planning: Counterexample Guided Inductive Synthesis using Large Language Models and Satisfiability Solving

Generative large language models (LLMs) with instruct training such as GPT-4 can follow human-provided instruction prompts and generate human-like responses to these prompts. Apart from natural language responses, they have also been found to be effective at generating formal artifacts such as code, plans, and logical specifications from natural language prompts. Despite their remarkably improved accuracy, these models are still known to produce factually incorrect or contextually inappropriate results despite their syntactic coherence - a phenomenon often referred to as hallucination. This limitation makes it difficult to use these models to synthesize formal artifacts that are used in safety-critical applications. Unlike tasks such as text summarization and question-answering, bugs in code, plan, and other formal artifacts produced by LLMs can be catastrophic. We posit that we can use the satisfiability modulo theory (SMT) solvers as deductive reasoning engines to analyze the generated solutions from the LLMs, produce counterexamples when the solutions are incorrect, and provide that feedback to the LLMs exploiting the dialog capability of instruct-trained LLMs. This interaction between inductive LLMs and deductive SMT solvers can iteratively steer the LLM to generate the correct response. In our experiments, we use planning over the domain of blocks as our synthesis task for evaluating our approach. We use GPT-4, GPT3.5 Turbo, Davinci, Curie, Babbage, and Ada as the LLMs and Z3 as the SMT solver. Our method allows the user to communicate the planning problem in natural language; even the formulation of queries to SMT solvers is automatically generated from natural language. Thus, the proposed technique can enable non-expert users to describe their problems in natural language, and the combination of LLMs and SMT solvers can produce provably correct solutions.Comment: 25 pages, 7 figure

Visual Concept-Metaconcept Learning

Humans reason with concepts and metaconcepts: we recognize red and green from visual input; we also understand that they describe the same property of objects (i.e., the color). In this paper, we propose the visual concept-metaconcept learner (VCML) for joint learning of concepts and metaconcepts from images and associated question-answer pairs. The key is to exploit the bidirectional connection between visual concepts and metaconcepts. Visual representations provide grounding cues for predicting relations between unseen pairs of concepts. Knowing that red and green describe the same property of objects, we generalize to the fact that cube and sphere also describe the same property of objects, since they both categorize the shape of objects. Meanwhile, knowledge about metaconcepts empowers visual concept learning from limited, noisy, and even biased data. From just a few examples of purple cubes we can understand a new color purple, which resembles the hue of the cubes instead of the shape of them. Evaluation on both synthetic and real-world datasets validates our claims.Comment: NeurIPS 2019. First two authors contributed equally. Project page: http://vcml.csail.mit.edu

The role of Artificial Intelligence and Distributed computing in IoT applications

[ES] La serie «El rol de la inteligencia artificial y la computación distribuida en las aplicaciones IoT» contiene publicaciones sobre la teoría y aplicaciones de la computación distribuida y la inteligencia artificial en el Internet de las cosas. Prácticamente todas las disciplinas como la ingeniería, las ciencias naturales, la informática y las ciencias de la información, las TIC, la economía, los negocios, el comercio electrónico, el medio ambiente, la salud y las ciencias de la vida están cubiertas. La lista de temas abarca todas las áreas de los sistemas inteligentes modernos y la informática como: inteligencia computacional, soft computing incluyendo redes neuronales, inteligencia social, inteligencia ambiental, sistemas auto-organizados y adaptativos, computación centrada en el ser humano y centrada en el ser humano, sistemas de recomendación, control inteligente, robótica y mecatrónica, incluida la colaboración entre el ser humano y la máquina, paradigmas basados en el conocimiento, paradigmas de aprendizaje, ética de la máquina, análisis inteligente de datos, gestión del conocimiento, agentes inteligentes, toma de decisiones inteligentes y apoyo, seguridad de la red inteligente, gestión de la confianza, entretenimiento interactivo, inteligencia de la Web y multimedia. Las publicaciones en el marco de «El rol de la inteligencia artificial y la computación distribuida en las aplicaciones IoT» son principalmente las actas de seminarios, simposios y conferencias. Abarcan importantes novedades recientes en la materia, tanto de naturaleza fundacional como aplicable. Un importante rasgo característico de la serie es el corto tiempo de publicación. Esto permite una rápida y amplia difusión de los resultados de las investigaciones[EN] The series «The Role of Artificial Intelligence and Distributed Computing in IoT Applications» contains publications on the theory and applications of distributed computing and artificial intelligence in the Internet of Things. Virtually all disciplines such as engineering, natural sciences, computer and information sciences, ICT, economics, business, e-commerce, environment, health and life sciences are covered. The list of topics covers all areas of modern intelligent systems and computer science: computational intelligence, soft computing including neural networks, social intelligence, ambient intelligence, self-organising and adaptive systems, human-centred and people-centred computing, recommendation systems, intelligent control, robotics and mechatronics including human-machine collaboration, knowledge-based paradigms, learning paradigms, machine ethics, intelligent data analysis, knowledge management, intelligent agents, intelligent decision making and support, intelligent network security, trust management, interactive entertainment, web intelligence, and multimedia. The publications in the framework of «The Role of Artificial Intelligence and Distributed Computing in IoT Applications» are mainly the proceedings of seminars, symposia and conferences. They cover important recent developments in the field, whether of a foundational or applicable character. An important feature of the series is the short publication time. This allows for the rapid and wide dissemination of research results

The role of Artificial Intelligence and distributed computing in IoT applications

[EN]The exchange of ideas between scientists and technicians, from both academic and business areas, is essential in order to ease the development of systems which can meet the demands of today’s society. Technology transfer in this field is still a challenge and, for that reason, this type of contributions are notably considered in this compilation. This book brings in discussions and publications concerning the development of innovative techniques of IoT complex problems. The technical program focuses both on high quality and diversity, with contributions in well-established and evolving areas of research. Specifically, 10 chapters were submitted to this book. The editors particularly encouraged and welcomed contributions on AI and distributed computing in IoT applications.Financed by regional government of Castilla y León and FEDER funds