PhysQ: A Physics Informed Reinforcement Learning Framework for Building Control

Large-scale integration of intermittent renewable energy sources calls for substantial demand side flexibility. Given that the built environment accounts for approximately 40% of total energy consumption in EU, unlocking its flexibility is a key step in the energy transition process. This paper focuses specifically on energy flexibility in residential buildings, leveraging their intrinsic thermal mass. Building on recent developments in the field of data-driven control, we propose PhysQ. As a physics-informed reinforcement learning framework for building control, PhysQ forms a step in bridging the gap between conventional model-based control and data-intensive control based on reinforcement learning. Through our experiments, we show that the proposed PhysQ framework can learn high quality control policies that outperform a business-as-usual, as well as a rudimentary model predictive controller. Our experiments indicate cost savings of about 9% compared to a business-as-usual controller. Further, we show that PhysQ efficiently leverages prior physics knowledge to learn such policies using fewer training samples than conventional reinforcement learning approaches, making PhysQ a scalable alternative for use in residential buildings. Additionally, the PhysQ control policy utilizes building state representations that are intuitive and based on conventional building models, that leads to better interpretation of the learnt policy over other data-driven controllers.Comment: 15 pages, 4 figures

Hybrid intelligent framework for automated medical learning

This paper investigates the automated medical learning and proposes hybrid intelligent framework, called Hybrid Automated Medical Learning (HAML). The goal is the efficient combination of several intelligent components in order to automatically learn the medical data. Multi agents system is proposed by using distributed deep learning, and knowledge graph for learning medical data. The distributed deep learning is used for efficient learning of the different agents in the system, where the knowledge graph is used for dealing with heterogeneous medical data. To demonstrate the usefulness and accuracy of the HAML framework, intensive simulations on medical data were conducted. A wide range of experiments were conducted to verify the efficiency of the proposed system. Three case studies are discussed in this research, the first case study is related to process mining, and more precisely on the ability of HAML to detect relevant patterns from event medical data. The second case study is related to smart building, and the ability of HAML to recognize the different activities of the patients. The third one is related to medical image retrieval, and the ability of HAML to find the most relevant medical images according to the image query. The results show that the developed HAML achieves good performance compared to the most up-to-date medical learning models regarding both the computational and cost the quality of returned solutions.publishedVersio

Hybrid intelligent framework for automated medical learning

This paper investigates the automated medical learning and proposes hybrid intelligent framework, called Hybrid Automated Medical Learning (HAML). The goal is the efficient combination of several intelligent components in order to automatically learn the medical data. Multi agents system is proposed by using distributed deep learning, and knowledge graph for learning medical data. The distributed deep learning is used for efficient learning of the different agents in the system, where the knowledge graph is used for dealing with heterogeneous medical data. To demonstrate the usefulness and accuracy of the HAML framework, intensive simulations on medical data were conducted. A wide range of experiments were conducted to verify the efficiency of the proposed system. Three case studies are discussed in this research, the first case study is related to process mining, and more precisely on the ability of HAML to detect relevant patterns from event medical data. The second case study is related to smart building, and the ability of HAML to recognize the different activities of the patients. The third one is related to medical image retrieval, and the ability of HAML to find the most relevant medical images according to the image query. The results show that the developed HAML achieves good performance compared to the most up-to-date medical learning models regarding both the computational and cost the quality of returned solutionspublishedVersio

Side-channel attack analysis on in-memory computing architectures

In-memory computing (IMC) systems have great potential for accelerating data-intensive tasks such as deep neural networks (DNNs). As DNN models are generally highly proprietary, the neural network architectures become valuable targets for attacks. In IMC systems, since the whole model is mapped on chip and weight memory read can be restricted, the system acts as a "black box" for customers. However, the localized and stationary weight and data patterns may subject IMC systems to other attacks. In this paper, we propose a side-channel attack methodology on IMC architectures. We show that it is possible to extract model architectural information from power trace measurements without any prior knowledge of the neural network. We first developed a simulation framework that can emulate the dynamic power traces of the IMC macros. We then performed side-channel attacks to extract information such as the stored layer type, layer sequence, output channel/feature size and convolution kernel size from power traces of the IMC macros. Based on the extracted information, full networks can potentially be reconstructed without any knowledge of the neural network. Finally, we discuss potential countermeasures for building IMC systems that offer resistance to these model extraction attack

A Mixed-Response Intelligent Tutoring System Based on Learning from Demonstration

Intelligent Tutoring Systems (ITS) have a significant educational impact on student's learning. However, researchers report time intensive interaction is needed between ITS developers and domain-experts to gather and represent domain knowledge. The challenge is augmented when the target domain is ill-defined. The primary problem resides in often using traditional approaches for gathering domain and tutoring experts' knowledge at design time and conventional methods for knowledge representation built for well-defined domains. Similar to evolving knowledge acquisition approaches used in other fields, we replace this restricted view of ITS knowledge learning merely at design time with an incremental approach that continues training the ITS during run time. We investigate a gradual knowledge learning approach through continuous instructor-student demonstrations. We present a Mixed-response Intelligent Tutoring System based on Learning from Demonstration that gathers and represents knowledge at run time. Furthermore, we implement two knowledge representation methods (Weighted Markov Models and Weighted Context Free Grammars) and corresponding algorithms for building domain and tutoring knowledge-bases at run time. We use students' solutions to cybersecurity exercises as the primary data source for our initial framework testing. Five experiments were conducted using various granularity levels for data representation, multiple datasets differing in content and size, and multiple experts to evaluate framework performance. Using our WCFG-based knowledge representation method in conjunction with a finer data representation granularity level, the implemented framework reached 97% effectiveness in providing correct feedback. The ITS demonstrated consistency when applied to multiple datasets and experts. Furthermore, on average, only 1.4 hours were needed by instructors to build the knowledge-base and required tutorial actions per exercise. Finally, the ITS framework showed suitable and consistent performance when applied to a second domain. These results imply that ITS domain models for ill-defined domains can be gradually constructed, yet generate successful results with minimal effort from instructors and framework developers. We demonstrate that, in addition to providing an effective tutoring performance, an ITS framework can offer: scalability in data magnitude, efficiency in reducing human effort required for building a confident knowledge-base, metacognition in inferring its current knowledge, robustness in handling different pedagogical and tutoring criteria, and portability for multiple domain use

Using ontologies to synchronize change in relational database systems

Ontology is a building block of the semantic Web. Ontology building requires a detailed domain analysis, which in turn requires financial resources, intensive domain knowledge and time. Domain models in industry are frequently stored as relational database schemas in relational databases. An ontology base underlying such schemas can represent concepts and relationships that are present in the domain of discourse. However, with ever increasing demand for wider access and domain coverage, public databases are not static and their schemas evolve over time. Ontologies generated according to these databases have to change to reflect the new situation. Once a database schema is changed, these changes in the schema should also be incorporated in any ontology generated from the database. It is not possible to generate a fresh version of the ontology using the new database schema because the ontology itself may have undergone changes that need to be preserved. To tackle this problem, this paper presents a generic framework that will help to generate and synchronize ontologies with existing data sources. In particular we address the translation between ontologies and database schemas, but our proposal is also sufficiently generic to be used to generate and maintain ontologies based on XML and object oriented databases

Historical SDI, thematic maps and analysis of a complex network of medieval towers (13th-15th century) in the moorish strip

The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Volume XLII-4, 2018 ISPRS TC IV Mid-term Symposium “3D Spatial Information Science – The Engine of Change”, 1–5 October 2018, Delft, The NetherlandsThis work is part of an investigation into the use of GIS for the documentation and comprehension of medieval architectural heritage in the ancient Kingdom of Seville. The research was done in the framework of the project “Sustainable guardianship of cultural heritage through digital BIM and GIS models: contribution to knowledge and social innovation”, an interdisciplinary project focused on the applications of information technology in architectural heritage in Spain. The study case of this paper is located in the Guadalquivir valley during the period between 13th and 15th centuries. It concerns the Moorish Strip site, fortified by the Christian Kingdom of Castile with the aim of creating a barrier with the Moorish Kingdom. Its deteriorated state has led us to create a historical and spatial database in order to contribute to its conservation management plan. Apart from the historical documentation research and the data gathering, intensive fieldwork was also done to collect information about the buildings. In this paper we present a Historical SDI to investigate the hypothesis that the spatial patterns of the Moorish Band obey rules of “inter-visibility” control. Some analysis has been done on the site scale, such as: i) a thematic map of building material; ii) a spatiotemporal analysis; iii) the density of the distribution of towers over the territory; iv) a simulation of the territory visibility from the towers; v) the inter-visibility among towers; iv) thematic maps using attribute values. These analyses permitted us to highlight the need to create a preservation plan that should consider the network visibility system as an important value for heritage interpretation and knowledge.Spain’s Ministry of Economy and Competitiveness HAR2016–78113-