The main objective of meta-level control is to decide what and how much reasoning to do instead of what actions to do. Meta-level control domain involves a large number of processes and actions with terminology that become confusing. For this reason, an ontology to describe the semantic relationships and hierarchical structure of terms related to metacognition is proposed. The ontology was developed based on definitions found in the literature. Experts validated the ontology using a survey. The validation result indicated that the design of an ontology based on the meta-level control domain allows reusing and sharing knowledge defining a common vocabulary

Madera-Doval, Dalia Patricia

English

Universidad de Córdoba Portal de Revistas

Acta Scientiæ Informaticæ                          Publicación Vol. 3 Núm. 3 – Enero a diciembre 2019   A Validated Ontology for Meta-Level Control Domain  Pag. 1 A VALIDATED ONTOLOGY FOR META-LEVEL CONTROL DOMAIN  Dalia Patricia Madera-Doval dmaderadoval@correo.unicordoba.edu.co Departamento de Informática Educativa Universidad de Córdoba, Montería, Colombia    SUMMARY  The main objective of meta-level control is to decide what and how much reasoning to do instead of what actions to do. Meta-level control domain involves a large number of processes and actions with terminology that become confusing. For this reason, an ontology to describe the semantic relationships and hierarchical structure of terms related to metacognition is proposed. The ontology was developed based on definitions found in the literature. Experts validated the ontology using a survey. The validation result indicated that the design of an ontology based on the meta-level control domain allows reusing and sharing knowledge defining a common vocabulary.  Key words:  Meta-level control, ontology, metareasoning, metacognition, reasoning process.   I.  INTRODUCTION  Metareasoning refers to the processes that monitor the progress of reasoning and problem solving activities to regulate the time and effort spent on them [1]. Metareasoning consists of both the meta-level control of computational activities and the introspective monitoring of reasoning. The meta-level is a level of representation of the reasoning of an artificial intelligent agent. The meta-level includes the components, knowledge and mechanisms necessary for a system to monitor and control its own learning and reasoning processes. Introspective monitoring is necessary to gather sufﬁcient information with which to make effective meta-level control decisions.  The meta-level control decides whether or not to invoke a task, which task to invoke, and how much resource to invest in the reasoning process [2]. The main objective of meta-level control tasks is to decide what and how much reasoning to do as opposed to what actions to do [3]. Meta-level control domain involves a large number of processes and actions with terminology that become confusing.  In this sense, studies have developed ontologies in the domain of cognitive control [4], [5]. However, the developed ontologies address the domain from the field of cognitive sciences and psychology, but not from the computer sciences, which is the focus of this study. On the other hand, some works [6]–[8] were developed in order to make cognitive control of the meta-level in the specific domain of failure recognition. The main objective of the research is to create a common language and conceptualization of the hierarchical organization and semantic relations of the actions of meta-level control. In this sense, the contribution of this paper is the presentation of an ontology of meta-level control actions. The paper is structured as follows. Section 2 describes the materials and methods. Section 3 presents the meta-level control ontology. Section 4 shows de evaluation process of the meta-level control ontology. Finally, the conclusions are presented.  II.  MATERIALS AND METHODS  The methodology proposed by Bravo [9] was used for the development of the ontology. The selection was made based on the ease of its implementation in a time - effort relationship. The language used to design the ontology is OWL (Web Ontology Language). The ontology can include descriptions of classes, properties and their instances. In ontology, the formal semantics of OWL specify how to derive its logical consequences. Protégé was used to design and manage the ontology [18]. Protégé is an editor for the design of ontologies, which uses the Jena framework to manage the designed ontology (queries, data reading, data writing and inference engine).  Acta Scientiæ Informaticæ                          Publicación Vol. 3 Núm. 3 – Enero a diciembre 2019   A Validated Ontology for Meta-Level Control Domain  Pag. 2 III.  ONTOLOGY OF ACTIONS FOR METACOGNITIVE CONTROL A. Ontology Requirements Specification According to the selected methodology, in this phase the domain and scope of the ontology is established.  1) Metalevel control Domain The main function of the self-regulation control mechanism is to recommend to the object-level the best computational strategy to resolve a reasoning failure; in this way the meta-level control improves the quality of decisions made by the object-level. The meta-level control decides whether or not to invoke a task, which task to invoke, and how much resource to invest in the reasoning process [2]. ControlActivation and Strategy-Selection are the main control functions in meta-level control ontology. When a reasoning failure is detected then the meta-level control mechanism is activated. The implementation of the failure solution plan is the main action started by ControlActivation. Once a ReasoningFailure is detected and explained by the meta-level, then this metacognitive task assesses the available strategies to be selected and the most appropriate one to address the reasoning failure at the object-level.  2) Scope This ontology is focused on the actions taken in meta-level control and their relationships with other components of meta-level control.  B. Ontology Design This phase consists of term elicitation, ontology modules identification, individual ontology design and formalization.  1) Term elicitation Meta-level control actions were extracted from the specialized literature based on the papers listed in Table 1.  TABLE 1. Papers selected from literature review Ref Paper [1] Meta-Reasoning: Monitoring and Control of Thinking and Reasoning [2] Toward meta-level control of autonomous agents [10] Metareasoning: A Manifesto  [11] A framework for meta-level control in multi-agent systems [12] A review of recent research in metareasoning and metalearning [13] Meta-level control of anytime algorithms with online performance prediction [14] An Analysis of Time-Dependent Planning [15] Monitoring and control of anytime algorithms: A dynamic programming approach [16] Reflection and Action Under Scarce Resources: Theoretical Principles and Empirical Study [6] Ontologies for Reasoning about Failures in AI Systems   2) Meta-level control Ontology The ontology described in this section is based on the meta-level control domain, Fig 1. The ontology consists of 18 classes that include MetacognitiveTask, ControlTask, Judgment and ReasoningTask.   Figure 1. Meta-level control ontology   The meta level control domain represented by classes in ontology is the following.  Acta Scientiæ Informaticæ                          Publicación Vol. 3 Núm. 3 – Enero a diciembre 2019   A Validated Ontology for Meta-Level Control Domain  Pag. 3 a. Metacognitive Task Metacognitive tasks are a type of high-level executive cognitive task that is executed in the metalevel. Introspective monitoring and metalevel control are the main functions of meta reasoning tasks.  b. Control Task The main objective of control tasks is to decide what and how much reasoning to do as opposed to  what actions to do [11], [13], [17]. The control tasks allow AI systems to select effective strategies for metalevel control. Meta level control helps AI systems to decide how to act by choosing computations that contribute most to the action selection problem in the object level.  c. Decision Function The decision functions are components of the decision tasks that aim to generate decision judgments that allow to stop or continue the reasoning process in the object level.  d. Judgment A judgment is the evaluation of evidence to make a decision. Recommendation judgments, Failure judgments and Decision judgments are triggered in the metalevel control.  e. Decision Judgment The decision judgments of the metacognitive control have to do with: i) decisions based on the type of reasoning to be performed (what reasoning to do). This includes for example lists of strategies for reasoning, variables for strategy selection and strategy selection methods; ii) decisions about how much reasoning should be made, including stopping condition for reasoning. The most basic decision of the meta level control is to decide whether the reasoning stops, or reasoning continues until a better plan is generated than the current one.   f. Recommendation judgments The recommendation judgments represent the possible decisions that the meta level makes on a reasoning task that is executed at the object level.  • Abort task. The meta level recommends stopping a task in execution.  • Resume task. This recommendation applies to a reasoning task whose current status is "pause" and there are the necessary conditions to continue its execution.  • Pause task. The meta level recommends pausing a reasoning task when possible problems that may affect the operation of the system are detected.  • Run task. The meta level recommends executing a new reasoning task. This implies the verification of the preconditions of the task, as well as the execution of subtasks if necessary.  • Restructure task. A reasoning task can be composed of one or more subtasks. The meta level recommends restructuring a task when it finds it necessary to replace, delete or add subtasks to a reasoning task.  • Execute action. The metalevel recommends stopping the reasoning process therefore the actions of the current plan are executed.  IV.  EVALUATION OF THE ONTOLOGY  A survey based on Kitchenham and Pfleeger [18] was designed to obtain expert opinion. The survey is based on the Gruber criteria [19] and the questions were focused on how adequate and complete the meta-level control vocabulary is for experts. The survey was composed of the following questions:  Q1: Do you think this ontology is clear? Q2: Is the ontology logically consistent? Q3: In your opinion, could this ontology be extended? Q4: Do you think this ontology is biased? Q5: Do you accept this ontology as a common of the domain?  The survey was sent to 5 experts in the metacognition in computation field who had published papers in this domain. In addition, the answers ware standardized Acta Scientiæ Informaticæ                          Publicación Vol. 3 Núm. 3 – Enero a diciembre 2019   A Validated Ontology for Meta-Level Control Domain  Pag. 4 using the following ordinal scale:1: strongly disagree; 2: disagree; 3: neither agree nor disagree; 4. agree; 5: strongly agree. The results of the surveys are summarized in Table 2.    TABLE 2. Papers selected from literature review Question E1 E2 E3 E4 E5 1 4 4 4 4 4 2 4 5 3 4 4 3 3 4 3 5 4 4 5 4 4 4 5 5 3 4 4 5 4  With regards to the results analysis, taking the Gruber criteria to meta-level control ontology, the results were as follows: A. Clarity: (Q1) Do you think this ontology is clear? The evaluation of the experts about the clarity of the terms and the logical relationships between concepts, resulted in a median of 4 for this question. Experts were asked to suggest adjustments to the relationships between the concepts, if deemed necessary. No relationship or suggestion was added by the experts, which indicates that the design of the ontology is clear. B. Coherence: (Q2) Is the ontology logically consistent?  The results show that four experts agree or strongly agree, one of the experts rated this question with 3 (neither agree nor disagree) and suggested an adjustment in the definition of the RestructureTask, which was accepted. In addition, the consistency and coherence of the ontology as a whole was evaluated taking into account: i) that the granularity of the terms is consistent at all levels of abstraction; and ii) duplication or conflict in concepts was avoided. C. Extendibility: (Q3) In your opinion, could this ontology be extended? The design of an ontology should allow the inclusion of new concepts without the need to redefine existing concepts. In this question two experts answered, "I neither agree nor disagree", while three agree or strongly agree. The two experts proposed some ideas on how to extend the ontology. For example, “the ontology could be extended if we relate it to the concepts described in [6], [8] to specify possible explanations in the decision making of the meta-level”. When new concepts were added as validation, there was no need to change any of the others or the relationships between them, as shown in Fig. 2.   Figure 2. Meta-level control ontology after expert evaluation D. Minimum encoding bias: (Q4) Do you think this ontology is biased? The ontology of meta-level control was designed in the "level of knowledge" avoiding committing it to a particular implementation language that could limit it.  In Q4, the five experts disagreed or strongly disagreed. Protégé and natural language were used to describe this ontology. Protégé provides a reliable framework to describe the knowledge and relationships between the concepts of a domain in a consistent manner. The description of each concept is easy to understand, so we have tried to avoid any bias. Therefore, it was avoided that the experts had a conflict of interest with the authors of the selected papers, in the same way the experts come from different research areas such as Cognitive Acta Scientiæ Informaticæ                          Publicación Vol. 3 Núm. 3 – Enero a diciembre 2019   A Validated Ontology for Meta-Level Control Domain  Pag. 5 Sciences, Education, Psychology and Engineering.  E. Minimal ontological commitment: (Q5) Do you accept this ontology as a common of the domain? The objective was to design a simple, easy-to-use and generic ontology, the ambiguous relationships and concepts were eliminated after analyzing the recommendations of the experts.  Q5 focused on discovering whether experts will use the ontology in their research papers and if they will recommend the ontology to others. In the survey, the average in this question was 4, where four experts said they agreed or strongly agree.  V. CONCLUSIONS  Meta-level control domain involves a large number of processes and actions with terminology that become confusing. In this paper, an ontology composed of the most relevant concepts and relationships related to the meta-level control domain has been presented and evaluated. The ontology concepts were obtained from specialized literature. The ontology described consists of 18 classes that include MetacognitiveTask, ControlTask, Judgment and Reasoning-Task.  The main objective of the research was to create a common language and conceptualization in the domain of meta-level control through the development of an ontology. To achieve the main objective, a rigorous research method was followed in which a systematic mapping of literature was performed, and the ontology was improved and validated by experts in the field through a survey. The results obtained in the survey were very important, since they were provided by experts with research experience in meta-level control. The suggestions of the experts were valuable for the progress in the investigation.  The design of an ontology based on the meta-level control domain allows reusing and sharing knowledge defining a common vocabulary. This vocabulary allows research community to share what other researchers are doing in different parts of the world in the same research area.   VI. ACKNOWLEDGEMENTS  This work was supported in part by a grant from the University of Córdoba, Colombia.   VII. REFERENCES  [1] R. Ackerman and V. A. Thompson, “Meta-Reasoning: Monitoring and Control of Thinking and Reasoning,” Trends in Cognitive Sciences. 2017. [2] D. Dannenhauer, M. Cox, S. Gupta, M. Paisner, and D. Perlis, “Toward Meta-level Control of Autonomous Agents,” in Procedia Computer Science. 5th Annual International Conference on Biologically Inspired Cognitive Architectures, 2014 BICA, 2014, vol. 41, pp. 226–232. [3] M. Cox and A. Raja, “Metareasoning: An introduction,” in In Metareasoning: Thinking about thinking, Cox and Raja, Ed. Cambridge. MA: MIT, 2012, pp. 1–23. [4] D. Badre, “Defining an ontology of cognitive control requires attention to component interactions,” Top. Cogn. Sci., 2011. [5] A. Lenartowicz, D. J. Kalar, E. Congdon, and R. A. Poldrack, “Towards an Ontology of Cognitive Control,” Top. Cogn. Sci., 2010. [6] M. Schmill et al., “Ontologies for Reasoning about Failures in AI Systems,” in in Proceedings from the Workshop on Metareasoning in Agent Based Systems at the Sixth International Joint Conference on Autonomous Agents and Multiagent Sytems, 2007. [7] D. Josyula and S. Fults, “Application of MCL in a dialog agent,” Third Lang. …, no. Figure 1, 2007. [8] M. Schmill et al., “The Metacognitive Loop and Reasoning about Anomalies,” in Metareasoning: Thinking about thinking, M. Cox and A. Raja, Eds. Cambridge, MA: The MIT Press, 2011, pp. 183–198. [9] M. C. B. Contreras, L. F. H. Reyes, and J. A. R. Ortiz, “Methodology for ontology design and construction,” Contaduría y Adm., vol. 0, no. 0, pp. 1–24, 2019. [10] F. Morbini and L. Schubert, “Metareasoning as an integral part of commonsense and autocognitive reasoning,” in AAAI-08 Workshop on Metareasoning, 2008. [11] A. Raja and V. Lesser, “A framework for meta-level control in multi-agent systems,” Auton. Agent. Multi. Agent. Syst., vol. 15, no. 2, pp. 147–196, 2007. [12] M. L. Anderson and T. Oates, “A review of recent research in metareasoning and metalearning,” AI Acta Scientiæ Informaticæ                          Publicación Vol. 3 Núm. 3 – Enero a diciembre 2019   A Validated Ontology for Meta-Level Control Domain  Pag. 6 Magazine. 2007. [13] J. Svegliato, K. H. Wray, and S. Zilberstein, “Meta-level control of anytime algorithms with online performance prediction,” in IJCAI International Joint Conference on Artificial Intelligence, 2018. [14] T. Dean and M. Boddy, “An Analysis of Time-Dependent Planning,” Seventh AAAI Natl. Conf. Artif. Intell., 1988. [15] E. A. Hansen and S. Zilberstein, “Monitoring and control of anytime algorithms: A dynamic programming approach,” Artif. Intell., vol. 126, no. 1–2, pp. 139–157, 2001. [16] E. J. Horvitz, G. F. Cooper, and D. E. Heckerman, “Reflection and Action Under Scarce Resources: Theoretical Principles and Empirical Study,” in IJCAI-89: Proceedings of the Eleventh International Joint Conference on Artificial Intelligence, August 20-25, 1989, Detroit, Michigan, USA, 1989. [17] E. A. Hansen and S. Zilberstein, “Monitoring and control of anytime algorithms: A dynamic programming approach,” Artif. Intell., 2001. [18] B. a. Kitchenham and S. L. Pfleeger, “Principles of survey research part 2: designing a survey,” Softw. Eng. Notes, vol. 27, no. 1, 2002. [19] T. R. Gruber, “Toward principles for the design of ontologies used for knowledge sharing,” Int. J. Hum. - Comput. Stud., vol. 43, no. 5–6, pp. 907–928., 1995.      Para citar este artículo / to cite this article: Madera-Doval, D., (2020). A Validate Ontology for Meta-level Control Domain. Acta Scientiæ Informaticæ 1(1), 26-30.  

A validated ontology for meta-level control domain

https://revistas.unicordoba.edu.co/index.php/asinf/article/download/1977/2411

A validated ontology for meta-level control domain

Abstract

Similar works

Full text

Available Versions

Universidad de Córdoba Portal de Revistas