Hierarchical Framework for Interpretable and Probabilistic Model-Based Safe Reinforcement Learning

The difficulty of identifying the physical model of complex systems has led to exploring methods that do not rely on such complex modeling of the systems. Deep reinforcement learning has been the pioneer for solving this problem without the need for relying on the physical model of complex systems by just interacting with it. However, it uses a black-box learning approach that makes it difficult to be applied within real-world and safety-critical systems without providing explanations of the actions derived by the model. Furthermore, an open research question in deep reinforcement learning is how to focus the policy learning of critical decisions within a sparse domain. This paper proposes a novel approach for the use of deep reinforcement learning in safety-critical systems. It combines the advantages of probabilistic modeling and reinforcement learning with the added benefits of interpretability and works in collaboration and synchronization with conventional decision-making strategies. The BC-SRLA is activated in specific situations which are identified autonomously through the fused information of probabilistic model and reinforcement learning, such as abnormal conditions or when the system is near-to-failure. Further, it is initialized with a baseline policy using policy cloning to allow minimum interactions with the environment to address the challenges associated with using RL in safety-critical industries. The effectiveness of the BC-SRLA is demonstrated through a case study in maintenance applied to turbofan engines, where it shows superior performance to the prior art and other baselines.Comment: arXiv admin note: text overlap with arXiv:2206.1343

A Data-driven, High-performance and Intelligent CyberInfrastructure to Advance Spatial Sciences

abstract: In the field of Geographic Information Science (GIScience), we have witnessed the unprecedented data deluge brought about by the rapid advancement of high-resolution data observing technologies. For example, with the advancement of Earth Observation (EO) technologies, a massive amount of EO data including remote sensing data and other sensor observation data about earthquake, climate, ocean, hydrology, volcano, glacier, etc., are being collected on a daily basis by a wide range of organizations. In addition to the observation data, human-generated data including microblogs, photos, consumption records, evaluations, unstructured webpages and other Volunteered Geographical Information (VGI) are incessantly generated and shared on the Internet. Meanwhile, the emerging cyberinfrastructure rapidly increases our capacity for handling such massive data with regard to data collection and management, data integration and interoperability, data transmission and visualization, high-performance computing, etc. Cyberinfrastructure (CI) consists of computing systems, data storage systems, advanced instruments and data repositories, visualization environments, and people, all linked together by software and high-performance networks to improve research productivity and enable breakthroughs that are not otherwise possible. The Geospatial CI (GCI, or CyberGIS), as the synthesis of CI and GIScience has inherent advantages in enabling computationally intensive spatial analysis and modeling (SAM) and collaborative geospatial problem solving and decision making. This dissertation is dedicated to addressing several critical issues and improving the performance of existing methodologies and systems in the field of CyberGIS. My dissertation will include three parts: The first part is focused on developing methodologies to help public researchers find appropriate open geo-spatial datasets from millions of records provided by thousands of organizations scattered around the world efficiently and effectively. Machine learning and semantic search methods will be utilized in this research. The second part develops an interoperable and replicable geoprocessing service by synthesizing the high-performance computing (HPC) environment, the core spatial statistic/analysis algorithms from the widely adopted open source python package – Python Spatial Analysis Library (PySAL), and rich datasets acquired from the first research. The third part is dedicated to studying optimization strategies for feature data transmission and visualization. This study is intended for solving the performance issue in large feature data transmission through the Internet and visualization on the client (browser) side. Taken together, the three parts constitute an endeavor towards the methodological improvement and implementation practice of the data-driven, high-performance and intelligent CI to advance spatial sciences.Dissertation/ThesisDoctoral Dissertation Geography 201

An assessment on the effect of collaborative groups on students’ problem-solving strategies and abilities

This paper reports the use of tools to probe the effectiveness of using small-group interaction to improve problem solving. We find that most students' problem-solving strategies and abilities can be improved by working in short-term, collaborative groups without any other intervention. This is true even for students who have stabilized on a problem-solving strategy and who have stabilized at a problem-solving ability level. Furthermore, we find that even though most students improve by a factor of about 10% in student ability, there are two exceptions: Female students who are classified as pre-formal on a test of logical thinking improve by almost 20% when paired with concrete students; however if two students at the concrete level are paired together no improvement is seen. It has been said that problem solving is the ultimate goal of education (1), and certainly this is true in any chemistry course (2). To be sure, most instructors value this skill and try to instill the ability to solve problems in their students. However, the term "problem solving" means different things to different audiences, from algorithmic problems to complex, open-ended problems that do not have one particular solution. A number of attempts have been made to define problem solving, including "any goal-directed sequence of cognitive operations" (3), and many now agree with the general definition: "what you do when you don't know what to do" (4). Problem solving can be closely allied to critical thinking (5), that other goal of most science courses, in that it involves the application of knowledge to unfamiliar situations. Problem solving also requires the solver to analyze the situation and make decisions about how to proceed, which critical thinking helps. A number of information processing models for problem solving have been developed (6-8) and attempts made to develop uniform theories of problem solving (9). However, many of these studies involve knowledge-lean, closed problems (2) that do not require any specific content knowledge to solve, and that have a specific path to the answer. The truth is that many types of problems exist and there is not one model that will be effective for all categories (10). For example, in teaching science we are ultimately concerned with knowledge-rich problems requiring scientific content knowledge. Studies on problem solving in chemistry have typically revolved around development of strategies derived from research on closed-ended problems, usually pinpointing areas of difficulty that students encounter in specific subject types, such as stoichiometry or equilibrium. A number of studies where students are given strategies or heuristics allowing them resolve word problems in order to produce a numerical answer by application of an algorithm Open-ended problem solving that requires students to use data to make inferences, or to use critical thinking skills, is much more difficult to incorporate into introductory (and even higher level) courses; it is even more difficult to assess, particularly when large numbers of students are involved. Traditional assessment methods, such as examinations and quizzes-including both short answer and multiple choice-give very little insight into the problem-solving process itself. If a student does not have a successful problem-solving strategy, these methods may not allow either the student or the instructor to see where the difficulty lies, or to find ways to improve. While other investigation methods such as think-aloud protocols and videotaped problem-solving sessions (14) give a more nuanced picture of the problem-solving process (15-17), these techniques are time consuming, expensive, and require specific expertise to analyze. These methods are certainly not applicable for the formative assessment of large numbers of students, and while they give a snapshot of a student's problem-solving ability at the time of observation, it is even more difficult to monitor students' development of problem-solving expertise over an extended period. The upshot of all this previous research is that while we know a great deal about the problem-solving process in an abstract environment, we do not in fact have much insight into how students solve many types of scientific problems. Since we lack this information about how students approach problems and how students achieve competence, it is not easy to address the difficulties that students encounter as they develop problemsolving abilities. Indeed, while instructors value problem-solving skills highly, it is often the case that the only explicit instruction that many students are exposed to is the modeling of the skill as the instructor solves problems for students. So we have a situation where a valued skill is often not fully developed in students, even though we implicitly expect that they will become competent problem solvers by the end of the course. The most common assessments give no real insight into student strategies for problem solving, and therefore there is little feedback the instructor can give in terms of how to improve. The traditional assessments also tend to measure and reward algorithmic problem-solving skills rather than critical thinking and application of knowledge to new situations. It seems clear that if we are serious about wanting to incorporate meaningful problem solving into our courses, then we must go beyond the traditional assessments and design systems that allow us t

Promoting transfer and an integrated understanding for pre-service teachers of technology education

The ability of pre-service teachers (PSTs) to transfer learning between subjects and contexts when problem solving is critical for developing their capability as technologists and teachers of technology. However, a growing body of literature suggests this ability is often assumed or over-estimated, and rarely developed explicitly within courses or degree programmes. The nature of the problems tackled within technology are such that solutions draw upon knowledge from a wide range of contexts and subjects, however, the internal organization and structure of institutions and schools tends to compartmentalize rather integrate these. Providing a knowledge base and strategies to enhance PSTs’ awareness of and skills in transferring knowledge may allow for a more integrated understanding to develop. The importance of developing this ability to transfer knowledge is heightened as PSTs will, in turn, be responsible for developing the similar capabilities of their future students. This paper begins by considering problem solving in technology education and some of the issues associated with learning transfer. Thereafter, a framework and strategy for better integrating learning between courses is described and forms the basis for developments in an initial teacher education degree programme for technology education. Provisional data from evaluations and PSTs’ work indicated a positive effect in enhancing their thinking and additional data collected in the form of questionnaires, interviews and course work further illuminate this finding. It is argued that the development framework and approach enhances PSTs’ mental models of teaching technology and offers a significant step forward in promoting skills in the transfer of future learning between subjects; something increasingly critical for 21st century STEM Education

Using Gameplay Patterns to Gamify Learning Experiences

Gamification refers to the use of gaming elements to enhance user experience and engagement in non-gaming systems. In this paper we report the design and implementation of two higher education courses in which ludic elements were used to enhance the quality of the learning experience. A game can be regarded as a system of organised gameplay activities, and a course can be regarded as a system of organised learning activities. Leveraging this analogy, analysing games can provide valuable insights to organise learning activities within a learning experience. We examined a sample of successful commercial games to identify patterns of organisation of gameplay activities that could be applied to a course design. Five patterns were identified: quest structure, strategic open-endedness, non-linear progression, orientation, and challenge-based reward. These patterns were then used to define the instructional design of the courses. As a result, courses were organised as systems of quests that could be tackled through different strategies and in a non-linear way. Students received frequent feedback and were rewarded according to the challenges chosen, based on mechanics common in quest-based games. The courses involved two lecturers and 70 students. Learning journals were used throughout the term to collect data regarding student perceptions on the clarity and usefulness of the gamified approach, level of motivation and engagement in the courses, and relevance of the activities proposed. Results show that students felt challenged by the activities proposed and motivated to complete them, despite considering most activities as difficult. Students adopted different cognitive and behavioural strategies to cope with the courses’ demands. They had to define their own team project, defining the objectives, managing their times and coordinating task completion. The regular and frequent provision of feedback was highly appreciated. A sense of mastery was promoted and final achievement was positively impacted by the gamified strategy

A Domain-Independent Algorithm for Plan Adaptation

The paradigms of transformational planning, case-based planning, and plan debugging all involve a process known as plan adaptation - modifying or repairing an old plan so it solves a new problem. In this paper we provide a domain-independent algorithm for plan adaptation, demonstrate that it is sound, complete, and systematic, and compare it to other adaptation algorithms in the literature. Our approach is based on a view of planning as searching a graph of partial plans. Generative planning starts at the graph's root and moves from node to node using plan-refinement operators. In planning by adaptation, a library plan - an arbitrary node in the plan graph - is the starting point for the search, and the plan-adaptation algorithm can apply both the same refinement operators available to a generative planner and can also retract constraints and steps from the plan. Our algorithm's completeness ensures that the adaptation algorithm will eventually search the entire graph and its systematicity ensures that it will do so without redundantly searching any parts of the graph.Comment: See http://www.jair.org/ for any accompanying file

Designing as Construction of Representations: A Dynamic Viewpoint in Cognitive Design Research

This article presents a cognitively oriented viewpoint on design. It focuses on cognitive, dynamic aspects of real design, i.e., the actual cognitive activity implemented by designers during their work on professional design projects. Rather than conceiving de-signing as problem solving - Simon's symbolic information processing (SIP) approach - or as a reflective practice or some other form of situated activity - the situativity (SIT) approach - we consider that, from a cognitive viewpoint, designing is most appropriately characterised as a construction of representations. After a critical discussion of the SIP and SIT approaches to design, we present our view-point. This presentation concerns the evolving nature of representations regarding levels of abstraction and degrees of precision, the function of external representations, and specific qualities of representation in collective design. Designing is described at three levels: the organisation of the activity, its strategies, and its design-representation construction activities (different ways to generate, trans-form, and evaluate representations). Even if we adopt a "generic design" stance, we claim that design can take different forms depending on the nature of the artefact, and we propose some candidates for dimensions that allow a distinction to be made between these forms of design. We discuss the potential specificity of HCI design, and the lack of cognitive design research occupied with the quality of design. We close our discussion of representational structures and activities by an outline of some directions regarding their functional linkages

Unpacking capabilities underlying design (thinking) process

Engineering graduates must know how to frame and solve non-routine problems. While design classes explicitly teach problem framing and solving, it is lacking throughout much of the rest of the engineering curriculum and is often relegated to capstone classes at the end of the students’ educational experience. This paper explores problem framing and solving through the lens of experiential learning theory. It captures core problem framing and solving approaches from critical, design and systems thinking and concludes with a table of learning outcomes that might be drawn upon in designing an engineering curriculum that more fully develops the problem framing and solving capabilities of its students