The Process of Solving Complex Problems

This article is about Complex Problem Solving (CPS), its history in a variety of research domains (e.g., human problem solving, expertise, decision making, and intelligence), a formal definition and a process theory of CPS applicable to the interdisciplinary field. CPS is portrayed as (a) knowledge acquisition and (b) knowledge application concerning the goal-oriented control of systems that contain many highly interrelated elements (i.e., complex systems). The impact of implicit and explicit knowledge as well as systematic strategy selection on the solution process are discussed, emphasizing the importance of (1) information generation (due to the initial intransparency of the situation), (2) information reduction (due to the overcharging complexity of the problem’s structure), (3) model building (due to the interconnectedness of the variables), (4) dynamic decision making (due to the eigendynamics of the system), and (5) evaluation (due to many, interfering and/or ill-defined goals)

Personen-, System- und Situationsmerkmale als Einflussfaktoren auf den problemlösenden Umgang mit technischen Alltagsgeräten

Um die im Alltag vermehrt vorausgesetzte Fähigkeit zum problemlösenden Umgang mit technischen Geräten im allgemeinbildenden Technikunterricht fördern zu können, widmet sich dieser Beitrag der Frage, wodurch die Interaktion mit einem technischen Gerät überhaupt zu einem Problem wird. Hierzu werden auf Grundlage soziotechnischer Systeme Merkmale auf Ebene der Systeme, der Situation und der ein System bedienenden Personen ausgemacht, von denen ein Einfluss auf die Problemsituation anzunehmen ist. Die empirisch ermittelten Schwierigkeitsindizes können mithilfe multipler Regressionsanalysen zu einem großen Teil durch die Merkmale Komplexität und Transparenz erklärt werden. Von den angenommenen Zusammenhängen zu angrenzenden Konstrukten konnte nur ein verhältnismäßig geringer Teil bestätigt werden.Schlüsselwörter: Problemlösen, Techniknutzung, Alltagsgeräte, soziotechnische Systeme, Merkmale einer Problemsituatio

The process of solving complex problems

Abstract This article is about Complex Problem Solving (CPS), its history in a variety of research domains (e.g., human problem solving, expertise, decision making, and intelligence), a formal definition and a process theory of CPS applicable to the interdisciplinary field. CPS is portrayed as (a) knowledge acquisition and (b) knowledge application concerning the goal-oriented control of systems that contain many highly interrelated elements (i.e., complex systems). The impact of implicit and explicit knowledge as well as systematic strategy selection on the solution process are discussed, emphasizing the importance of (1) information generation (due to the initial intransparency of the situation), (2) information reduction (due to the overcharging complexity of the problem's structure), (3) model building (due to the interconnectedness of the variables), (4) dynamic decision making (due to the eigendynamics of the system), and (5) evaluation (due to many, interfering and/or ill-defined goals)

Assessment of Problem Solving Skills by means of Multiple Complex Systems – Validity of Finite Automata and Linear Dynamic Systems

The assessment of highly domain-general problem solving skills is increasingly important as problem solving is increasingly demanded by modern workplaces (e.g., Autor, Levy, & Murnane, 2003) and increasingly present in international large-scale assessments such as the Programme for International Student Assessment (PISA, e.g., OECD, 2014). This thesis is about the computer-based assessment of problem solving skills based on Multiple Complex Systems (MCS, Greiff, Fischer, Stadler, & Wüstenberg, 2014): The main idea of the MCS approach is to present multiple computer-simulations of “minimally complex” problems (Greiff, 2012) in order to reliably assess certain problem solving skills. In each simulation, the problem solver has to interact with a problem in order to find out (a) how to adequately represent the problem, and (b) how to solve the problem. Up to now, two instances of the MCS approach have been proposed: (1) the MicroDYN approach (based on simulations of linear equation systems) and – more recently, in the second paper of this thesis – (2) the MicroFIN approach (based on simulations of finite state machines). In the current thesis I will elaborate on three research questions regarding the validity (cf. Bühner, 2006) of the MCS approach: (1) its content validity with regard to the concept of complex problem solving; (2) the convergent validity of different instances of the MCS approach; (3) the discriminant validity of the interactive problems of the MCS approach with regard to traditional static measures of reasoning and analytic problem solving skills. Each research question will be addressed in one corresponding paper: In a first paper (Fischer, Greiff, & Funke, 2012) complex problem solving is defined as the goal-oriented control of systems that contain multiple highly interrelated elements. After reviewing some of the major strands of research on complex problem solving (e.g., research on strategy selection, information reduction, intelligence, or on the interplay of implicit and explicit knowledge in the process of complex problem solving) a theoretical framework outlining the most important cognitive processes involved in solving complex problems is derived. The theoretical framework highlights both interactive knowledge acquisition (problem representation) and interactive knowledge application (problem solution) as the two major phases in the process of complex problem solving. Both phases are represented in all current instances of the MCS approach. In a second paper (Greiff, Fischer et al., 2013) the convergent validity of MicroDYN and MicroFIN is investigated (thereby introducing MicroFIN as an alternative to MicroDYN) in order to demonstrate that both instances address the same kind of problem solving skills. Based on a multitrait-multimethod analysis of a sample of university students (N = 339) it is demonstrated that – in addition to method-specific skills – both instances assess a common set of skills (method-general traits) related to (1) representing and (2) solving different kinds of interactive problems. In a regression of science grades on reasoning and the skills assessed by the instances of the MCS approach it is demonstrated that only the method-general representation trait and reasoning have substantial unique contributions. Thus, MicroDYN and MicroFIN seem to address a common set of skills and this set of skills is relevant for explaining school grades in science classes even beyond reasoning. In a third paper (Fischer et al., in press) the discriminant validity of the interactive MicroDYN test is investigated by relating it to reasoning and traditional static measures of Analytic Problem Solving skills (APS) as they were applied in PISA 2003 (OECD, 2004). Besides a common core of problem solving skills addressed by both kinds of tasks (e.g., analyzing complex information about the information given at a certain moment in time) Fischer et al. (in press) expected to find evidence for additional skills that were related to interactive problems only (e.g., systematically generating information and interactively testing hypotheses). Results indicate that MicroDYN shares a lot of variance with APS even after controlling for reasoning in a sample of high-school students (N = 577) and the university student sample (see above). With regard to the explanation of school grades MicroDYN had an incremental value compared to reasoning and APS in the high-school student sample but not significantly so in the university student sample (whereas APS had an incremental value in both samples). Basically these findings highlight both potential and limitations of the MicroDYN approach in its current form. Current instances of the MCS approach address a small set of problem solving skills reliably, but it takes more than these skills to competently solve complex problems. Implications for future research on the assessment of problem solving skills are discussed

Évaluation de la compétence à résoudre un problème en science et technologie dans le contexte québécois de la réforme du renouveau pédagogique à l'aide d'une simulation informatisée

Plus d'une décennie après les débuts de la mise en application du Renouveau pédagogique, c'est-à-dire la réforme du curriculum de formation de l'école québécoise, notre recherche a évalué, à l'aide d'une simulation informatisée, la compétence en résolution de problème d'ordre scientifique et technologique d'élèves de la 5e secondaire d'une cohorte formée avant la mise en application de la réforme, puis d'une autre cohorte formée après. Nos résultats laissent croire que les sujets formés dans le système éducatif québécois du Renouveau pédagogique sont plus compétents à résoudre un problème en science et technologie que ceux qui ne l'ont pas été. Cette recherche est l'une des rares études scientifiques qui permettent d'apporter des éléments de réponse à propos de l'impact de la réforme sur la compétence des élèves. Par ailleurs, elle pourrait également servir aux praticiens et aux chercheurs qui désirent développer des solutions alternatives d'évaluation de la compétence, notamment par l'intermédiaire de simulation informatisée.\ud ______________________________________________________________________________ \ud MOTS-CLÉS DE L’AUTEUR : Compétence, Réforme, Évaluation, Simulation informatisée, Science et technologi

Determinants of the control of dynamic systems: The role of structural knowledge

In educational and organisational settings it has become common practice to use computer-based complex problems that represent dynamic systems for assessment and training purposes. In the interpretation of performance scores and the design of training programs, it is often assumed that the capacity to effectively control the outcomes of a dynamic system depends on the acquisition of structural knowledge. Control performance scores are generally interpreted as evidence of individual differences in the capacity to acquire and utilise structural knowledge and training programs typically try to improve learners‘ mental models of the system of interest. However, a causal relationship between the acquisition of structural knowledge and successful system control has not been established, and some findings suggest that it may be possible to control dynamic systems in the absence of structural knowledge. Therefore, the goals of this project were to determine the conditions that are required to learn how to control dynamic systems and the psychological processes that separate successful from less successful problem solvers in the performance of this task. The main emphasis of this investigation was to clarify the role of structural knowledge in the control of dynamic systems and to identify sources of individual differences in problem solvers‘ capacity to acquire such knowledge and apply it in a goal-orientated application. In a series of studies, a combined experimental and differential approach was adopted to address these goals. This consisted of the experimental manipulation of the task and structural characteristics of complex problems combined with the use of process indicators and external psychometric tests. Study 1 examined whether problem solvers need to directly interact with a dynamic system in order to acquire structural knowledge that is useful for system control. Study 2 examined whether increments in structural knowledge lead to improvements in control performance and whether dynamic systems can be successfully controlled without structural knowledge. Study 3 examined whether the relationship between structural knowledge and control performance is moderated by system complexity. Each of these studies also investigated the role of fluid intelligence in the acquisition and application of knowledge. Additional methodological contributions include the application of Cognitive Load Theory to the design of the instructions used to manipulate structural knowledge, the use of randomly generated control performance scores to evaluate the success of performance and the development of a theoretically driven operationalisation of system complexity. Across the studies, it was found that structural knowledge was a necessary condition of better than random performance and that there was a causal relationship between structural knowledge and control performance. However, the likelihood that structural knowledge would be acquired and utilised was found to be dependent on the complexity of the system. Small increments in system complexity resulted in floor effects on performance. Fluid intelligence was found to play a crucial role in the acquisition and subsequent application of knowledge. Overall, the results indicate that the complexity of the system determines the amount of knowledge that is acquired by the problem solver, which in turn, combined with their intelligence, determines the quality of their control performance

The Relationship between Complex Problem Solving and Intelligence: An Analysis of Three Computer Simulated Scenarios

The gap between field research and laboratory research has always been a problem in psychology. With the introduction of computers into the laboratory, computer simulated tasks allowed the observation of complex problem solving performance in the laboratory with a higher degree of ecological validity than ever before. The main aim of this thesis was to explore the relationship between complex problem solving ability and intelligence by presenting the results of two studies, using over 400 adults. Complex problem solving ability was assessed by performance on three computer simulations: Furniture Factory, Tailorshop, and Forestry System. The theory of fluid and crystallised intelligence guided the selection of cognitive abilities tests. Relationships between broad cognitive abilities including Fluid reasoning (Gf), Acculturation knowledge (Gc), Visual processing (Gv), Quantitative knowledge (Gq), and Processing speed (Gs) with computer simulation performance were explored. Previous research exploring the relationship between complex problem solving and intelligence has led to inconsistent and often contradictory findings. Scoring problems in previous research were addressed and for all three computer simulations, relationships between intelligence and complex problem solving were found. Overall, Gf and Gc explained 20% of the variance in complex problem solving. Correlations between intelligence and complex problem solving increased when specific cognitive abilities tests and aggregated computer simulation scores were employed, rather than the employment of general or factor scores of intelligence and final computer simulation scores. A new aggregated scoring technique (goal achievement) that allowed consistent scoring across different computer simulations was developed. The strongest relationship between intelligence and complex problem solving was observed between goal achievement scores and specific tests of cogn itive abilities such as esoteric analogies and critical reas! oning. There were significant correlations between goal achievement on the Furniture Factory and both esoteric analogies and critical reasoning (r = .37, p < .05, r = .41, p < .05) respectively. Correlations between goal achievement on the Tailorshop and both esoteric analogies and critical reasoning were significant (r = .25, p < .05, r = .29, p < .05) respectively. Correlations between goal achievement on the Forestry System and both esoteric analogies and critical reasoning were also significant (r = .38, p < .05, r = .30, p < .05) respectively. In addition, performance scores on all three computer simulations were correlated with one another. These findings support the application of the Brunswik lens model to complex problem solving research. Negative correlations, albeit rather modest, were observed between neuroticism and complex problem solving performance on the Furniture Factory (r = -.17, p < .05) and the Tailorshop (r = -.21, p < .05), indicating that emoti on may also mediate complex problem solving performance. Results of this thesis may bring individual differences research in this area a step closer to obtaining stable results from which generalisations about complex problem solving tasks can be made

The Effects of Exploratory Learning Environments on Students' Mathematics Achievements

The objective of this dissertation was to advance the knowledge about mathematics instruction regarding the use of exploratory graphical embodiments in Pre-K to College levels. More specifically, the study sought to find out which graphical representations generate the highest learning effect sizes as well as which teaching method is the most supportive when graphical representations are applied. The dissertation is organized into three coherent research studies that correspond to different schooling levels. The primary method of data analysis in this study was meta-analysis supported by synthesis of qualitative and comparative studies. A total of 73 primary studies (N = 9055) from 22 countries conducted over the past 13 years met the inclusion criteria. Out of this pool, 45 studies (N = 7293) were meta-analyzed. The remaining 28 studies (N = 1762) of qualitative or mixed method designs where scrutinized for common themes. The results support the proposed hypothesis that visualization aids mathematics learning. At the primary level, the mean effect size for using exploratory environment was ES = 0.53 (SE = 0.05, 95% CI: 0.42-0.63), the mean effect size for using computerized programs at the grade levels 1-8 was ES = 0.60 (SE = 0.03, 95% CI: 0.53-0.66), and the results of applying congruent research techniques at the high school and college levels revealed an effect size of ES = 0.69 (SE = 0.05, 95% CI: 0.59–0.79). At each of the teaching level, implementing an exploratory environment generated a moderate effect size when compared to traditional teaching methods. These findings support a need for a broader implementation of exploratory learning media to mathematics school practice and provide evidence to formulate a theoretical instructional framework

Analytisches Problemlösen: Validität und Potenzialnutzung

Die Ergebnisse der PISA-Studie 2003 zeigen für Schülerinnen und Schüler in Deutschland geringere Kompetenzen in der Mathematik und den Naturwissenschaften, als aufgrund ihrer (fächerübergreifenden) analytischen Problemlösekompetenz zu erwarten wären. Diese Diskrepanz kann als mangelnde Ausschöpfung des beim analytischen Problemlösen offensichtlich werdenden kognitiven Potenzials zum Aufbau fachbezogener Kompetenzen interpretiert werden (Potenzialausschöpfungshypothese). Um dieses Potenzial zur Förderung fachbezogener Kompetenzen didaktisch nutzen zu können, müssen zuvor zentrale Aspekte der Validität der Modellierung der analytischen Problemlösekompetenz eingehender untersucht werden. In den ersten drei Studien der Arbeit werden daher Aspekte der faktoriellen, diskriminanten, prognostischen und inkrementellen Validität der Modellierung der analytischen Problemlösekompetenz untersucht. In der vierten Studie wird mit der Potenzialnutzungshypothese eine ergänzende Erklärung für die bei PISA gefundenen Ergebnisse geprüft, sowie die Bedeutung motivationaler und emotionaler Faktoren für erfolgreiches analytisches Problemlösen untersucht. Die Ergebnisse deuten auf eine dreidimensionale Struktur der analytischen Problemlösekompetenz hin, und liefern zudem weitere Belege für die empirische Trennbarkeit der analytischen Problemlösekompetenz von fluiden Fähigkeiten, sowohl querschnittlich als auch im Längsschnitt. Sie zeigen ferner, dass analytische Problemlösekompetenz geeignet ist, zukünftige Kompetenzen in der Mathematik und den Naturwissenschaften auch über den Effekt der fluiden Fähigkeiten hinaus vorherzusagen. Zusätzlich konnte im Sinne der Potenzialnutzungshypothese gezeigt werden, dass eine mathematische Kontexteinbettung von sowohl analytischen Problemlöse- als auch von Mathematikaufgaben einen negativen Effekt auf die Leistungen in diesen Aufgaben haben kann. Dies gilt insbesondere für Schülerinnen und Schüler mit ungünstigen motivationalen und emotionalen Merkmalsausprägungen. Die Ergebnisse der Studien werden im Hinblick auf theoretische und praktische Implikationen sowie im Hinblick auf Limitationen und sich daraus ergebende Implikationen für zukünftige Forschung kritisch diskutiert.Results from the PISA study 2003 show lower mean performances in mathematics and science for students in Germany than could be expected compared to their mean performance in (cross-curricular) analytical problem-solving. This discrepancy can be interpreted to the effect that students have a cognitive potential, which may not be fully exploited for the development of subject-related competencies (cognitive potential exploitation hypothesis). In order to make use of this potential to foster subject-related competencies, key aspects of the validity of the assessment of analytical problem solving competence have to be studied more profoundly. In the first three studies of this work, aspects of factorial, discriminant, prognostic, and incremental validity of the assessment of analytical problem solving are examined. In the fourth study, an additional explanation of the PISA results is introduced, the cognitive potential application hypothesis, and the significance of motivational and emotional factors for successful analytical problem solving are explored. The results point to a three-dimensional structure of analytical problem solving competence, and provide further evidence of the empirical distinction of analytical problem solving competence and fluid intelligence, in cross-sectional as well as in longitudinal perspective. Furthermore, the results show that analytical problem solving competence can predict future competencies in mathematics and science over and above fluid intelligence. In accordance with the cognitive potential application hypothesis, it was also demonstrated that a mathematical context embedding of analytical problem solving- as well as mathematics-tasks can have a negative effect on students’ performance in those tasks. This is especially pronounced for students with unfavorable motivational and emotional dispositions. The results of all four studies are critically discussed with regard to theoretical and practical implications as well as limitations and consequences for future research