This paper briefly examines the relationship between unsupervised machine learning models, the learning affordances that such models offer, and the mental models of those who use them. We consider the unsupervised models as learning affordances. We use a case study involving unsupervised modelling via commonly used methods such as clustering, to argue that unsupervised models can be used as learning affordances, bychanging participants’ mental models, precisely because the models are unsupervised, and thus potentially lead to learning from unexpected or inexplicit patterns

Bashir, I

Chaudhry, MA

Cukurova, M

du Boulay, B

Jenkins, C

Kent, C

Luckin, R

Pickard, H

English

UCL Discovery

Companion Proceedings 11th International Conference on Learning Analytics & Knowledge (LAK21) Creative Commons License, Attribution - NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0)   On how Unsupervised Machine Learning Can Shape Minds:  a Brief Overview Carmel Kent, Ibrahim Bashir            Hannah Pickard, Chris Jenkins                               EDUCATE Ventures, UK                    ZISHI Adaptive, OSTC, UK                                             carmel@educateventures.com Muhammad Ali Chaudhry, Mutlu Cukurova, Rosemary Luckin            Benedict du Boulay                                 University College London                                            University of Sussex, UK ABSTRACT: This paper briefly examines the relationship between unsupervised machine learning models, the learning affordances that such models offer, and the mental models of those who use them.  We consider the unsupervised models as learning affordances. We use a case study involving unsupervised modelling via commonly used methods such as clustering, to argue that unsupervised models can be used as learning affordances, by changing participants’ mental models, precisely because the models are unsupervised, and thus potentially lead to learning from unexpected or inexplicit patterns. Keywords: Learners’ mental models, unsupervised machine learning, clustering. 1 INTRODUCTION It is well established in the learning literature that presenting learners with a simplified model of whatever is to be understood is a helpful step in learning (Seel, 2017).  This paper makes the argument that machine learning (ML) models, generated by unsupervised methods, can be used as learning affordances to support and shape the development of an organization’s mental models. To make that argument, we briefly outline a case study of a trading and education company (ZISHI/OSTC) which came to learn about their trainees’ and mentees’ behaviour via data analytics.  Before using ML modelling, OSTC’s trainers certainly had a strong sense that different traders traded in different ways and had developed a partial typology of trading behaviours: for example, some traders preferred to work in volatile markets, others in more stable markets.  Based on such intuitions, trainers might suggest different training strategies. However, the typology had remained largely as a tacit understanding of trading behaviour. To better understand the traders’ trading behaviour, we used unsupervised ML methods to arrive at four multidimensional profiles of trading behaviour. In parallel, we asked OSTC’s trainers to generate their own, till then largely tacit, trading behaviour profiles into written descriptions of trading “personas”.  We then compared these data-driven profiles with OSTC’s self-generated qualitative profiling of different kinds of traders.  The data-driven profiles were then used as the predictive basis in a tool to assist OSTC to hire new traders and also formed the basis of a mentoring tool for traders currently in development.  An ML model, whether developed through supervised or unsupervised methods, will always be a simplification from a particular point of view on this complexity.  This simplification and loss of detail is a strength that enables new insight; and even more so when the “point of view” on the complexity is less determined by prior expectations, such as occurs with unsupervised methods. 43 Companion Proceedings 11th International Conference on Learning Analytics & Knowledge (LAK21) Creative Commons License, Attribution - NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0)   2 AFFORDANCES OF MACHINE LEARNING FOR HUMAN LEARNING 2.1 Human Learning In an effective process of learning, a mental model will be stored in the long-term memory of an individual, serving later as a schema (Anderson, 1984), or a script (Preece et al., 1994). Once the model has been created, it exists independently of its sources. Visualizations, images and text can serve as mental affordances (McClelland, 2020) or as we term them – learning affordances –. These affordances may support the functionality of short-term memory (Henderson, & Tallman, 2006) to reduce cognitive load, and therefore assist learning. Our proposition is that unsupervised ML models can do that too, for example, by profiling or by simplifying and reducing the number of dimensions used. 2.2 Unsupervised Machine Learning Raw data are not independent, contextless, self-sufficient repositories of meaning (Fjørtoft & Lai, 2020). Contextualized modelling of data, using statistical methods and, particularly ML, create possibilities for assigning existing semantics to the models, as well as for creating new semantics, which in turn, can be used as “learning affordances”. The concept of affordance describes the complementary relationship between an environment and what it offers or provides to the actors within it (Gibson, 2014). The process of data modelling can start from a phase of feature engineering, in which the existing semantics can be attached to the raw data to shape it in a contextualized way.  In many senses, supervised ML and reinforcement algorithms inherently include the aspiration to mimic and optimize human behaviour.  Unsupervised ML, on the other hand, can reveal factors and behaviours that human guidance might have been preventing us from seeing. Unsupervised ML algorithms (such as clustering, dimension reduction or association techniques), are designed without a top-down supervision component. Thus, unsupervised algorithms are more about identification than recognition, are freer to observe the data, and are freer to learn (Amershi & Conati, 2009). In our case study. Cluster analysis was carried out and revealed four different profiles based on trading behaviour features. This was done to challenge OSTC’s existing profiling mental model of traders that had been used to tailor support. We deliberately did not add to the clustered features any feature having a direct relationship with performance measures (such as profit), for the purpose of making behavioural patterns salient, and to support formative feedback.  2.3 Reflections of the Domain Experts To explore the validity of our hypothesis that the unsupervised model had indeed affected the mental model of the organization, we invited two ZISHI/OSTC managers to compare the mental and the computed models. The interview was semi-structured around Edwards-Leis’s (2012) ‘transitory mental model’, focusing on the model’s effects on language, prediction, diagnosis and supporting their learners. In terms of the unsupervised models’ affordances for human learning, it was noted that the ML model helped the trainers to focus on traders’ behaviours. This contrasts with the trainers’ former focus on traders’ performance, which in many cases reduced to the single figure of profit. The ML models created a handy, bias-reducing shorthand to encapsulate a large number of low-level behavioural variables. These behavioural variables were usually not directly observable by the trainers themselves before the modelling, as developing such a mental model would typically take significant 44 Companion Proceedings 11th International Conference on Learning Analytics & Knowledge (LAK21) Creative Commons License, Attribution - NonCommercial-NoDerivs 3.0 Unported (CC BY-NC-ND 3.0)   cognitive effort and time. In addition, the initial model was regarded as “subjective”, in the sense that it had been derived from long experience of training traders, whereas the model generated with the unsupervised approach was regarded as “objective”, in the sense that it had emerged from the data and was therefore trusted differently.  A related difference was in the number of trading personas vs. the number of clusters.  OSTC’s trainers felt that they were struggling to determine what would be a sufficient set of profiles to cover the field.  By contrast, arriving at four ML clusters rather than some other number was driven by the usual needs for parsimony vs. coverage of the data in unsupervised ML. Another important difference between the models was that the ML model more clearly articulated “how engaged a trader is” compared to the first model as it brought to the fore issues around order activity and diversity.   3 CONCLUSIONS In this paper, we have very briefly described an organizational learning process, designed to help a trading and education organization develop a refined mental model of themselves via the use of unsupervised ML models.  The generated model was used as a learning affordance, not just because it simplified, corrected and highlighted different aspects of an existing mental model, but also because it enabled the creation of new semantics and a new language.  Using the case-study we compared the “before” and “after” models of trading behaviour. The former was subjective and formed tacitly. The latter was created via several ML methods including cluster analysis. We found that four different profiles best fitted the data, and that these had interesting similarities and differences to the “before” (subjective) version of trader personas. We acknowledge that our models were built on limited data, so future work involves remodeling as new and richer trading data become available. Further work is concentrated in designing a mentoring tool, that makes use of the profiles as the resulted profiles. REFERENCES   Amershi, S., and Conati, C. (2009). Combining unsupervised and supervised classification to build user models for exploratory learning environments. Journal of Educational Data Mining, 1(1), 18-71. Anderson, R. C. (1984) The notion of schemata and the educational enterprise: General discussion of the conference. In R. C Anderson, R. J. Spiro & W. E. Montague (Eds.), Schooling and the acquisition of knowledge. (1984). Hillsdale, NJ: Lawrence Erlbaum. Edwards-Leis, C. E. (2012). Challenging learning journeys in the classroom: Using mental model theory to inform how pupils think when they are generating solutions. in T. Ginner, J. Hallstrom, & M. Hulten (Eds.), Technology Education in the 21st Century, PA. Fjørtoft, H., and Lai, M. K. 2020. Affordances of narrative and numerical data: A social-semiotic approach to data use. Studies in Educational Evaluation, 100846. Gibson, J. J. 2014. The ecological approach to visual perception: classic edition. Psychology Press. Henderson, L. and Tallman, J. (2006). Stimulated recall and mental models. Lanham, ML: Scarecrow Press, Inc. McClelland, T. (2020). The mental affordance hypothesis. Mind, 129(514), 401-427. Preece, J., Rogers, Y., Sharp, H., Benyon, D., Holland, S., and Carey, T. (1994). Human computer interaction, NJ: Addison Wesley. Seel, N. M. (2017). Model-based learning: a synthesis of theory and research. Educational Technology Research and Development, 65(4), 931–966.  45 

On how Unsupervised Machine Learning Can Shape Minds: a Brief Overview

https://discovery.ucl.ac.uk/10126481/1/LAK21%20poster.pdf

On how Unsupervised Machine Learning Can Shape Minds: a Brief Overview

Abstract

Similar works

Full text

Available Versions

UCL Discovery