Abstract: Modern classrooms are complex socio-technical spaces where multiple heterogeneous legacy tools and novel technologies coexist. The design and the study of such technological ecosystems (or of single technologies that have to integrate into one such ecosystem) is still under-researched, and we lack established conceptual tools to understand how a learning scenario is orchestrated across such heterogeneous sets of resources, and the role and affordances of each technology in it. In this contribution, we present the orchestration matrix, a simple table-like form that can help technology designers and researchers to analyze an existing technological classroom ecosystem from the point of view of teacher orchestration of a learning scenario, or propose new ecosystems that are (hopefully) more effective

Luis P Prieto

Pierre Dillenbourg

CiteSeerX

 
Orchestrated Collaborative Classroom Workshop 2015, June 7, 2015, Gothenburg, Sweden. 
Copyright held by the author(s). 
The Orchestration Matrix:  
A Tool to Design Heterogeneous Classroom Ecosystems 
 
Luis P. Prieto and Pierre Dillenbourg 
CHILI Lab, École Polytechnique Fédérale de Lausanne (EPFL),  Switzerland 
{luis.prieto, pierre.dillenbourg}@epfl.ch 
  
 
Abstract:  Modern classrooms are complex socio-technical spaces where multiple 
heterogeneous legacy tools and novel technologies co xist. The design and the study of such 
technological ecosystems (or of single technologies that have to integrate into one such 
ecosystem) is still under-researched, and we lack established conceptual tools to understand 
how a learning scenario is orchestrated across such heterogeneous sets of resources, and the 
role and affordances of each technology in it. In this contribution, we present the orchestration 
matrix, a simple table-like form that can help technology designers and researchers to analyze 
an existing technological classroom ecosystem from the point of view of teacher orchestration 
of a learning scenario, or propose new ecosystems that are (hopefully) more effective. 
 
Keywords: orchestration, classroom technology, classroom as eco ystem, technology design 
 
Introduction 
Classrooms have always been complex, both in terms of social dynamics and processes. They are public spa es 
with their own history, highly immediate, where multiple activities take place simultaneously (Doyle, 2006). 
And now that technologies are permeating also these spaces, classrooms represent also a complex mix of digital 
and legacy technologies: pen and paper, laptops, traditional blackboards and tablets and interactive whiteboards 
and specialized and general-purpose pieces of software running on each of those devices… the list, and its 
heterogeneity, is almost endless. 
As educational technology researchers, we not only strive to find new ways of using technology to 
learn. We also need to take into account which other technologies already exist there, which may fulfill a similar 
role, or a complementary one, in a sort of ecosystem where the different new and legacy technologies compete 
(or collaborate) for teacher and student usage (Zhao & Frank, 2003). However, there is no standard, structured 
way for researchers (or technology designers) to think about this classroom ecosystem, and how teaching and 
learning within it can be facilitated or hampered. 
In this contribution we present a simple, table-like form to do precisely this kind of analysis, which we 
call the “orchestration matrix”. We have already used it during the design of our own research efforts in o 
classroom technologies, and we hope it can be useful in the context of the ‘orchestrated collaborative classroom’ 
workshop, to understand current and future classroom ecosystems, and to ideate new technologies (or new 
ecosystems of technologies) for them. 
The next section reviews briefly the notion of classroom as a technological ecosystem, and other 
ecosystemic approaches in education and human-computer interaction (HCI). Then, we present the orchestration 
matrix itself, and illustrate its usage with a real xample from our own research. We end the contribution with a 
few conclusions and projections about future work in this line of research. 
Learning Ecosystems vs. Orchestration Ecosystems 
The metaphor of socio-technical spaces (such as the classroom) ‘ecosystems’1 has been present in the fields of 
HCI and education for a long time. Before the turn of the 21st century, Nardi & O’Day (1999) speak of 
‘information ecologies’ as “system of people, practices, technologies, and values in a local environment”, and 
we can even trace the idea further back, to find ecosystemic approaches to interaction design of complex 
industrial settings, such as nuclear plants (Vicente & Rasmussen, 1992), or even further back, when Gibson's 
                                                     
1 Some authors use the word ‘ecologies’ in an equivalent fashion, albeit ‘ecosystem’ seems to be the most 
formally correct, when talking about complex systems of living things and the relationships among them. We 
will use ‘ecosystem’ throughout the paper, except in those cases in which we directly quote other authors. 
 
(1979) notion of ‘affordances’ became widely adopted in HCI. This interest in looking at “technological 
artifacts as ‘species’, whose survival is determined by a dynamically unfolding interaction with other species in 
their shared natural environment”, and the practices that emerge in this interplay of multiple actors and pieces of 
nearby technology, is what Kaptelinin & Bannon (201) denominate ‘the ecological turn in interaction design’. 
In the field of software engineering, (Bosch, 2009) pioneered the concept of “software ecosystems”, as the 
evolution of software product lines into platforms open to third parties (e.g., Android and its application 
marketplace), thus supporting associated human activity (social and commercial) ecosystems. 
This interest in ecosystems as a metaphor can also be f und in the field of educational research, where 
we can find “ecological approaches to classroom management” (Doyle, 2006), or “learning ecologies” (Brown, 
2000) when referring to how the web can change (informal) learning. 
Closer to our topic as educational technology research rs and designers, Zhao & Frank (2003) also use 
this metaphor to talk about technology adoption in schools: basically, within the school ecosystems, computer 
uses are living species, with teachers acting as ‘keystone species’, and technological/pedagogical innovations 
representing the invasion of an exotic species. Luckin (2008) expands on the student side of the equation by 
proposing a “learner-centric ecology of resources”, as the availability of technologies for capturing data, or 
publishing/creating content increases – which can be exploited for learning purposes if designed adequately.  
From the teacher perspective, the notion of the classroom as a complex ecosystem has been retaken 
recently by Dillenbourg et al.'s (2011) notion of ‘rchestration’, as the teacher manages the multiple earning 
processes at multiple social planes and using this ecosystem of resources. They also propose that classroom 
technologies should be designed, not only taking into account how to best encourage the individual/group 
learning processes, but also this class-wide teacher orchestration process (what they call the ‘third circle of 
usability’). Unfortunately, as of today there is no widespread model of what the orchestration process entails for 
the teacher, or what sub-processes may lie within this complex, multi-layered activity.  
Hence, we can see that it may be useful, when designing technologies for the classroom, to consider not 
only our proposed innovation, but also the different resources that teachers and students have at their disposal. 
Also, it may be useful to consider the role that each of these elements can have in the different (teaching and 
learning) practices that will take place within this ecosystem: will they be supported by the different 
technological (or legacy) elements? Will they be hampered by them? Will they collaborate or compete in 
providing this support? 
The Orchestration Matrix 
In order to understand the role of the different technological (or legacy) resources of the classroom in the 
teaching practice, we created the ‘orchestration matrix’, whose basic principle is to map such resources and the 
orchestration activities that occur in the classroom. Although no clear model of orchestration activities exists, 
drawing from existing literature on orchestration ad classroom management in general (Dillenbourg, 2013; 
Doyle, 2006; Prieto, Holenko-Dlab, Abdulwahed, Gutiérrez, & Balid, 2011), we could propose that, at lest, the 
following processes take place (more or less simultaneously) during the run-time orchestration of a lesson: the 
management of the content and learning objectives (e.g., remembering what concepts and curriculum topics to 
cover today, and delivering the corresponding explanations, etc.), the awareness of the ongoing learning 
processes (including the modelling of students’ understanding of the contents), the logistics of the classroom 
(handing out worksheets, logging into a system, moving chairs around for groupwork, etc.), and the self-
regulation and adaptation of the activities (deciding to cut an exercise short due to lack of time, or improvise an 
explanation to address an unexpected misconception). 
 
Table 1: Structure of the orchestration matrix 
 Teacher orchestration processes 
Technologies Content 
management 
Modelling Classroom 
logistics 
Adaptation & 
Regulation 
Tech 1 (How Tech 1 
supports, 
hampers or 
affects content 
management, if 
applicable) 
   
Tech 2     
Tech 3     
…     
 
 
Then, by creating a table with the different classroom technological resources (including legacy, 
hardware and software), we obtain the orchestration matrix (see table 1). Within this table, in each cell we write 
how each element affects the corresponding orchestration process (e.g., if students are using laptops, the 
awareness of students’ progress may only be obtained if the teacher goes behind each student, not fromthe front 
of the class; if the teacher is using a web application to have students do individual exercises, the logistics of 
distributing the task will require the URL to be deliv red somehow to each student). When filling in this table, it 
is important to consider the concrete learning activity taking place; thus, we can have a different orchestration 
matrix for each pedagogical scenario taking place in a concrete classroom ecosystem. The order of the columns 
presented in Table 1 has no specific significance, and can be modified. 
Illustrative case: Designing Tessellated Battleship 
To illustrate this approach, we briefly present how we used the ‘orchestration matrix’ when designing a recent 
classroom experiment in a secondary school classroom, in which a new technology was going to be used. We 
were considering the usage of multiple tabletop computers in the classroom (along with the existing classroom 
ecosystem that included a projector, a teacher computer, etc.), as part of the evaluation of a new educational 
game (called ‘Tessellated Battleship’). Tessellated Battleship is a review activity about geometry, coordinate 
systems and directed numbers (the activity was design d for 11-12 year old students), all topics within their 
mandatory curriculum. This experiment was framed within a larger project in which we explore the effects of 
paper (and especially, augmented paper technologies) in classroom orchestration (1). 
Figure 1 shows the actual orchestration matrix used, with the different technologies in the ecosystem 
(including the tabletops we were introducing, but also existing elements such as the wall display usually present 
in the classroom) represented as rows on the left side. Thinking about the learning scenario we had in m d 
(basically, a collaborative game in which different groups of students try to stop the advance of a virus infection 
using lasers with different geometrical forms, in a similar fashion to the classic battleship game), we filled in the 
role that we envisioned for each technology (e.g., the wall display would help in the classroom logistics by 
displaying the current phase and the overall state of he game; orchestration cards were to be used to facilitate 
disciplinary measures like freezing the game, etc.). 
 
 
Figure 1. Example of orchestration matrix during the ‘Tessellated Battleship’ classroom experiment design 
In this orchestration matrix example we can also see its usage as we were considering several variants 
in the design of the classroom ecosystem, and in the orchestration of the scenario (represented by the arrows 
between different cells). For instance, in the right side of the figure, we can see how we were considering two 
 
different ways of aiding the teacher in the regulation/adaptation of the lesson: either by showing prompts at 
different times on the wall display, or by providing teacher with a ‘cheatsheet’ of prompts and adaptations to be 
performed at different times during the lesson.  
It is worth noting that, in this example, the designer was operating alone, considering especially how 
the different research questions (e.g., what is the role of augmented paper elements on the orchestration load of 
the teacher) could be investigated by modifying different elements in the ecosystem. Later on, the diff rent 
alternatives that emerged while brainstorming with the orchestration matrix were discussed with the teach r, 
helping shape which ecosystem variations were feasible within her particular classroom. 
Conclusions and the future 
We have presented here a new way of organizing information about the technology in a classroom ecosystem, 
aimed at helping researchers and designers better understand their use in classrooms, from the point of view of 
how each of the present pieces of technology affect the orchestration processes. As presented here, the focus is 
on classroom orchestration (often performed by the teacher) in run-time. However, this table form could be 
extended in the future to cover other orchestration processes (such as the activity preparation beforehand or the 
post-hoc evaluation of the learning process). It could also be expanded to include the student view of things 
(e.g., how the different technologies affect the learning processes of students), for example using Bloom's 
(1956) or Krathwohl's (2002) taxonomy of learning activities. Also, this classroom-centric design tool c uld 
also be expanded to take into account the learning that happens across different contexts (e.g., in a field trip, in a 
museum, etc.): these different ecosystems could be designed using a separate table, or by expanding the table to 
include non-classroom technologies and moments. All these additions, and its use and refinement in real other 
research efforts, can help in having a more global understanding of what will happen in the classroom 
ecosystem when all these processes interplay. 
Endnotes 
(1) http://chili.epfl.ch/miocti  
References 
Bloom, B. S. (1956). Taxonomy of educational objectives: The classification of educational goals (1st ed.). 
Bosch, J. (2009). From software product lines to software ecosystems, 111–119. Retrieved from 
http://dl.acm.org/citation.cfm?id=1753235.1753251 
Brown, J. S. (2000). Growing up digital: How the web changes work, education, and the ways people learn. 
Change: The Magazine of Higher Learning, 32(2), 11–20. 
Dillenbourg, P. (2013). Design for classroom orchestration. Computers & Education, 69, 485–492. 
doi:10.1016/j.compedu.2013.04.013 
Dillenbourg, P., Zufferey, G., Alavi, H., Jermann, P. Do-Lenh, S., Bonnard, Q., … Kaplan, F. (2011). 
Classroom orchestration: The third circle of usability. In Proceedings of the International Conference of 
Computer-Supported Collaborative Learning (CSCL2011). 
Doyle, W. (2006). Ecological approaches to classroom management. In C. M. Evertson & C. S. Weinstein 
(Eds.), Handbook of classroom management: Research, practice, and contemporary issues (pp. 97–125). 
Lawrence Erlbaum Associates Publishers. 
Gibson, J. J. (1979). The ecological approach to visual perception. 
Kaptelinin, V., & Bannon, L. J. (2012). Interaction design beyond the product: Creating technology-enhanced 
activity spaces. Human–Computer Interaction. Retrieved from 
http://www.tandfonline.com/doi/abs/10.1080/07370024.2011.646930#.VRQo6nXd-90 
Krathwohl, D. R. (2002). A revision of Bloom’s taxon my: an overview. Theory Into Practice, 41(4), 212–218. 
doi:10.1207/s15430421tip4104_2 
Luckin, R. (2008). The learner centric ecology of resources: a framework for using technology to scaffold 
learning. Computers & Education, 50, 449–462. 
Nardi, B. A., & O’Day, V. L. (1999). Information ecologies: using technology with heart. Retrieved from 
http://dl.acm.org/citation.cfm?id=299579 
Prieto, L. P., Holenko-Dlab, M., Abdulwahed, M., Gutiérrez, I., & Balid, W. (2011). Orchestrating Technology 
Enhanced Learning: a literature review and a conceptual framework. International Journal of Technology-
Enhanced Learning (IJTEL), 3(6), 583–598. 
Vicente, K. J., & Rasmussen, J. (1992). Ecological interface design: theoretical foundations. IEEE Transactions 
on Systems, Man, and Cybernetics, 22(4), 589–606. doi:10.1109/21.156574 
 
Zhao, Y., & Frank, K. A. (2003). Factors affecting technology uses in schools: An ecological perspective. 
Acknowledgments 
This research was supported by a Marie Curie Fellowship within the 7th European Community Framework 
Programme (MIOCTI, FP7-PEOPLE-2012-IEF project no. 327384). 


The Orchestration Matrix: A Tool to Design Heterogeneous Classroom Ecosystems

http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=51F25A3A8B2575C54ED63D54296D63A0?doi=10.1.1.1041.7115&rep=rep1&type=pdf

The Orchestration Matrix: A Tool to Design Heterogeneous Classroom Ecosystems

Abstract

Similar works

Full text

Available Versions

CiteSeerX