Mate Marote is a flexible framework for large-scale educational interventions that include games developed, based on aspects from cognitive sciences as an effective way of learning. MemoMarote is a computer game designed for training working memory. The pilot study of this platform was an intervention in La Rioja province (Argentina). In this article we analyze the registered statistics of use of the unsupervised version of the game. We found that playing activity had a peak when recently installed, decreasing during the first year and observing an increase when starting school classes in 2013. Analyzing the usage statistics we observed that players have a poor performance, with more incorrect answers than correct ones. These results suggest that children did not read instructions or they were not understood, and so they played the game incorrectly. We plan the creation of an interactive tutorial to increase the understanding of the game.Sociedad Argentina de Informática e Investigación Operativ

Fernández Slezak, Diego

Goldin, Andrea Paula

Lipina, Sebastián

Lopez Rosenfeld, Matías

Sigman, Mariano

English

El Servicio de Difusión de la Creación Intelectual

11vo Simposio sobre la Sociedad de la Información, SSI 2013
MemoMarote: a working memory training game
Matías Lopez-Rosenfeld1, Andrea Paula Goldin2, Sebastián Lipina3, Mariano 
Sigman2,4, and Diego Fernandez Slezak1
1 Laboratorio de Inteligencia Artificial Aplicada, Departamento de Computación, 
FCEyN, UBA, Pabellón 1, Ciudad Universitaria, 1428 Buenos Aires, Argentina
2 Departamento de Física, FCEN, UBA and IFIBA, Conicet, Pabellon 1, Ciudad
Universitaria, 1428 Buenos Aires, Argentina
3 Unidad de Neurobiología Aplicada, UNA, CEMIC-CONICET
4 Universidad Torcuato Di Tella, Alte. Juan Saenz Valiente 1010, Buenos Aires 
C1428BIJ, Argentina
Abstract. Mate Marote is a flexible framework for large-scale educa- 
tional interventions that include games developed, based on aspects from 
cognitive sciences as an effective way of learning. MemoMarote is a com- 
puter game designed for training working memory. The pilot study of 
this platform was an intervention in La Rioja province (Argentina). In 
this article we analyze the registered statistics of use of the unsuper- 
vised version of the game. We found that playing activity had a peak 
when recently installed, decreasing during the first year and observing 
an increase when starting school classes in 2013. Analyzing the usage 
statistics we observed that players have a poor performance, with more 
incorrect answers than correct ones. These results suggest that children 
did not read instructions or they were not understood, and so they played 
the game incorrectly. We plan the creation of an interactive tutorial to 
increase the understanding of the game.
42 JAIIO - SSI 2013 - ISSN: 1850-2830 - Page 222
11vo Simposio sobre la Sociedad de la Información, SSI 2013
Introduction
In the last 20 years, the world has suffered profound changes in digital inclusión 
into the society. Two decades ago news were read in printed newspaper, we had 
to go the library to read specialized books and mail was sent by writing in a piece 
of paper and going to the post office. Ten years ago, many things changed with 
the popularization of the Internet, such as email or newspapers and books in 
our PCs. Nowadays, a digital revolution is occuring: the massiffication of mobile 
devices and massive connectivity.
However, these breakthroughs have not penetrated deeply into educaction 
and research in educacional issues. One of the most remarkable innovations in 
education is the one-to-one model which consists in giving one computer per 
child. The first huge implementation of the one-to-one idea was One Laptop 
Per Child (OLPC)5 which started in 2006. This program developed a netbook 
that costs around 100U$D specially designed for children, based on open-source 
initiatives which include Sugar6: a learning desktop specially built for children.
5 http://laptop.org/
6 http://sugarlabs.org/
7 http://wiki.sugarlabs.org/go/What_is_Sugar%3F
8 http://www.ceibal.edu.uy/
9 http://www.conectarigualdad.gob.ar/
Sugar redefines how computers are used by proposing an environment for 
collaboration, reflection, and discovery directly integrated into the user interface. 
One of the main directives in Sugar is you learn through doing, so if you want 
more learning you want more doing7.
OLPC has many implementations around the world. One of the most impor- 
tant is the Plan Ceibal, deployed in Uruguay. The Plan Ceibal8 has more than 
half a millon netbooks given to all students of public primary and secondary 
school. The main 1-to-1 programs in Argentina are Joaquín V. González (run- 
ning on all primary schools in La Rioja) and ConectarIgualdad9 (running on all 
secondary schools of the country).
One of the most remarkable things of these implementations is that all stu- 
dents have the same digital platform. This fact represents a huge opportunity: a 
nation-wide educational study for research in education practices and cognitive 
psychology. This massive availablility of human force have been capitalized on 
human computation to solve open problems, such as Games With A Purpose 
[1-3]. We propose using this digital platform to study stereotyped patterns of 
human thought, together with real-time observation of the learning and teaching 
process.
Mate Marote is a project to train cognitive skills in children by using com­
puter games [4]. Goldin and colleagues showed that children improve their perfor­
mance in some subjects at school by playing to Mate Marote games. The games 
included in Mate Marote are: Avioncito (which mainly trains inhibitory control), 
42 JAIIO - SSI 2013 - ISSN: 1850-2830 - Page 223
11vo Simposio sobre la Sociedad de la Información, SSI 2013
Casitas (oriented to planning tasks) and MemoMarote (focused on working mem- 
ory).
In the original Mate Marote project the educational intervention was super- 
vised by researchers that assisted to the participant schools as tutors of students. 
In 2012, Mate Marote was redesigned as an unsupervised on-line/off-line plat- 
form. This intervention consisted in installing the off-line Mate Marote appli- 
cation in the students personal netbooks in the La Rioja program[5]. Students 
were allowed to play spontaneously with supervision. The developed platform 
registers children behaviour while playing games and synchronizes it with Mate 
Marote server for further analysis.
In this work we present the analysis of usage data from MemoMarote, the 
game which mainly trains working memory. Working memory refers to the ca- 
pacity to store and manipulate information for very brief periods of time [6]. This 
capacity is limited to a few items both in adults and children [7]. This ability 
is trained when playing MemoMarote, a game based on a non-spatial, pattern 
recognition working memory task, a paradigm that measures recognition mem- 
ory for visual patterns, but not spatial locations [8,9].
42 JAIIO - SSI 2013 - ISSN: 1850-2830 - Page 224
11vo Simposio sobre la Sociedad de la Informacion, SSI 2013
Materials and methods
Mate Marote is a flexible framework for large-scale educational interventions. 
The first case study, a province-wide intervention in La Rioja province (Ar­
gentina) consisted of three games that train basic cognitive abilities: inhibitory 
control, planning and working memory. MemoMarote is the game of project 
which mainly trains working memory. The game consists of solving a board 
where there are some cards and the player should do some actions to win.
The board is a grid of 4x6 cells where in each round some of the cells have a 
box which may contain a card. Each card consists of features, e.g. shape, color, 
number of stars, character. To win a round the player must click with the mouse 
each card once. Each time the player clicks a card, all of them dissappear and 
re-appear in a random box. The player must think a strategy to avoid clicking 
on a previous card.
Fig. 1 shows a screenshot of a round of MemoMarote that has 8 boxes (3 in 
the first row, 3 in in third, an 2 in the fourth), and 3 cards (the first one has a 
shape similar to a finger, the background is pink and has a character wearing a 
diving suit; the has a shape of a drop with green background and has a character; 
and the third one has a circled, the background is a beach landscape and has 
the same character as the second one).
In Table 1 all features available are listed with their possible values. Each 
card of a round is unique, with its assigned features values. Some feature-values 
may be shared, but there will be always a singular feature-value that makes the 
card unique.
At the beggining the player starts with round that have 6 boxes and only 
3 cards, this is the easiest level. Each round has two possible results: win or 
lose. After winning 5 continuos rounds, the player changes to next level which 
includes more boxes and cards. Each time a player wins 5 continuous rounds 
he arrives to a checkpoint; however, if a player loses 3 times before getting to a 
checkpoint the game ends.
This game registers all the information about each round for later analysis. 
The format of this records can be seen in Listing 1.1.
The meaning of each line is detailed in the next list:
1. Add a box in position (523, 620)
3. Add a box in position (783, 230)
4. Add a card to a position with some features
5. Add a card to a position with some features
6. Add a card to a position with some features
7. Clicked card
8. Result of the click
42 JAIIO - SSI 2013 - ISSN: 1850-2830 - Page 225
11vo Simposio sobre la Sociedad de la Informacion, SSI 2013
Fig. 1. Screenshot of MemoMarote
Table 1. Features used to build cards of MemoMarote
42 JAIIO - SSI 2013 - ISSN: 1850-2830 - Page 226
11vo Simposio sobre la Sociedad de la Informacion, SSI 2013
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
[2012-05-29 01.56.44] Add BOX 523, 6 20)
[2012-05-29 01.56.44] Add BOX 783 230)
[2012 -05-29 01. 56. 44] Add CARD {pos: (523, 620), shape: 7, back : 4, props: [(1, Star, 2),]}
[2012 -05-29 01. 56. 44] Add CARD {pos: (263, 620), shape: 2, back : 7, props: [(1, Star, 2),]}
[2012 -05-29 01. 56. 44] Add CARD {pos: (523, 360), shape: 11, back: 2, props: [(1, Star, 8) ,]}
[2012 -05-29 01.56.48] Card { pos : (263 , 620) , shape: 2, back: 7, props: [(1, Star , 2) ,]}
[2012 -05-29 01.56.48] Card clicked OK
[2012 -05-29 01. 56. 48] Move CARD from (523, 620) to (653 , 360)
[2012 -05-29 01. 56. 48] Move CARD from (523, 360) to (653 , 620)
[2012 -05-29 01.57.08] Card clicked OK
[2012 -05-29 01.57.08] Game WON
[2012 -05-29 01. 57. 09] Status Correct: 1 - Wrong: 2 - Continuos correct: 1 - Checkpoint : 1
[2012 -05-29 01. 57. 22] Card { pos : (523 , 360) , shape: 6, back: 9, props: [(1, Star , 4) ,]}
[2012 -05-29 01. 57. 22] Card clicked WRONG
[2012 -05-29 01.57.23] Status Correct: 1 - Wrong: 3 - Continuos correct: 0 - Checkpoint : 1
Listing 1.1. Summary of the record of a player behaviour playing MemoMarote
9. New position of the card
11. New position of the card
13. Result of the click
14. End of round, all card were clicked once successfully
15. New round
17. Clicked card
18. End of round. The card was clicked before
19. Status of the game
42 JAIIO - SSI 2013 - ISSN: 1850-2830 - Page 227
11vo Simposio sobre la Sociedad de la Informacion, SSI 2013
Results
In this work we analyze the registered data obtained from the usage statistics 
in La Rioja pilot study. The pilot study started on May 2012, and we use data 
received until March 2013.
In Fig. 2, we show how many rounds were received per month (blue line). As 
expected, we observe a peak on May 2012, when pilot study was launched. On 
the next running months activity decreases reaching the end of year (Jan/Feb 
2013) with almost no activity. In march 2013, we observe again an increase of 
logging information.
Month
Fig.2. The red line represents the amount of rounds played per month, and the blue 
one represents the amount of rounds received in the server per month
Mate Marote synchronizes usage statistics when computers have connection, 
thus the received results consisted of a package containing many logs that may 
include many days of records. Then, we discriminate logs by their date of play 
(instead of date of reception). In Fig. 2, the red plot shows the amount of round 
42 JAIIO - SSI 2013 - ISSN: 1850-2830 - Page 228
11vo Simposio sobre la Sociedad de la Informacion, SSI 2013
played by month, shifted from the blue plot. This shift is consistent as received 
rounds (blue line) depend on connection and may take days or even weeks to 
synchronize.
Now, we set to study when children play MemoMarote and the performance 
obtained. In Fig. 3 we show the amount of played rounds per hour. Blue bars 
show wrong rounds and green bars show correct rounds. As observed, the rate 
of correct versus incorrect answers (overlaid solid red line) shows a very poor 
performance of this game. Almost every hour shows a performance rate below 1 
(see reference dotted red line), indicating that there were more incorrect answers 
than correct ones.
Fig. 3. The green bars shows the amount of winned rounds and the blue ones the 
lost rounds along the day. The red line indicate the rate of correct rounds versus the 
incorrect ones. The dotted red line is indicating that there is the same amount of 
correct rounds than incorrect.
Finally, we present the results of kids playing. In Fig. 4 we present the per­
formance of children at the different levels. The game increases cards as players 
advance in levels (see methods for details). At each level, a correct play consists 
of the same clicks as the number of cards, i.e. at level 1 with 3 cards a correct 
round consists of 3 clicks, one on each card. Thus, we plot the success rate for 
each click at the different levels (solid lines, red: 3 cards, blue: 4 cards, green: 5 
cards and yellow: 6 cards). Trivially, the first click is always correct as choosing 
any card is a correct answer, and thus successful rate is always 1. Then, for
42 JAIIO - SSI 2013 - ISSN: 1850-2830 - Page 229
11vo Simposio sobre la Sociedad de la Informacion, SSI 2013
example in the 3-cards rounds, the second click goes down to 0.7 indicating that 
almost one third of players choose wrong the second cards. Finally, the third 
click goes down 0.6.
0.9
0.8
0.7
Zj 
</>
o
<1)
0.6
0.5
0.4
---------  3 cards
---------  4 cards
---------  5 cards
---------  6 cards
---------3 cards chance
---------4 cards chance 
---------5 cards chance 
---------6 cards chance
34 
Click number
Fig. 4. The X axis represents the number of click in a round, and the Y axis the rate 
of success. In solid lines the behaviour of children playing MemoMarote is shown for 
the differet leves played: in red: 3 cards level; blue: 4 cards level; green: 5 cards level, 
yellow: 6 cards level. In dotted lines the rate of success of an automated agent that 
plays randomly is shown with the same colors as before (red: 3 cards; blue: 4 cards; 
green: 5 cards, yellow: 6 cards).
We decided to compare players behaviour to a simulated agent that plays 
randomly, i.e. at each round it chooses the cards randomly. For instance in the 
3-cards level, the first choice is always correct, then the probability of choosing 
a correct card is 2/3 and in the last turn, the probability is 1/3. We plot this 
random agent (see dotted lines in Fig. 4) for each level, red: 3 cards, blue: 4 cards, 
green: 5 cards and yellow: 6 cards. Comparing players with random agents, we 
observe that in all levels children play better than random agents.
42 JAIIO - SSI 2013 - ISSN: 1850-2830 - Page 230
11vo Simposio sobre la Sociedad de la Información, SSI 2013
Discussion and Conclusión
One-to-one models open a door to a new world of research and education inter- 
ventions. Mate Marote proposes a unified digital platform to study stereotyped 
patterns of human thought by playing games in educational contexts. In this 
work we present the results of a game which mainly trains working memory 
(MemoMarote) included in the pilot intervention at La Rioja province.
We analyzed the recevied logs during the year 2012 (and beginning of 2013). 
We observed a difference between the date when children play and the date when 
the results of playing are received. In La Rioja there is not complete coverage 
of Internet access, and so connectivity is not available everywhere. This fact 
causes computers to send information when Internet connection is detected and 
this might not be in real-time. We confirmed this situation by the shift between 
received logs and date of playing included in these records.
In previous work, we demonstrated that new computers are registered every 
day [5]. This fact suggests that many other computers could have been used but 
- due to the untrustful connection - their registered data was never sent to the 
server, until connection is available.
Based on the registered data, we analyzed how children play MemoMarote. 
We found that players show a poor performance in most hours of the day reported 
by a extremely low successful rate, with more incorrect answers than correct 
ones. Performance with greater correct answers than incorrect ones were only 
obtained in three cases: 4, 18 and 20 hour.
These results suggest that children do not play the game correctly. We com­
pared the success rate with an automated agent playing randomly. We observed 
that children playing 3-cards level had the worst performance, indicated by close- 
ness to random agent results. We theorize that many of the children do not un- 
derstand the rules of this game, so they never pass the first level showed by the 
worst results in it. This pilot study was done in children between 6 and 8 years 
old which do not know how to read fluidly and they are anxious to start playing 
without following the instructions. In the cases where the children learned the 
rules and passed to the next levels, we found that performance is much better 
than random agents.
Finally, we conclude that instructions were not clearly displayed. We plan 
to create a mandatory interactive tutorial that would increase the number of 
players that learn and apply the game rules correctly.
Acknowledgements
This research was supported by UBA, CONICET, ANPCyT, CEMIC, IBM Scal- 
able Data Analytics Innovation Awards and Human Frontiers Program. Mariano 
Sigman is sponsored by the James McDonnell Foundation.
42 JAIIO - SSI 2013 - ISSN: 1850-2830 - Page 231
11vo Simposio sobre la Sociedad de la Informacion, SSI 2013
Authors thank Walter Flores (Minister of Education, La Rioja, Argentina) 
and Rita Taquías (J. V. Gonzalez Program director) for their collaboration dur- 
ing the pilot deployment, Varinia Telleria for the design of characters and draw- 
ings used in the games, Martín Elias Costa who has greatly contributed to the 
development of this work and Cecilia Calero and Antonio Battro for fruitful 
discussions and useful comments on the manuscript.
References
1. Luis Von Ahn and Laura Dabbish. Designing games with a purpose. Communica- 
tions of the ACM, 51(8):58-67, 2008.
2. Luis Von Ahn, Ruoran Liu, and Manuel Blum. Peekaboom: a game for locating 
objects in images. In Proceedings of the SIGCHI conference on Human Factors in 
computing systems, pages 55-64. ACM, 2006.
3. Luis Von Ahn and Laura Dabbish. Labeling images with a computer game. In 
Proceedings of the SIGCHI conference on Human factors in computing systems, 
pages 319-326. ACM, 2004.
4. Andrea P. Goldin, M Soledad Segretin, M Julia Hermida, Luciano Paz, Sebastian J 
Lipina, and Mariano Sigman. Training planning and working memory in third 
graders. Mind, Brain, and Education, in press.
5. M. Lopez-Rosenfeld, A. P. Goldin, M. Sigman, and D. Fernandez Slezak. Mate 
marote: a flexible automated framework for large-scale educational interventions. 
Computers & Education, in press.
6. A.D. Baddeley and G.J. Hitch. Working memory. The psychology of learning and 
motivation, 8:47-89, 1974.
7. N. Cowan. The magical number 4 in short-term memory: A reconsideration of 
mental storage capacity. Behavioral and brain sciences, 24(1):87-114, 2001.
8. M. Luciana and C.A. Nelson. The functional emergence of prefrontally-guided work- 
ing memory systems in four-to eight-year-old children. Neuropsychologia, 36(3):273- 
293, 1998.
9. M. Petrides and B. Milner. Deficits on subject-ordered tasks after frontal-and 
temporal-lobe lesions in man. Neuropsychologia, 20(3):249-262, 1982.
42 JAIIO - SSI 2013 - ISSN: 1850-2830 - Page 232


MemoMarote: a working memory training game

http://sedici.unlp.edu.ar/bitstream/handle/10915/94646/Documento_completo.pdf?sequence=1

MemoMarote: a working memory training game

Abstract

Similar works

Full text

Available Versions

El Servicio de Difusión de la Creación Intelectual