An update to Volker Grimm and colleagues\' Overview, Design concepts and Details (ODD) protocol for documenting individual and agent based models (I/ABM) has recently been published in Ecological Modelling. This renames the \'State variables and scales\' element to \'Entities, state variables and scales\', and the \'Input\' element to \'Input data\', introduces two new Design concepts (\'Basic principles\' and \'Learning\'), and renames another (\'Fitness\' is now generalised to \'Objectives\'). The Design concepts element can now also be shortened such that it is not required to include any design concept that is irrelevant to the model, and expanded to include new design concepts more appropriate to the model being described. Other clarifications of intentions in the original protocol have been made.ODD, Individual Based Models, Agent Based Models, Replication, Documentation

J. Gary Polhill

English

Research Papers in Economics

ﾩCopyright JASSS
J. Gary Polhill (2010)
ODD Updated
Journal of Artiﬁcial Societies and Social Simulation 13 (4) 9
<http://jasss.soc.surrey.ac.uk/13/4/9.html>
Received: 13-Oct-2010    Accepted: 17-Oct-2010    Published: 31-Oct-2010
Keywords:
1.1
1.2
1.3
Abstract
An update to Volker Grimm and colleagues' Overview, Design concepts and Details (ODD) protocol
for documenting individual and agent based models (I/ABM) has recently been published in
Ecological Modelling. This renames the 'State variables and scales' element to 'Entities, state
variables and scales', and the 'Input' element to 'Input data', introduces two new Design concepts
('Basic principles' and 'Learning'), and renames another ('Fitness' is now generalised to 'Objectives').
The Design concepts element can now also be shortened such that it is not required to include any
design concept that is irrelevant to the model, and expanded to include new design concepts more
appropriate to the model being described. Other clariﬁcations of intentions in the original protocol
have been made.
ODD, Individual Based Models, Agent Based Models, Replication, Documentation
 Introduction
Grimm et al. (2006) originally developed the Overview, Design concepts and Details (ODD) protocol
for documenting individual and agent based models (I/ABM) to address frustrations with the lack of
consistency in the text describing such models in journal articles (mainly in the ﬁeld of ecology). In
particular, such descriptions were often incomplete in the sense that they did not provide enough
information to replicate the results, and/or were difﬁcult to read because the information was not
presented in a logical order. Similar frustrations have been expressed by authors concerned with
agent-based social simulations (Treibig and Klugl 2009; Janssen et al. 2008; Richiardi et al. 2006),
some making their own proposals for documentation standards. The ODD protocol has been widely
adopted in ecological models (about 70% of articles citing Grimm et al. (2006) are in that area (Grimm
et al. 2010)).
Issues with replication are commonplace in social simulation just as they are in ecology, and there
have been various activities in the community that have been speciﬁcally aimed at addressing them.
Three model-to-model workshops have been held, two of which have been the subject of special
sections in this journal (Hales et al. 2003; Rouchier et al. 2008). Furthermore, the forum of JASSS
has been allocated to reports on replications of agent-based models (recent examples include
Miodownik et al. (2010), Macy and Sato (2010) and Will (2009)). Further, the annual Volterra prize for
replication has been issued with the aim of demonstrating the robust use of agent-based models, the
ﬁrst winner of which was Radax and Rengs (2010).
In the meantime, the use of ODD in the social simulation community is small but growing. Grimm et
al.'s (2010) review of uses of ODD categorised just two of 54 papers as being in the social sciences,
though two of the other papers might also be regarded as social simulations. Polhill et al. (2008)
demonstrated ODD for three agent-based social simulations of land-use change. Between the review
being conducted and October 2010, ﬁve further papers documenting social simulations have used
ODD: Grifﬁth et al.'s (2010) simulation of how habitat changes could have inﬂuenced populations of
early hominids; Hartig and Drechsler's (2010) model of farmers trading in a biodiversity market; Le et
al.'s (2010) application of their Land Use Dynamic Simulator to policy scenarios; Naivinit et al.'s
(2010) documentation of a model co-constructed with rice farmers in Thailand; and Valbuena et al.'s
(2010) model of farmers' responses to liberalisation and regionalisation scenarios in the Netherlands.1.4
2.1
2.2
2.3
2.4
2.5
2.6
2.7
2.8
2.9
With growing interest in, and use of, ODD, the time is right to update the protocol in the light of
experiences with it to date.
Updating the protocol
In the update to ODD, Grimm et al. (2010) review citations of Grimm et al. (2006) on Web of Science
at 14 December 2009, checking their use of ODD against the protocol, and ﬁnding that whilst 75% of
uses were (almost) correct, 11% of the papers had signiﬁcantly deviated from it. Based on this
review, and feedback from users, including Polhill et al. (2008), Grimm et al. (2010) make the
following changes to the protocol:
Purpose
Many articles were found to have left out 'Purpose', one possible motivation being that it repeats
material that may already have been provided in the introduction to the article. Though the new
protocol retains 'Purpose' there is an emphasis on keeping the statement short.
Entities, state variables and scales
This element has been renamed from 'State variables and scales' to 'Entities, state variables and
scales', to move beyond a point where all that matters is the variables in the model; often grouping
the variables into entities allows the model to be more clearly described. It is also clariﬁed that 'state
variables' can include non-numeric discriminants of objects, such as variables determining behaviour
or strategies.
Process overview and scheduling
The review found that schedules were often still not described in enough detail to enable replication,
despite using ODD. The deﬁnition of 'Process overview and scheduling' has therefore been
speciﬁed more precisely, and recommends the use of pseudo-code to describe the schedule, which
should include descriptions of how execution is ordered when a process is conducted for multiple
agents.
Design concepts
Two new headings were introduced to this element: 'Basic principles' and 'Learning', and the 'Fitness'
design concept was generalised to 'Objectives'. However, the main change to the 'Design concepts'
element was to introduce greater ﬂexibility. For smaller models, some of the design concepts are not
relevant, in which case they can be ignored. For other models, authors may feel that important
concepts underlying the design are not in the ODD protocol. Here, these concepts should be given a
short heading and included at the end of the element, making it clear that they are not 'ODD-
standard' design concepts.
The 'Objectives' design concept encompasses a more general idea than that of 'Fitness', which
applies to any aspect of an individual's behaviour that is aimed at increasing their success in
achieving one or more objectives. Here, the protocol stipulates that authors record those objectives,
how success is measured, and how agents choose between several options with differing expected
success (a particular issue where there are multiple objectives). The 'Learning' design concept was
introduced to allow authors to describe how agents adapt their decision-making processes to
improve the expected success with which objectives are achieved. The distinction between
'Objectives' and 'Learning' is somewhat subtle; if an agent decides what it will do using a set of rules
R, then the 'Objectives' are what the agent is trying to do by applying R. By contrast, 'Learning'
refers to how R is changed in the light of experience to improve the agent's ability to achieve its
objectives.
The 'Basic principles' design concept is provided to allow authors to describe theories, hypotheses,
and modelling approaches underpinning the design.
Initialisation
No signiﬁcant changes were made to the 'Initialisation' element.
Input data
The name and description of the 'Input' element were changed slightly to clarify that it refers to time-2.10
3.1
3.2
series data used while the model is running. (Initialisation, by contrast, describes how values are
found for the state variables at time 0.) Such data might be referred to as scenario, or driving data,
and the variables they give values for as exogenous.
Submodels
The review found a mismatch between the processes named in the 'Process overview and
scheduling' element and subheadings of the 'Submodels' element. It was also not clear in the original
protocol speciﬁcation that the factual description of a submodel (i.e. its equations, rules and
algorithms) and its rationale should be separated. The protocol description was amended to make
appropriate clariﬁcations.
Conclusion
Hopefully the revisions to ODD increase its relevance and utility to the social simulation community.
Indeed, at the 2010 World Congress on Social Simulation in Kassel, Janssen's (2010) plenary talk
announced progress on initiating a new journal emphasising empirically embedded uses of agent-
based models, and that this journal would encourage use of ODD when describing models in its
articles. However, an issue acknowledged in Polhill et al. (2008) was also raised in the ensuing
debate: whilst the revised ODD has attempted to distance itself further from a focus that might be
deemed better suited to describing models built with object-oriented programming languages, it may
be that it is still not a good ﬁt for models built in declarative programming languages popular with
many researchers in agent-based social simulation (e.g. Alam et al. 2007; Edmonds and Hales 2004;
Harbers et al. 2009; Moss 1998; Pajares et al. 2004). To assess this properly will require such
researchers to evaluate ODD and see where it presents obstacles to the more natural description of
declarative models. One area likely to be difﬁcult is the 'Process Overview and Scheduling' element:
in declarative models the order of execution of the rules is determined by the question asked of the
model and the inference engine's mechanics[1]. One suggestion is that this element be used to give
such information instead, with, perhaps, an interaction diagram showing which rules ﬁre which other
rules[2], albeit that possibly the title of the element wouldn't match quite with the content. Similarly,
though the element title and information content don't quite match, the 'Submodels' element could be
used to give details on the rules.
This is unlikely to be the ﬁnal version of ODD, and should it become popular in the social simulation
community there will be more feedback on which to base future revisions. In the meantime, you are
encouraged to use and try ODD in your own articles, and invited to comment through the
openabm.org forum.
 Acknowledgements
This work was funded in part by the Scottish Government Rural and Environment Research and
Analysis Directorate.
Notes
1 'Pure' declarative models, where the issues described here apply, should be distinguished from
'mixed' models whereby the schedule is procedural, but the model uses declarative methods for
agent cognition (as is the case with some of the models cited). Though it cannot be conﬁrmed until
ODD is more widely used, 'mixed' models should be no more difﬁcult to describe using ODD than
models built using more conventional object-oriented programming languages.
2 Thanks to Scott Moss for this suggestion.
 References
ALAM S J, Meyer R, Ziervogel G and Moss S (2007). The impact of HIV/AIDS in the context of
socioeconomic stressors: An evidence-driven approach. Journal of Artiﬁcial Societies and Social
Simulation 10 (4), 7. http://jasss.soc.surrey.ac.uk/10/4/7.html
EDMONDS B and Hales D (2004). When and why does haggling occur? Some suggestions from a
qualitative but computational simulation of negotiation. Journal of Artiﬁcial Societies and Social
Simulation 7 (2), 9. http://jasss.soc.surrey.ac.uk/7/2/9.htmlGRIFFITH C S, Long B L and Sept J M (2010). HOMINIDS: An agent-based spatial simulation model
to evaluate behavioural patterns of early Pleistocene hominids. Ecological Modelling 221 (5), 738-
760. [doi:10.1016/j.ecolmodel.2009.11.009]
GRIMM V, Berger U, Bastiansen F, Eliassen S, Ginot V, Giske J, Goss-Custard J, Grand T, Heinz S
K, Huse G, Huth A, Jepsen J U, J￸rgensen C, Mooij W M, M￼ller B, Pe'er G, Piou C, Railsback S F,
Robbins A M, Robbins M M, Rossmanith E, R￼ger N, Strand E, Souissi S, Stillman R A, Vab￸ R,
Visser U and DeAngelis D L (2006). A standard protocol for describing individual-based and agent-
based models. Ecological Modelling 198 (1-2), 115-126. [doi:10.1016/j.ecolmodel.2006.04.023]
GRIMM V, Berger U, DeAngelis D L, Polhill J G, Giske J and Railsback S F (2010). The ODD
protocol: A review and ﬁrst update. Ecological Modelling 221 (23), 2760-2768.
[doi:10.1016/j.ecolmodel.2010.08.019]
HALES D, Rouchier J and Edmonds B (2003). Model-to-model analysis. Journal of Artiﬁcial
Societies and Social Simulation 6 (4), 5. http://jasss.soc.surrey.ac.uk/6/4/5.html
HARBERS M, Meyer J-J and van den Bosch K (2009). Explaining simulations through self
explaining agents. Journal of Artiﬁcial Societies and Social Simulation 12 (3), 6.
http://jasss.soc.surrey.ac.uk/12/3/6.html
HARTIG F and Drechsler M (2010). Stay by thy neighbor? Social organization determines the
efﬁciency of biodiversity markets with spatial incentives. Ecological Complexity 7 (1), 91-99.
[doi:10.1016/j.ecocom.2009.07.001]
JANSSEN M (2010). Towards collective action among social simulation. Presentation to the Third
World Congress on Social Simulation, Kassel, Germany, September 6-9, 2010.
JANSSEN M A, Alessa L N, Barton M, Bergin S and Lee A (2008). Towards a community framework
for agent-based modelling. Journal of Artiﬁcial Societies and Social Simulation 11 (2), 6.
http://jasss.soc.surrey.ac.uk/11/2/6.html
LE Q B, Park S J and Vlek P L G (2010). Land Use Dynamic Simulator (LUDAS): A multi-agent
system model for simulating spatio-temporal dynamics of coupled human-landscape system. 2.
Scenario-based application for impact assessment of land-use policies. Ecological Informatics 5 (3),
203-221. [doi:10.1016/j.ecoinf.2010.02.001]
MACY M and Sato Y (2010). The surprising success of a replication that failed. Journal of Artiﬁcial
Societies and Social Simulation 13 (2), 9. http://jasss.soc.surrey.ac.uk/13/2/9.html
MIODOWNIK D, Cartrite B and Bhavnani R (2010). Between replication and docking: "Adaptive
agents, political institutions, and civic traditions" revisited. Journal of Artiﬁcial Societies and Social
Simulation 13 (3), 1. http://jasss.soc.surrey.ac.uk/13/3/1.html
MOSS S (1998). Critical incident management: An empirically derived computational model. Journal
of Artiﬁcial Societies and Social Simulation 1 (4), 1. http://jasss.soc.surrey.ac.uk/1/4/1.html
NAIVINIT W, Le Page C, Tr￩buil G and Gajaseni N (2010). Participatory agent-based modeling and
simulation of rice production and labor migrations in Northeast Thailand. Environmental Modelling &
Software 25 (11), 1345-1358. [doi:10.1016/j.envsoft.2010.01.012]
PAJARES J, Hern￡ndez-Iglesias C and L￳pez-Paredes A (2004). Modelling learning and R&D in
innovative environments: A cognitive multi-agent approach. Journal of Artiﬁcial Societies and Social
Simulation 7 (2), 7. http://jasss.soc.surrey.ac.uk/7/2/7.html
POLHILL J G, Parker D, Brown D and Grimm V (2008). Using the ODD protocol for describing three
agent-based social simulation models of land-use change. Journal of Artiﬁcial Societies and Social
Simulation 11 (2), 3. http://jasss.soc.surrey.ac.uk/11/2/3.html
RADAX W and Rengs, B (2010). 'Prospects and Pitfalls of Statistical Testing: Insights from
Replicating the Demographic Prisoner's Dilemma'. Journal of Artiﬁcial Societies and Social
Simulation 13(4),1. http://jasss.soc.surrey.ac.uk/13/4/1.html.
RICHIARDI M, Leombruni R, Saam N and Sonnessa M (2006). A common protocol for agent-based
social simulation. Journal of Artiﬁcial Societies and Social Simulation 9 (1), 15.
http://jasss.soc.surrey.ac.uk/9/1/15.html.
ROUCHIER J, Ciofﬁ-Revilla C, Polhill J G and Takadama K (2008). Progress in model-to-model
analysis. Journal of Artiﬁcial Societies and Social Simulation 11 (2), 8.
http://jasss.soc.surrey.ac.uk/11/2/8.htmlTREIBIG C and Klugl F (2009). Elements of a documentation framework for agent-based simulation
models. Cybernetics and Systems 40 (5), 441-474. [doi:10.1080/01969720902922459]
VALBUENA D, Verburg P, Veldkamp A, Bregt A K and Ligtenberg A (2010). Effects of farmers'
decisions on the landscape structure of a Dutch rural region: An agent-based approach. Landscape
and Urban Planning 97 (2), 98-110. [doi:10.1016/j.landurbplan.2010.05.001]
WILL O (2009). Resolving a replication that failed: News on the Macy & Sato model. Journal of
Artiﬁcial Societies and Social Simulation 12 (4), 11. http://jasss.soc.surrey.ac.uk/12/4/11.html.

A common protocol for agent-based social simulation.

Elements of a documentation framework for agent-based simulation models.

Progress in model-to-model analysis.

Scenario-based application for impact assessment of land-use policies.

The impact of HIV/AIDS in the context of socioeconomic stressors: An evidence-driven approach.

Towards a community framework for agent-based modelling.

ODD Updated

http://jasss.soc.surrey.ac.uk/13/4/9/9.pdf

ODD Updated

Abstract

Similar works

Full text

Available Versions

Research Papers in Economics