Courseware validation should locate Learning Objects inconsistent with the courseware instructional design being used. In order for validation to take place it is necessary to identify the implicit and explicit information needed for validation. In this paper, we identify this information and formally define an information architecture to model courseware validation information explicitly. This promotes tool-support for courseware validation and its interoperability with the courseware specifications

Melia, Mark

Pahl, Claus

English

Irish Universities

An information architecture for validating courseware

DCU Online Research Access Service

An Information Architecture for ValidatingCoursewareMark Melia and Claus PahlSchool of Computing, Dublin City University, Dublin 9, Irelandmmelia@computing.dcu.ieAbstract. Courseware validation should locate Learning Objects incon-sistent with the courseware instructional design being used. In order forvalidation to take place it is necessary to identify the implicit and ex-plicit information needed for validation. In this paper, we identify thisinformation and formally define an information architecture to modelcourseware validation information explicitly. This promotes tool-supportfor courseware validation and its interoperability with the coursewarespecifications.1 IntroductionThe assembly of Learning Objects (LOs) into courseware is increasingly becom-ing the norm in courseware construction. LO reuse offers course creators anefficient methodology to create courseware from tried and tested components.This methodology also has challenges. One of the most pressing challenges thecourse creator faces is placing the right LO in the right place in courseware [8].This is difficult as course creators typically only have high level knowledge ofthe content and internal behaviour of each LO, due to 3rd party constructionand increasing internal complexity.Quality courseware requires a consistent and suitable instructional design.Due to a lack of understanding, a course creator may place LOs in coursewarethat are not compliant with courseware’s instructional design. This causes in-consistent pedagogy which can confuse, demotivate and isolate the learner andcould ultimately lead to the rejection of the course by the learner [7].In this paper, we detail the courseware information that must be captured inorder to accurately validate courseware developed using LOs and define a layeredinformation architecture to that effect. The formal definition of the informationarchitecture allows for a validation engine to be built around it, providing toolsupport to the course creator when validating courseware. Our formal definitionalso facilitates for translation to and from external courseware representationspecifications such as SCORM [1] and IMS LD [5], allowing for interoperability.2 Identification of Information NeedsCourseware validation is a complex task which involves evaluating courseware inthe context of its scope (the knowledge the courseware wishes to teach), instruc-Proceedings of the First International Workshop on Learning Object Discovery & Exchange72tional approach (how it teaches this knowledge) and content used (educationalmaterials used in teaching). In the real world this may mean delegating vali-dation tasks to experts. Validation of course structure may be delegated to aninstructional design expert, while content related issues, may be delegated to asubject matter expert.Fig. 1. Courseware Validation ConcernsIn order for the courseware validation process to be automated, the implicitknowledge and information used to validate courseware, known as the “course-ware validation concerns”, must be identified and defined. Courseware infor-mation changes over time (e.g. a more detailed learner model is possible post-delivery) definitions must focus on one stage of the courseware life-cycle. Herewe define courseware information at the post-construction/pre-delivery stage ofthe courseware life-cycle (i.e. after the courseware has been developed by thecourse creator but before it is delivered to learners).Fig. 1 outlines the courseware validation concerns for the post-construction/pre-delivery stage of the coureware life-cycle. Three expert roles are represented inthe figure, the domain expert, the instructional designer and course accredita-tion. These roles may be embodied by the one course creator, or may representseveral specialised course creators. Domain information outlines the knowledgeto be taught to the learner. The domain expert can also define basic rules ofthumb on how particular domain knowledge is taught (e.g. concept A alwaysbefore B). The scope of the courseware is defined in the context of the domaininformation specifying the course pre-requisites and the course goals. The coursescope is usually defined by the course accreditation body. The instructional de-sign defines a strategy for the transfer of knowledge to the learner and is derivedfrom general instructional design principles.Proceedings of the First International Workshop on Learning Object Discovery & Exchange73Courseware is defined as instructional logic combined with LOs. LOs arethe content which teach aspects of domain knowledge to the learner and aredescribed using metadata, outlining the material covered by the LO and themethod of instruction. Instructional logic specifies how the learner proceedsthrough the courseware, from one LO to another.3 Layered Information Architecture3.1 Information Architecture OverviewBy explicitly representing the courseware validation concerns identified in theprevious section it is possible to automatically validate courseware constructed.The Courseware Authoring Validation Information Architecture (CAVIAr) al-lows for the representation of the validation concerns using a system of lay-ered models. CAVIAr extends the LAOS model, used in Adaptive EducationalHypermedia (AEH) authoring. LAOS simplifies AEH authoring by separatingAEH concerns allowing the course creator to deal with each individually. [3].Our model replicates the benefits of using the LAOS model in AEH authoringby separating the courseware validation concerns.Fig. 2. The Courseware Authoring Validation Information Architecture (CAVIAr)The bottom three layers of the validation model, the domain model, thegoal and constraint model and the learner model, are replicated from the LAOSmodel but adapted for validation. On top of these base layers the coursewaremodel and validation model are defined.Proceedings of the First International Workshop on Learning Object Discovery & Exchange743.2 The Domain ModelThe domain model is a formalism of knowledge in the form of a concept map,where a node represents a concept and an edge represents the relationship fromone concept to another.A conceptual graph for domain modeling in AEH authoring is formally de-fined in [3]. In AEH the domain model serves as the main navigational tool,while in courseware validation it serves as a semantic point of reference for LOannotation (section 3.5).We can define the courseware validation domain model requirements as fol-lows:Definition 1. A concept map CM is determined by the tuple <C,L> where Cis a set of concepts and L is a set of links.– A concept c ∈ C is an abstract notion which is described using attributes.There must be at least one attribute for each concept, the name of the conceptdescribed as nc.– A link l ∈ L is a tuple <c1, c2, tl, nl> with c1 ∈ C, c2 ∈ C start and endconcept respectively, nl is the name of the link. There are many types oflinks, the link type adds semantics to the link, indicating what is meant bythe link. tl refers to the link type (e.g. “is-a”, “has-a”).– Each concept c must have at least one link, the hierarchical link, which linksa concept to its parent concept. An exception of this is the root concept.3.3 The Goal and Constraints ModelThe purpose of the goal and constraints model in CAVIAr is to specify domainpedagogic information and course conceptual goals. The course creator can ex-press which concepts from the domain model are to be taught to the learner andconceptual pre-requisite constraints between concepts.We distinguish between two types of knowledge, deep knowledge and shal-low knowledge, where deep knowledge implies an understanding of a conceptsunderlying concepts or sub-concepts and shallow knowledge implies a passingknowledge or a mere familiarity with a concept. To this effect, when defining amodel in terms of knowledge (i.e. the domain model), we must distinguish if wemean deep knowledge or shallow knowledge. To do this, goals and constraintshave a strength - weak or strong, which specify deep knowledge or strong knowl-edge respectively. This accommodates the needs of real courses as specified in[2].We now give a more formal definition of the elements which make up thegoal and constraint model.Definition 2. The goal and constraint model is made up of the tuple <P,G>,where P is a set of pre-requisite constraints, which specify a relationship betweenconcepts where one concept must be understood before the other concept and Gis a set of Goals.Proceedings of the First International Workshop on Learning Object Discovery & Exchange75– A pre-requisite constraint p ∈ P is described using the tuple <c1, c2, ps>,where c1 ∈ C, c2 ∈ C and ps describes the strength of the pre-requisite con-straint, weak or strong. Weak pre-requisites require only passing knowledgeof the pre-requisite concept, a strong pre-requisite requires deep knowledge ofthe pre-requisite concept and its underlying concepts.– A goal g∈G is described by the tuple <GC, alt, gg>, where GC is a set ofgoal concepts, alt defines alternative goal and gg is a nested goal within thegoal g.– A goal concept gc ∈ GC is made up of the tuple <c, gs>, where c refers toa concept in the domain model gc ∈ C and gs is the strength of the goal, aweak goal or a strong goal.We can infer that the goal and constraint model is an overlay model on thedomain model in that the goal and constraint model is specified in terms ofthe domain model. This is due to each gc ∈ C, meaning that GC ⊆ C andpre-requisite constraints are expressed using concepts from the domain model.3.4 The Learner ModelIn CAVIAr, the learner model is used to represent the stereotypical learnerknowledge or pre-requisite knowledge for a courseware. The learner model is anecessary layer of CAVIAr as concepts not covered in the courseware may be pre-requisite of concepts in the courseware, but this knowledge is assumed learnerknowledge. In order for a validation engine to acknowledge the learner’s initialassumed knowledge it must be modelled.As we specify the learner model in terms of conceptual knowledge the learnermodel elements also have a strength, weak or strong, as is the case for goals andconstraints.We can also describe the learner model formally as follows:Definition 3. The learner model is described by the tuple <kc, ks>, where kcrefers to a concept in the domain model kc ∈ C and ks is the strength of theassumed knowledge, weak or strong.The learner model is also an overlay model on the domain model as eachkc ∈ C, again meaning kc ⊆ C.3.5 The Courseware ModelA course structure can be modeled as a directed graph. Learning resources arenodes on the graph. A learner’s traversal through the graph’s edges depends onvariables, such as learner choice, assessment results, learning styles, feedback,which are assessed at run-time. Learning resources in the course are annotatedusing some metadata standard such as IEEE LOM [6]. The metadata maps thelearning resource to the concept(s) it addresses in the domain model.Fig. 3 demonstrates how each LO in a course is associated with at least oneconcept in the domain model, specified in the LO annotation (LO conceptualProceedings of the First International Workshop on Learning Object Discovery & Exchange76Fig. 3. Grouping LOs according to the concept they coverannotation is indicated in the diagram with an arrow from a LO to a concept).LO conceptual associations can be used to group LOs. Grouping are made up ofLOs concerned with the same concept. This type of LO grouping is illustratedin the diagram in figure 3 (dotted circle on course model indicates conceptualgroupings). Once we can group LOs by concept we can discriminate betweenpedagogical strategies concerned with inter-conceptual pedagogy (strategy con-cerning sequencing of the conceptual groupings) and intra-conceptual pedagogy(pedagogical strategy concerning the LOs within each conceptual grouping).We now give a more formal definition of the course model.Definition 4. We consider courseware CW to be determined by the tuple <LO,LP, SP,EP>,where LO represents a set of Learning Objects and LP is the set of learningpaths. SP and EP represent the start points and end points respectively in agiven courseware model.– A learning path lp ∈ LP is defined by the tuple <lo1, lo2, G>, where lo1 ∈LO, lo2 ∈ LO start and end learning objects respectively. G refers to aboolean gate condition, which must be true for the learner to proceed downthat learning path.– A learning object lo ∈ LO is defined by the tuple <LC,Alo>, where LC islearning content and Alo is the LO’s annotation.– A learning object’s annotation Alo is a tuple <M,CA>, M is a metadataset used to describe a LO. CA is a set of conceptual annotation.– Metadata m where m ∈ M is a tuple <att, val>, where att is the name ofthe metadata attribute and val refers to its value.– A Conceptual Annotation ca ∈ CA is a tuple <pur, c, s>, where pur is thepurpose of the annotation (i.e. to specify a competency in a particular con-Proceedings of the First International Workshop on Learning Object Discovery & Exchange77cept), c∈C a reference to domain concept and s the strength of the link tothe concept.The connection between the courseware model and the lower layers is in theLO’s annotation. Each LO’s annotation points to a concept c, where c ∈ C.In our definition we have defined a LP with a boolean gate condition, G.This gate condition allows the course creator to specify what learning path thelearner should take based on various variables found in e.g. the learner model.3.6 The Validation ModelThe Validation Model captures pedagogical rules that courseware must adhereto. In the validation model layer the course creator can express undesirable prop-erties of courseware and also properties which must be present in the courseware.Validation can be split into two distinct parts, validation which is concernedwith learning content of one domain concept (i.e. intra-conceptual validation)and validation which looks at how the course proceeds from one concept toanother in the course (i.e. inter-conceptual validation). Typical intra-conceptualvalidation will ensure that each concept is taught in a uniformed manner, whileinter-conceptual sequencing might ensure that the strategy undertaken in thecourse is one of depth-first (covering each concept in detail before moving on tothe next concept).The formal definition of the lower four layers of CAVIAr allows for the forma-tion of pedagogical rules. Here we specify a rule which states for every concept cthat is a goal concept, there must exist a LO annotated with that concept whichis of “Type” “Lecture”.∀x∃y(c(x), c ∈ gci) ∧ (CWi.LO(y).A.CA.c = x) ∧ (CWi.LO(y).A.M(z).Att =′′Type′′) ∧ (CWi.LO(y).A.M(z).value =′′ Lecture′′)In the example below, we demonstrate how the rule above can be imple-mented using the JESS rule language, a Java-based, Lisp-like, rule language [4].The rule states that each concept in the goal model, “concepts-in-gm”, musthave a LO which is annotated to have a “resource type” of “lecture”. If this isnot the case the rule prints out the violating concept with the message “HASNO LECTURE”.( defquery findConceptsLOs //QUERY: f ind the LOs a s s o c i a t ed with the given concept( de c l a r e ( v a r i a b l e s ? c ) )( concept−to−l o ( concept ? c ) ( l o ? l o ) ) )( defquery findLoResourceType //QUERY: Get the LOs with the given re source type( dec l a r e ( v a r i a b l e s ?x ? resourceType ) )( lom ( id ?x ) ( r e sou r c e type ? resourceType ) ) )( fo reach ?c ? concepts−in−gm // f o r each concept in goa l model( bind ? l i s t (new java . u t i l . ArrayList ) )( bind ? los−in−concept (new java . u t i l . ArrayList ) )( bind ? concept−ok FALSE)( bind ? i t r ( run−query findConceptsLOs ?c ) ) // get LOs f o r concept( whi le (? i t r hasNext ) // f o r each LO( bind ? token ( c a l l ? i t r next ) )( bind ? f a c t ( c a l l ? token f a c t 1) )( bind ? lo ( fact−s l o t−value ? f a c t l o ) )(? l i s t add ? lo ) // add LO to l i s t)( fo reach ? l ? l i s t // f o r each LO in l i s t( bind ? i t r 2 ( run−query findLoResourceType ? l l e c t u r e ) ) // does LO have re sourceProceedings of the First International Workshop on Learning Object Discovery & Exchange78type l e c t u r e( i f (? i t r 2 hasNext ) then( bind ? concept−ok TRUE) // i f i t does then concept ok))( i f ( not ? concept−ok ) then // i f the ? concept−ok va r i ab l e has not been changed − nol e c t u r e type( pr intout t ”CONCEPT ”? c” HAS NO LECTURE” c r l f )))4 DiscussionIn this paper we have given an overview of the information needs for the vali-dation of courseware. We have then taken the information needs identified anddefined a layered information architecture (CAVIAr), which allows for the ex-plicit representation of each information need. The definition of each layer allowsfor the formulation of pedagogical rules, which can be validated in the contextof our information architecture.The formal definition of the CAVIAr allows for tool support to be builtaround it, in our future work, we will design and implement a validation enginebased on CAVIAr. Another advantage of CAVIAr’s formal definition is thatit allows for translation from external courseware representations to CAVIAr,allowing for interoperability with existing courseware specifications, thus em-bracing the state of the art in courseware construction.References1. Advanced Distributed Learning. SCORM 2004 Overview, 2004. Available from:http://www.adlnet.gov/scorm/index.cfm.2. P. Brusilovsky and J. Vassileva. Course sequencing techniques for large-scale web-based education. International Journal Continuing Engineering Education adn Life-long Learning, 13(1/2):75–94, 2003.3. A. I. Cristea and A. de Mooij. LAOS: Layered WWW AHS Authoring Modeland their corresponding Algebraic Operators. In Proceedings of The Twelfth Inter-national World Wide Web Conference (WWW03), Alternate Track on Education.ACM, May 20th - 24th 2003.4. E. Friedman-Hill. JESS in Action: Java Rule-Based Systems. Manning Publications,Greenwich, Connecticut, 2003.5. H. Hummel, J. Manderveld, C. Tattersall, and R. Koper. Educational modellinglanguage and learning design: New oportunities for instructional reusability andpersonalised learning. International Journal of Learning Technology, 1(1):110–126,2004.6. IEEE Learning Technology Standards Committee. LTSC WG12:Learning ObjectMetadata, 2002.7. J. W. Samples. The pedagogy of technology - our next frontier? Connexions,14(2):4–5, 2002.8. D. A. Wiley. The Instructional use of Learning Objects, chapter Connecting Learn-ing Objects to Instructional Design Theory: A definition, a methaphor and a tax-onomy. Association for Educational Communications and Technology, 2001.Proceedings of the First International Workshop on Learning Object Discovery & Exchange79

Course sequencing techniques for large-scale webbased education.

Educational modelling language and learning design: New oportunities for instructional reusability and personalised learning.

JESS in Action: Java Rule-Based Systems.

Layered WWW AHS Authoring Model and their corresponding Algebraic Operators.

The Instructional use of Learning Objects, chapter Connecting Learning Objects to Instructional Design Theory: A deﬁnition, a methaphor and a taxonomy. Association for Educational Communications and Technology,

The pedagogy of technology - our next frontier?

Name not available

An information architecture for validating courseware

Abstract

Similar works

Full text

Available Versions

Irish Universities

DCU Online Research Access Service

Name not available