Programmeringsoppgaver og auto-scoring: Inspera vs. andre løsninger

Retting av store antall eksamensbesvarelser i programmeringsfag innen stramme sensurfrister er krevende. Selv om oppgavetypen «Programmering» i Inspera gir klare fordeler framfor håndskrevet kode, er den manuelt rettet. I 2021 lanserte Inspera en ny oppgavetype, «Kompilering», hvor poeng kunne gis automatisk basert på testsuiter. Dessverre var oppgavetypen beheftet med en del problemer, og den er nå trukket tilbake for videreutvikling, dermed ikke lenger tilgjengelig for bruk i eksamen. Denne artikkelen beskriver våre erfaringer med oppgavetypen «Kompilering» og diskuterer hvilke behov for forbedringer som ville være ønskelig for at bruk av denne skulle kunne bli vellykket - særlig for eksamen, men også potensielt for bruk under formativ vurdering underveis i semesteret. Det gjøres en sammenligning med andre systemers implementasjon av lignende oppgavetyper, f.eks. Moodle CodeRunner, basert på praktisk utprøving og litteraturstudium av erfaringer med bruk i andre land. Artikkelen diskuterer også i hvilken grad karaktersetting kan basere seg fullt ut på automatisk poenggivning av kode, eller hvorvidt man fortsatt vil trenge manuell gjennomgang, for eksempel for å kunne gi noe poeng til studenter som har kode som ikke fungerer, men hvor løsningen likevel inneholder mye riktig tankegang

Studiebarometeret på programnivå: effekt av vurderingsformer og obligatoriske øvinger?

Avsluttende skriftlig eksamen er den dominerende vurderingsfor-men ved norske universiteter, særlig i tidligestudieår –gjerne i kombinasjon med obligatoriske øvinger. Dette gjelder også IT-studier, men deterbetydelig varia-sjon mellom studieprogrammer. Et interessant spørsmål er hvorvidt studenters fornøydhet påvirkes av variasjon i vurderingsformer. Denne artikkelen ser på den vektende sammensetningen av vurderingsformer for teknisk-naturvitenskapelige studieprogram ved NTNU og undersøker korrelasjon mellom dette og studenters tilfredshet med de ulike programmene i Studiebarometeret. Resultatene indikerer at studenter jevnt over er noe mer positive til programmer med variert bruk av vurderingsformer, men det er få av spørsmålene i Studiebarometeret hvor for-skjellen er signifikant. Særlig for obligatoriske øvinger er det likevel påfallende mange negative korrelasjoner med Studiebarometeret for 2020, som indikerer at dette er en praksis utdanningsinstitusjonene må tenke bedre gjennom. Problemet er ikke nødvendigvis obligatoriske aktiviteter som sådan, men den typiske måten disse gjøres på, hvor studentene får liten følelse av jobbrelevans, refleksjon og stimulering til nytenkning

A study on Mobile Requirements Elicitation by Boilerplate Requirements Specification Language

With the increasing use of mobile information systems, mobile devices are being used for gradually more complex tasks. Therefore it is also necessary to pay more attention to requirements methods for such systems. One way of supporting requirements engineering is through templates for how to write requirements, often guided by taxonomies. In this paper we propose templates especially for mobility-related requirements, based on a combination of so-called requirements boilerplates as presented by Hull et al. and a taxonomy of mobility-related requirements presented in a previous publication by ourselves. The proposed requirements templates are illustrated by examples, and our work indicates that mobility-related requirements specification may benefit from the use of boilerplates as long as natural language remains an important part of such specifications

E-exams and exam process improvement

While many tasks concerning the interaction between students and learning institutions have been successfully digitized, high-stakes examinations remain mainly a traditional paper-based process in most Norwegian learning institutions. A large scale shift towards e-exams can however be expected during the next 5-10 years, based on a number of perceived benefits of digital exams over paper-based exams. From a pedagogical perspective it is important to avoid a too narrow focus on e-exams mainly as a means to save money by making the examination and grading process more efficient, but rather as a possibility for more fundamental process and quality improvement. This paper analyzes what improvements might be possible, and what requirements this would entail for the e-exam system

Activity diagrams with location context: Experimental comparison of colour and icon annotations

In mobile information systems, the location of the user when performing tasks may be important to take into account during development. Yet, mainstream process models seldom capture this aspect. In previous papers we have evaluated analytically a number of small adaptations for showing location of actions in UML activity diagrams. Two of the most promising adaptations have also been evaluated experimentally. However, another notation alternative that got a quite positive score in the analytical evaluation has not been experimentally evaluated, namely a notation attaching symbolic icons to the activity nodes. This paper reports on an experiment comparing this notation to the most promising one from previous experiments (colour). The results showed no significant difference between the two notations, neither in the quality of answers to the experimental task, the time spent performing the task, nor in opinions about the notation alternatives, as investigated in a post-task questionnaire

Evaluating the Effectiveness of Learning Interventions: An Information Systems Case Study

Currently, there is no standard instrument for evaluating learning effectiveness. While final examinations and end-of-semester course evaluation surveys can be used to do this, they are not designed for this purpose, and there are inherent problems using them in this way. This paper describes a survey instrument, called the Learning Effectiveness Survey, which can be used to evaluate and improve the effectiveness of learning interventions. Learning effectiveness is evaluated in the context of the learning goals of the course (short term learning), and in the context of the overall educational programme and future working life (long term learning). The instrument also provides feedback on the intervention and how it could be improved. A case study is described in which the instrument is used to evaluate the use of peer reviews as a learning activity in a requirements analysis course. The instrument was found to have relatively high validity, but reliability was below acceptable levels. Some interesting results were also found on the determinants of learning. In particular, attitude was found to have no effect on short term learning, but was found to be the primary determinant of long term learning

Et informatikkperspektiv på norsk lovgivning om karakterklager i høyere utdanning

KD har i en nylig proposisjon foreslått endringer av klagesensurordningen, hvor det leggesopp til at en modell med blind omsensur, som allerede har vært brukt ved en del læresteder,nå skal gjøres obligatorisk for alle læresteder. I denne artikkelen sammenlignes gammel ogny klagesensurordning med ordningene i en del andre land. Vi diskuterer ulike typersensurfeil (med eksempel fra programmering) og kvalitetskriterier for klagebehandling,hvorpå norske og en generisk utenlandsk ordning sammenlignes etter disse kriteriene.Sammenligningen er ikke empirisk, kun analytisk basert på "sunn fornuft". Den indikerer atden norske ordningen for klagesensur ikke holder mål mot internasjonal "state-of-the-art",og at den er unødvendig kostbar uten at dette gir seg utslag i høyere kvalitet avsensurresultatene eller muligheter for læring eller prosessforbedring

A Systematic Mapping Study on Requirements Engineering in Software Ecosystems

Software ecosystems (SECOs) and open innovation processes have been claimed as a way forward for the software industry. A proper understanding of requirements is as important for these IT-systems as for more traditional ones. This paper presents a mapping study on the issues of requirements engineering and quality aspects in SECOs and analyzes emerging ideas. Our findings indicate that among the various phases or subtasks of requirements engineering, most of the SECO specific research has been accomplished on elicitation, analysis, and modeling. On the other hand, requirements selection, prioritization, verification, and traceability has attracted few published studies. Among the various quality attributes, most of the SECOs research has been performed on security, performance and testability. On the other hand, reliability, safety, maintainability, transparency, usability attracted few published studies. The paper provides a review of the academic literature about SECO-related requirements engineering activities, modeling approaches, and quality attributes, positions the source publications in a taxonomy of issues and identifies gaps where there has been little research.Comment: Journal of Information Technology Research (JITR) 11(1

Introducing Peer Review in an IS Analysis Course

Peer reviews were introduced as a teaching technique for the 2002 offering of the course SIF8035 Information Systems at the Norwegian University of Science and Technology (NTNU). The students handed in conceptual models which were then double-blind reviewed by three independent peers. The review reports contained scores and defect lists, but were not used for grading. This paper conducts an analysis of the outcomes of the peer review exercises. Several approaches are used, both quantitative and qualitative, investigating the students’ performance and perceptions. The main conclusions are: 1) The scores given by the peer reviewers were not reliable enough to recommend their use for grading purposes. 2) The introduction of peer review exercises contributed positively to students’ learning in the course – but not equally so for all students. A substantial fraction of the students did so little that it is hard to claim any learning effect. The main reasons for this seem to have been poor motivation and unclear demands. In hindsight, we have discussed the distinguishing properties of three different purposes of peer-reviews that were not clear to us when the course started, and we have identified several possible ways of improving the peer reviews. In spite of the reported problems, experiences are more positive than negative, and it has been decided to continue with peer reviews in the course