[EN] Assessing speaking is one of the most difficult tasks in computer based language testing. Many countries all over the world face the  need  to  implement  standardized  language  tests  where  speaking  tasks  are  commonly  included.  However,  a  number  of  problems  make  them  rather  impractical  such  as  the  costs,  the  personnel  involved,  the  length  of  time  for  interviews  and  many  other factors. Additionally, reliability as compared to face-to-face tests is continually challenged by issues such as comfort with the interface, navigability and, among others and more important, the lack of visual interaction and the high anxiety created by interacting with a interlocutor with more than a limited interactional activity. This paper addresses a new approach to language testing by the use of VOIP devices. It also addresses its benefits and the way to implement it within the Spanish framework of nationally organized standardized tests 

Designing a Voip Based Language Test. Available from: https://www.researchgate.net/publication/281207173_Designing_a_Voip_Based_Language_Test [accessed Dec 28, 2015].S1307131119

Ellis

Garcia Laborda

Jesus Garcia Laborda

Jose L. Gimenez Lopez

Kinginger

Krashen

Lantolf

Nuria Otero de Juan

Poehner

Teresa Magal Royo

Vygotsky

English

AbstractAssessing speaking is one of the most difficult tasks in computer based language testing. Many countries all over the world face the need to implement standardized language tests where speaking tasks are commonly included. However, a number of problems make them rather impractical such as the costs, the personnel involved, the length of time for interviews and many other factors. Additionally, reliability as compared to face-to-face tests is continually challenged by issues such as comfort with the interface, navigability and, among others and more important, the lack of visual interaction and the high anxiety created by interacting with a interlocutor with more than a limited interactional activity. This paper addresses a new approach to language testing by the use of VOIP devices. It also addresses its benefits and the way to implement it within the Spanish framework of nationally organized standardized tests

Laborda, Jesus Garcia

Royo, Teresa Magal

Juan, Nuria Otero de

Lopez, Jose L. Gimenez

Elsevier - Publisher Connector 

 Procedia - Social and Behavioral Sciences  197 ( 2015 )  1307 – 1311 
Available online at www.sciencedirect.com
ScienceDirect
1877-0428 © 2015 The Authors. Published by Elsevier Ltd. This is an open access article under the CC BY-NC-ND license 
(http://creativecommons.org/licenses/by-nc-nd/4.0/).
Peer-review under responsibility of Academic World Education and Research Center.
doi: 10.1016/j.sbspro.2015.07.404 
7th World Conference on Educational Sciences, (WCES-2015), 05-07 February 2015, Novotel 
Athens Convention Center, Athens, Greece 
DESIGNING A VOIP BASED LANGUAGE TEST 
Jesus Garcia Labordaa* , Teresa Magal Royob, Nuria Otero de Juana, Jose L. Gimenez 
Lopezb 
aUniversidad de Alcala, c/ Trinidad 3, Alcala de Henares-Madrid 28801, Madrid 
bUniversitat Politecnica de Valencia,DEGI, Cami de Vera,s/n, Valencia 46022, Spain 
Abstract 
Assessing speaking is one of the most difficult tasks in computer based language testing. Many countries all over the world face 
the need to implement standardized language tests where speaking tasks are commonly included. However, a number of 
problems make them rather impractical such as the costs, the personnel involved, the length of time for interviews and many 
other factors. Additionally, reliability as compared to face-to-face tests is continually challenged by issues such as comfort with 
the interface, navigability and, among others and more important, the lack of visual interaction and the high anxiety created by 
interacting with a interlocutor with more than a limited interactional activity. This paper addresses a new approach to language 
testing by the use of VOIP devices. It also addresses its benefits and the way to implement it within the Spanish framework of 
nationally organized standardized tests. 
© 2015 The Authors. Published by Elsevier Ltd. 
Peer-review under responsibility of Academic World Education and Research Center. 
Keywords: VOIP, Spanish framework, face-to-face 
1. Introduction 
Assessing speaking and writing have been traditionally considered difficult skills to assess in foreign language 
learning. It is not because they are specifically harder but because they are visually more productive and complex. 
Although there is much evidence to re-consider this opinion, this judgment is usually supported by those who have 
no clear and deep knowledge of the complex processes that intervene in second language acquisition. It is certainly 
true that according to many researchers like Stephen Krashen (1985) or Rod Ellis (2005) input precedes to output but 
 
 
* Jesus Garcia Laborda. Tel.: +3491 885 5041. 
  E-mail address: jesus.garcialaborda@uah.es 
 2015 The Authors. Published by Elsevi r Ltd. This is an open access article under the CC BY-NC-ND license 
(http://creativecommons.org/licenses/by-nc-nd/4.0/).
Peer-review under responsibility of Academic World Education and Research Center.
1308   Jesus Garcia Laborda et al. /  Procedia - Social and Behavioral Sciences  197 ( 2015 )  1307 – 1311 
this perspective has been challenged by the Social Constructivist Theory (Vygotsky, 1978, 1985) who claims that 
input and output are continuously interacting in the Zone of Proximal Development (Kinginger, 2001). This position 
has been supported by those who believe that tests are not just a way of externalizing the speaker’s internal 
knowledge. For instance, current research run at Penn State University (Poehner, 2008; Lantolf & Poehner, 2011) in 
the United States suggest that dialogue between interlocutors exist both in written and speaking grounds either in 
face-to-face oral exams, in paper-and-pen tests or even in computer-based assessments. Although the way these 
interactions and the role of feedback varies accordingly especially in computer based computers where Lantolf and 
Poehner consider support to change a response in multiple choice test not just as a guessing effect but as a part of a 
socio-constructivist interaction in the ZPD. What seems self-evident is that two things are totally necessary at this 
point: 1) Revise our perspective of what a language test is and 2) revise the degree of interaction in communicative 
testing. This paper will partially address both aspects in the Spanish University Entrance Examination in order to 
consider the effect of VOIP applications in language testing.    
 
2. What is a language test? Key issues in the Spanish Entrance Examination 
Technology has made us revise the paradigms of learning especially languages. However, whether learning has 
changed dramatically due to the inclusion of video repositories and podcasts, language testing has followed quite a 
different way. In many cases, language tests are nothing but a traditional technological application of pen-and-paper 
older tests. This is based on three key issues: 
a. Lack of a new definition of what communicating in a language implies; 
b. Lack of a clear image of what role of feedback and interaction is; 
c. Lack of consideration of how technology (especially web 2.0) can currently be used in communication. 
The Spanish University Entrance Examination has been mostly the same for the last thirty years. Small variations 
can be found in different school districts. It generally includes four tasks: two reading tasks (True/False and two 
open questions), four questions on grammar and vocabulary and finally a writing essay. Two regional boards also a 
listening task (one multiple choice selection question). However, none of the Spanish regional educational boards 
includes speaking tasks. The reasons are varied but the most significant is the high cost of delivering oral interviews 
whether individual or in pairs. Some Chinese regions address this problem by only allowing the top 10% of the 
testees to proceed to the oral interview (nevertheless, the number of test takers is immense!).  In Spain, given the 
current constraints, it is almost impossible to organize these interviews nationally. The main key issues that need to 
be reviewed are the following: 
a. Cost of delivery – cost of moving teachers to different schools. 
b. Time 
c. Technology implementation  
The Ministry of Education has funded a number of research projects to develop computer based language tests 
that could facilitate the delivery of speaking interviews including those delivered through mobile phones, desktops 
and tablet PC’s (Figure 1). However, these projects will have to face a number of challenges to be implemented: 
a. The initial cost of computer based tests is usually high 
b. Students and teachers need to become fully familiar with the test format thus they need materials to 
train with 
c. Security is a must because a large number of pirate attacks are done against high-stakes test delivery 
institutions to obtain the different tests that they administer 
Regularly these three issues are solved by formal institutions but in the particular case of language testing, two 
more should be considered: d) the connection between candidate-administrators-raters, and e) the particular 
specifications of the speaking test.   
These challenges were addressed by the Spanish OPENPAU (University Entrance Examination) project. 
Additionally, they faced the need to incorporate a speaking test. Between 2011 and 2013 the researchers worked 
mostly on two devices: mobile phones and Tablet PC’s (Figure 1). However whether the results have been positive 
so far (Garcia Laborda, Magal Royo, Litzler & Gimenez Lopez, 2014), a serious constraint is funding. The software 
1309 Jesus Garcia Laborda et al. /  Procedia - Social and Behavioral Sciences  197 ( 2015 )  1307 – 1311 
is extremely expensive and requires specific equipment that might not be present in a large number of schools in the 
country.    
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Figure 1. Interfaces of mobile and tablet PC based OPENPAU University Entrance Examination foreign language section 
3. What does the country actually need? 
Maybe the most significant need is an inexpensive speaking test. Speaking tests are usually costly, especially 
when done at large scale, because they require a number of interviewers and raters for many days. The average time 
of many speaking tests whether delivered individually or in pairs is about ten minutes so delivering the test in an 
average school can take from two to three days. That cost needs to be reduced because it implies testing fees and 
tester’s allowances (salary, food, travel and so). These costs were highly reduced by the use of mobiles and tablet 
PC’s but using this devices also requires the high expense to buy the equipment on the first year.  
In 2014, the research team began to consider the possibility to use a Voice Over Internet Protocol (VOIP) device 
(Figure 2).  
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
    
 
 
 
Figure 2. Test delivery process through a VOIP application. 
1310   Jesus Garcia Laborda et al. /  Procedia - Social and Behavioral Sciences  197 ( 2015 )  1307 – 1311 
These applications offer immense benefits. While the students can do the written test with the rest of the class, 
the speaking test can be delivery shortly after the written test either to part or all the students. The benefits of using 
such a delivery method are: 
a. Reduced cost: One or no raters would be necessary, only ID controllers. 
b. Standard digital equipment is sufficient 
c. Quick response and feedback 
d. Students can or not belong to the same institution and other groupings are also possible 
e. It can be delivered individually or in groups (pairs / trios) 
f. Technically reduces the anxiety of semi-directly delivered speaking tests 
g. The communication is human-human and not machine-human 
h. Security is enhanced by the fact that the questions are not recorded previously 
i. A great deal of interaction among interview participants can be achieved 
j. Body language can also be recorded and is the adequate to the communicative situation  
k. It permits the incorporation of test writing along with the image (Figure 3) 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
Figure 3. A simulation of a VOIP application for language testing. 
 
4. Conclusions 
Traditionally, speaking tests tend to be very expensive to deliver due to the human resources costs. One 
significant way to reduce the expenses is by using computers in their delivery. However, specifically design 
software is usually expensive and requires certain specifications in both the guest and the host equipment. Therefore, 
the use of VOIP non-commercial system reduces the costs. As for security, a number of things can be done like 
private-pay channels on skype and similar solutions. Nevertheless, the possibility of cheating is almost non-existing 
and the interviews can be recorded through image-casting applications similar to screen-o-matic 
(http://www.screencast-o-matic.com/), for instance. Obviously, this paper has presented evidence that serve to 
enhance validity and reliability for the test but this is just a beginning of a much more complex research. 
 
 
1311 Jesus Garcia Laborda et al. /  Procedia - Social and Behavioral Sciences  197 ( 2015 )  1307 – 1311 
Acknowledgements 
The researchers would like to express their gratitude to the Ministry of Research and Innovation of Spain 
(MICINN) for supporting the development and implementation OPENPAU research project (FFI2011-22442) with 
cofounding with ERDF funds under the 2008-2011 plan.  
References 
Ellis, R. (1997). Second language acquisition. In Oxford Introduction to Linguistics. Oxford: Oxford University Press. 
Garcia Laborda, J., Magal Royo, T., Litzler, M.F. & Gimenez Lopez, J.L. (2014). Mobile phones for a University Entrance Examination language 
test in Spain. Educational Technology & Society, 17(2), 17-30. 
Kinginger, C. (2001).  i + 1 different from ZPD, Foreign Language Annals  34(5), 417-425. 
Krashen, S.D. (1985). The Input Hypothesis: Issues and Implications. New York: Longman  
Lantolf, J. P., & Poehner, M. E. (2011). Dynamic assessment in the classroom: Vygostkian praxis for second language development. Language 
Teaching Research, 15 (1), 11-33. 
Poehner, M. E. (2008). Dynamic Assessment: A Vygotskian Approach to Understanding and Promoting Second Language Development. Berlin: 
Springer Publishing. 
Vygotsky, L.  Mind in Society: The Development of Higher Psychological Processes.  Cambridge, MA: Harvard University Press, 1978. 
Vygotsky, L. S.  Thought and Language.  Cambridge, MA: The M.I.T. Press, 1985. 
 
 
 
 
 
 


Designing a Voip Based Language Test 

[EN] Assessing speaking is one of the most difficult tasks in computer based language testing. Many countries all over the world face the  need  to  implement  standardized  language  tests  where  speaking  tasks  are  commonly  included.  However,  a  number  of  problems  make  them  rather  impractical  such  as  the  costs,  the  personnel  involved,  the  length  of  time  for  interviews  and  many  other factors. Additionally, reliability as compared to face-to-face tests is continually challenged by issues such as comfort with the interface, navigability and, among others and more important, the lack of visual interaction and the high anxiety created by interacting with a interlocutor with more than a limited interactional activity. This paper addresses a new approach to language testing by the use of VOIP devices. It also addresses its benefits and the way to implement it within the Spanish framework of nationally organized standardized tests 

Designing a Voip Based Language Test. Available from: https://www.researchgate.net/publication/281207173_Designing_a_Voip_Based_Language_Test [accessed Dec 28, 2015].1307131119

García Laborda, Jesús

Magal Royo, Teresa

Otero de Juan, Nuria

Giménez López, José Luis

RiuNet

Designing a Voip Based Language Test

Crossref

Procedia - Social and Behavioral Sciences

https://riunet.upv.es/bitstream/10251/100411/1/1-s2.0-S1877042815044055-main.pdf

Designing a Voip Based Language Test

Abstract

Similar works

Full text

Available Versions

Elsevier - Publisher Connector

RiuNet

Crossref