This study reports the results of a pilot-testing project of a video-based pragmatics instrument whose purpose was the assessment of the receptive level of communicative competence of high school and university students in the Japanese context. The instrument was analyzed using Rasch modeling and the results show high levels of item reliability with low levels of person reliability for the sample population. Person logits of ability on the pragmatics assessment were subjected to linear regression (ANOVA) using SPSS as a criterion variable against a set of predictor variables including scores on EIKEN, TOEFL, TOIEC, a 64-item Willingness to Communicate motivation survey, educational year, gender, experience learning in an English-medium school, and time spent in residence abroad. Results indicate three variables to be statistically significant predictors of pragmatic ability: TOELF score, year of education (years 2 and 3 at the high school level and year 1 and 2 in university), and residence abroad
