Complementary evidence in the early-stage validation of language tests: Classical Test Theory and Rasch analyses

Keywords: validity, validation, theory of applied linguistics, design principles, language assessment


Test validation may more aptly be conceived of as the process of designing language tests responsibly. While a good test gains in reputation as it is administered over time, the early stages of its validation are perhaps the most critical. There is now general agreement that the validation process should be reported in the form of an argument that brings together multiple sets of evidence to justify the design and implementation of the measurement instrument, the language test. The format of such integration is, however, still contestable ground. Referring to an example of language test design and development, this paper seeks to demonstrate how a framework for responsible test design may be employed to achieve such an integrated argument, as well as how two of the methodological tools most frequently employed to muster empirical evidence for validating test design, namely classical test theory (CTT) and Rasch analyses, complement each other in designing tests responsibly. While most language tests designed in South Africa have used CTT, the employment of Rasch analyses has been more limited. A secondary aim of the paper is therefore to provide applied linguists who work in the subfield of language testing with an example of how the latter kind of analysis can complement the former. In all, however, these disparate approaches must be integrated into the theoretical justification for the development of language tests, in order to satisfy a number of conditions for their responsible design.

Author Biography

Albert Johannes Weideman, University of the Free State
Albert Weideman is Professor of Applied Language Studies and Research Fellow at the University of the Free State, and Extraordinary Professor in Language Education at the University of the Western Cape. His Responsible design in applied linguistics: Theory and practice (2017, Springer) has been followed by the jointly edited Assessing academic literacy in a multilingual society: Transition and transformation (2021, Multilingual Matters). He focuses on how language assessment relates to a theory of applied linguistics.E-mail address: website: