Corpus-based assessment of writing quality: Methodological issues and critical considerations

By giving access to large quantities of naturally-occurring language, corpora constitute ideal resources to assess the quality of writing, including L2 writing. Different types of annotation (e.g. error tagging) and different measures (e.g. syntactic and lexical complexity) have been applied to corpora to determine the level of writing quality according to various aspects of language. In this presentation, it will be shown how writing assessment can be carried out using tools such as Lu’s (2010, 2012) L2 Syntactic Complexity Analyzer and Lexical Complexity Analyzer or Granger et al.’s (2023) UCLouvain Error Editor. Participants will be able to test these tools on a small data sample and discuss the process of annotation and the interpretation of the quantitative results. Particular attention will be paid to the linguistic realities behind the “omnibus measures” (Biber et al. 2020) that are often used to describe writing quality. It will be shown that global quantitative findings are sometimes misleading, especially in L2, and that more qualitative approaches are a useful, and sometimes necessary, complement for a comprehensive assessment of writing quality.