Measuring lexical diversity among L2 learners of French: an exploration of the validity of D, MTLD and HD-D as measures of language ability
Treffers-Daller, J. (2013) Measuring lexical diversity among L2 learners of French: an exploration of the validity of D, MTLD and HD-D as measures of language ability. In: Jarvis, S. and Daller, M. (eds.) Vocabulary knowledge: human ratings and automated measures. Benjamins, Amsterdam, pp. 79-104. ISBN 9789027241887
In this study two new measures of lexical diversity are tested for the first time on French. The usefulness of these measures, MTLD (McCarthy and Jarvis (2010 and this volume) ) and HD-D (McCarthy and Jarvis 2007), in predicting different aspects of language proficiency is assessed and compared with D (Malvern and Richards 1997; Malvern, Richards, Chipere and Durán 2004) and Maas (1972) in analyses of stories told by two groups of learners (n=41) of two different proficiency levels and one group of native speakers of French (n=23). The importance of careful lemmatization in studies of lexical diversity which involve highly inflected languages is also demonstrated. The paper shows that the measures of lexical diversity under study are valid proxies for language ability in that they explain up to 62 percent of the variance in French C-test scores, and up to 33 percent of the variance in a measure of complexity. The paper also provides evidence that dependence on segment size continues to be a problem for the measures of lexical diversity discussed in this paper. The paper concludes that limiting the range of text lengths or even keeping text length constant is the safest option in analysing lexical diversity.
• Anderson, R. C. and Freebody, P. (1981). Vocabulary knowledge. In: Guthrie, J. (ed.),Comprehension and Teaching: Research Reviews (pp. 77–117). Newark, DE: International Reading Association. • Bachman, L. (1990). Fundamental considerations in language testing. Oxford: Oxford University Press. • Bardovi-Harlig, K. (1992). A second look at T-unit analysis: Reconsidering the sentence. TESOL Quarterly, 26, 390-395. • Bates, E., & Goodman, J. C. (1997). On the inseparability of grammar and the lexicon: Evidence from acqui¬sition, aphasia, and real-time processing. Language and Cognitive Processes, 12, 507–584. • Bradac, J.J. (1977). The Role of Prior Message Context in Evaluative Judgments of High- and Low-Diversity Messages. Language and Speech 20, 295-307. • Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81–105. • Carroll, J.B. (1938). Diversity of vocabulary and the harmonic law of word frequency distribution. Psychological Record II, 379-386. • Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum. • Cossette, A. (1994). La richesse lexicale et sa mesure. Paris: Champion-Slatkine. • Crossley, S., Salsbury, T. and McNamara, D. (2009). Measuring L2 Lexical Growth Using Hypernymic Relationships. Language Learning, 59, 307–334. • Daller, H. (1999). Migration und Mehrsprachigkeit. Der Sprachstand türkischer Rückkehrer aus Deutschland.Spracherwerb und Sprachverlust (Multilingualism and Migration. The Language Proficiency of Turkish Returnees from Germany). Frankfurt: Peter Lang. • • David, A. (2008). A developmental perspective on productive lexical knowledge in L2 oral interlanguage. French Language Studies 18, 315–331. • Daller, M.H. and Huijian Xue (2009). Vocabulary knowledge and academic study success : a study of Chinese students in UK Higher Education. In B. Richards, M. H.Daller, D.D. Malvern, P.Meara, J.L.Milton and J.Treffers-Daller (eds.) Vocabulary Studies in first and second language acquisition. The interface between theory and applications (pp. 179-193). Houndmills Basingstoke: Palgrave Macmillan • Dewaele, J-M. (1993). Extraversion et richesse lexicale dans deux styles d'interlangue française. I.T.L., Review of Applied Linguistics, 100, 87-105. • Dickinson, O. K. and Tabors, P. O. (eds.) (2001). Beginning Literacy with Language: Young Children Learning at Home and School. Baltimore: Paul H. Brookes. • Dugast, D. (1980). La statistique lexicale. Genève: Slatkine. • Dunn, L. M., & Dunn, D. M. (1959/2006). Peabody picture vocabulary scale. San Antonio, TX: Pearson Assessments. • Eckes, Th. and Grotjahn, R. (2006). A closer look at the construct validity of C-tests. Language Testing 2006, 23, 290-325. • Edmondson, W. and House, J. (1991). Do learners talk too much? In Phillipson, R., Kellerman, E. Phillipson, R., Kellerman, E., Selinker, L., Sharwood Smith, M., & Swain. M. (eds.), Foreign/Second Language Pedagogy Research (pp. 273-86), Multilingual Matters, Clevedon • Gavin, W. J. (2004). Utterance length and lexical diversity in Cantonese-speaking children with and without specific language impairment. Journal of Speech, Language, and Hearing Research, 47, 1396–1410. • Hu, M. and Nation, I. S. P. (2000). Unknown vocabulary density and reading comprehension. Reading in a Foreign Language 13.1, 403–430. • Hunt, K. W. (1965). Grammatical structures written at three grade levels. NCTE Research Report No 3. Champaign, Ill.: National Council of Teachers of English. • Jarvis, S. (2002). Short texts, best fitting curves, and new measures of lexical diversity. Language Testing, 19, 57-84. • Jersild , A. T.and Ritzman, R. (1938). Aspects of language development: The growth of loquacity and vocabulary. Child Development, 9, 243-259. • Johnson, W. (1939). Language and Speech Hygiene. General Semantics Monographs, No. 1, Lakeville, Conn.: Institute of General Semantics. • Johnson, W. (1944). Studies in Language Behavior. A program of research. o Psychological Monographs, Vol 56(2)1-15. doi: 10.1037/h0093508. • Klee, T., Stokes, S. R., Wong, A. M.-Y., Fletcher, P., & Gavin, W. J. (2004). Utterance length and lexical diversity in Cantonese-speaking children with and without specific language impairment. Journal of Speech, Language, and Hearing Research, 47, 1396–1410. • Le Normand, M.-Th., Parisse, Chr. and Cohen, H. (2008). Lexical diversity and productivity in French preschooolers: developmental, gender and sociocultural factors. Clinical Linguistics & Phonetics 22, 1, 47–58. • Lindqvist, Chr. (2010). La richesse lexicale dans la production orale de l’apprenant avancé de français. The Canadian Modern Language Review / La revue canadienne des langues vivantes, Volume 66,3, 393-420. • Little, D. and Singleton, D. (1992). The C-test as an elicitation instrument in second language research. In. R.Grotjahn (ed.), Der C-Test. Theoretische Grundlagen und praktische Anwendungen (pp. 173-92). Bochum : Brockmeyer. • Lonngren-Sampaio, C. (in prep.). Portuguese-English code-switching. University of Hertfordshire. • Maas, H. D. (1972). Zusammenhang zwischen Wortschatzumfang und Länge eines Textes. Zeitschrift für Literaturwissenschaft und Linguistik, 8, 73-79. • Macaro, E. and Masterman, E. (2006) Does intensive explicit grammar instruction make all the difference? Language Teaching Research 10, 3, 297-327. • Malvern, D.D. and Richards, B.J. (1997) Anew measure of lexical diversity. In Ryan, A. and Wray, A., editors, Evolving models of language. Clevedon: Multilingual Matters, 58–71. • Malvern, D., Richards, B., Chipere, N. and Durán, P. (2004). Lexical Richness and Language Development: Quantification and Assessment. Houndmills, Basingstoke: Palgrave Macmillan. • McCarthy, P. M. (2005). An assessment of the range and usefulness of lexical diversity measures and the potential of the measure of textual, lexical diversity (MTLD). Dissertation Abstracts International, 66(12). (UMI No. 3199485). • McCarthy, P. M., and Jarvis, S. (2007). A theoretical and empirical evaluation of vocd. Language Testing, 24, 459-488. • Norris, J.M. and Ortega, L. (2009). Towards an Organic Approach to Investigating CAF in Instructed SLA: The Case of Complexity. Applied Linguistics 30, 4. 555–578. • Ntelitheos, D. , Idrissi, A. Tibi, S, Al Qahtani, S. Tamimi, O and Al Eisaei, F. (ms.) The Development of Morphosyntactic Complexity in Emirati Arabic. United Arab Emirates University. • Owen, A.J. & Leonard, L.B. (2002). Lexical diversity in the spontaneous speech of children with specific language impairment: Application of D. Journal of Speech and Hearing Research, 45, 927-937. • Perez-Bazan (2005) ¿Qué será, será?: A Methodological Tool for Predicting Early Bilingualism in a Family Setting. 2005 María Jesús Pérez-Bazán. ISB4: Proceedings of the 4th International Symposium on Bilingualism, ed. James Cohen, Kara T. McAlister, Kellie Rolstad, and Jeff MacSwan, 1821-1841. Somerville, MA: Cascadilla Press. • Plauen, E. O.  1996. Vater und Sohn, Band 2. Ravensburger Taschenbuch. • Read, J. (2007). Second language vocabulary assessment: Current practices and new directions. International Journal of English Studies, 7 (2), 105-125. • Richards, B. J. (1987). Type/Token Ratios: what do they Really Tell us? Journal of Child Language, 14, 201-9. • Silverman, S. & Bernstein Ratner, N. (2002). Measuring lexical diversity in children who stutter: application of VOCD. Journal of Fluency Disorders 27, 4, 289-303. • Spinelli, M. (2011). La valutazione della competenze lessicale nei bambini: una confronto tra indici [The assessment of lexical competence in children: a comparison of three indexes]. Thesis obtained for the Laurea vecchio ordinamento. Scienze della formazione primaria” at the University of Modena and Reggio Emilia. • Templin, M. (1957). Certain language skills in children. Minneapolis: University of Minnesota Press. • Tidball, F. and J. Treffers-Daller (2007). Exploring measures of vocabulary richness in semi-spontaneous speech of native and non-native speakers of French: a quest for the Holy Grail? In: Daller, H., Milton, J. and Treffers-Daller, J. (eds.) Modelling and assessing vocabulary knowledge (pp.133-149). Cambridge: CUP. • Treffers-Daller, J. (2009). Language dominance and lexical diversity: How bilinguals and L2 learners differ in their knowledge and use of French lexical and functional items. In B. Richards et al (eds.) Vocabulary Studies in first and second language acquisition. The interface between theory and applications (pp. 74-90). Houndmills Basingstoke: Palgrave Macmillan. • Treffers-Daller, J. (2011). Operationalizing and measuring language dominance. International Journal of Bilingualism 15, 2, 147-163. • Trochim, W.M.K. (2006). The multitrait multimethod matrix. http://www.socialresearchmethods.net/kb/mtmmmat.php [accessed 7th November 2011] • Wolfe-Quintero, K., S. Inagaki, and H.-Y. Kim. 1998. Second Language Development in Writing: Measures of Fluency, Accuracy, and Complexity. University of Hawai‘i, Second Language Teaching and Curriculum Center • Wu, T. (1993). An accurate computation of the hypergeometric distribution function. • ACM Transactions on Mathematical Software, 19, 33-43. • Yu, G. (2010) Lexical diversity in writing and speaking task performances. Applied Linguistics 31, 2, 236-259.
Centaur Editors: Update this record