S. Abney, Semisupervised Learning for Computational Linguistics, 2007.

T. Alexopoulou, M. Michel, A. Murakami, and D. Meurers, Task effects on linguistic complexity and accuracy: A large-scale learner corpus analysis employing natural language processing techniques, Language Learning, vol.67, issue.S1, pp.180-208, 2017.

T. Alexopoulou, H. Yannakoudakis, and A. Salamoura, Classifying intermediate learner English: a data-driven approach to learner corpora, Twenty Years of Learner Corpus research: Looking Back, Moving Ahead, pp.11-23, 2013.

Y. Attali and J. Burstein, Automated essay scoring with e-rater R v.2, The Journal of Technology, Learning and Assessment, vol.4, issue.3, 2006.

G. Balikas, Lexical bias in essay level prediction, 2018.

F. Barker, A. Salamoura, and N. Saville, The Cambridge Handbook of Learner Corpus Research, Cambridge Handbooks in Language and Linguistics, pp.511-534, 2015.

C. Baur, A. Caines, C. Chua, J. Gerlach, M. Qian et al., Overview of the 2018 spoken CALL shared task, pp.2354-2358, 2018.

C. Baur, C. Chua, J. Gerlach, E. Rayner, M. Russel et al., Overview of the 2017 spoken CALL shared task, Workshop on Speech and Language Technology in Education (SLaTE), vol.25, 2017.

A. Boyd, J. Hana, L. Nicolas, D. Meurers, K. Wisniewski et al., The MERLIN corpus: Learner language and the CEFR". In LREC, pp.1281-1288, 2014.

M. Callies and M. Paquot, Learner corpus research: An interdisciplinary field on the move, International Journal of Learner Corpus Research, vol.1, issue.1, pp.1-6, 2015.

X. Chen and D. Meurers, CTAP: A web-based tool supporting automatic complexity analysis, Proceedings of the Workshop on Computational Linguistics for Linguistic Complexity (CL4LC), pp.113-119, 2016.

, Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Strasbourg, Language Policy Division, 2001.

, Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Structured Overview of all CEFR Scales, Language Policy Division, 2001.

, Common European Framework of Reference for Languages: Learning, Teaching, Assessment; Companion Volume with New Descriptors, Language Policy Division, 2018.

S. A. Crossley, T. Salsbury, D. S. Mcnamara, and S. Jarvis, Predicting lexical proficiency in language learner texts using computational indices, Language Testing, vol.28, issue.4, pp.561-580, 2011.

S. Cushing-weigle, Validation of automated scores of TOEFL iBT tasks against non-test indicators of writing ability, Language Testing, vol.27, issue.3, pp.335-353, 2010.

D. Dahlmeier, H. T. Ng, and S. M. Wu, Building a large annotated corpus of learner English: The NUS corpus of learner English, Proceedings of the Eighth Workshop on Innovative Use of NLP for Building Educational Applications, pp.22-31, 2013.

R. Dale, I. Anisimoff, and G. Narroway, HOO 2012: A report on the preposition and determiner error correction shared task, Proceedings of the Seventh Workshop on Building Educational Applications Using NLP, NAACL HLT '12, pp.54-62, 2012.

N. Ballier,

R. Dale and A. Kilgarriff, Helping our own: The HOO 2011 pilot shared task, Proceedings of the 13th European Workshop on Natural Language Generation, ENLG '11, pp.242-249, 2011.

A. Díaz-negrillo, N. Ballier, and P. Thompson, Automatic Treatment and Analysis of Learner Corpus Data, 2013.

P. Flach, Machine Learning: The Art and Science of Algorithms that Make Sense of Data, 2012.

J. Friedman, T. Hastie, and R. Tibshirani, The Elements of Statistical Learning, vol.1, 2001.

J. Geertzen, T. Alexopoulou, and A. Korhonen, Automatic linguistic annotation of large scale L2 databases: The EF-Cambridge open language database (EFCAMDAT), Proceedings of the 31st Second Language Research Forum, 2013.

Y. Goldberg, Neural Network Methods for Natural Language Processing, 2017.

S. Granger, O. Kraif, C. Ponton, G. Antoniadis, and V. Zampa, Integrating learner corpora and natural language processing: A crucial step towards reconciling technological sophistication and pedagogical effectiveness, ReCALL, vol.19, issue.3, pp.252-268, 2007.
URL : https://hal.archives-ouvertes.fr/hal-00359051

J. A. Hawkins and P. Buttery, Criterial features in learner corpora: Theory and illustrations, English Profile Journal, issue.01, p.1, 2010.

J. A. Hawkins and L. Filipovi?, Criterial features in L2 English: Specifying the reference levels of the Common European Framework, English Profile Studies, vol.1, 2012.

D. Higgins, C. Ramineni, and K. Zechner, The Cambridge Handbook of Learner Corpus Research, Cambridge Handbooks in Language and Linguistics, pp.587-604, 2015.

, Machine Learning for learner English, vol.27

E. Hopman, B. Thompson, J. Austerweil, and G. Lupyan, Predictors of L2 word learning accuracy: A big data investigation, the 40th Annual Conference of the Cognitive Science Society, pp.513-518, 2018.

S. Jarvis, A Taste for Corpora: In Honour of Sylviane Granger, Amsterdam and Philadelphia, vol.45, pp.127-154, 2011.

S. Jarvis and M. Paquot, Learner corpora and native language identification, The Cambridge Handbook of Learner Corpus Research, pp.605-628, 2015.

Q. V. Le and T. Mikolov, Distributed representations of sentences and documents, 2014.

C. Leacock, M. Chodorow, M. Gamon, and J. Tetreault, Automated grammatical error detection for language learners, Synthesis Lectures on Human Language Technologies, vol.3, issue.1, pp.1-134, 2010.

T. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, Focal loss for dense object detection, Proceedings of the IEEE International Conference on Computer Vision, pp.2980-2988, 2017.

P. Lissón and N. Ballier, Investigating learners' progression in french as a foreign language: vocabulary growth and lexical diversity, 2018.

P. Lissón, Investigating the use of readability metrics to detect differences in written productions of learners: a corpus-based study, Bellaterra Journal of Teaching & Learning Language & Literature, vol.10, issue.4, pp.68-86, 2017.

B. Liu, Sentiment Analysis and Opinion Mining, 2012.

X. Lu, Automatic analysis of syntactic complexity in second language writing, International Journal of Corpus Linguistics, vol.15, issue.4, pp.474-496, 2010.

X. Lu, Computational methods for corpus annotation and analysis, 2014.

N. Ballier,

D. M. Magerman, Statistical decision-tree models for parsing, Proceedings of the 33rd Annual Meeting on Association for Computational Linguistics, pp.276-283, 1995.

S. Malmasi, K. Evanini, A. Cahill, J. Tetreault, R. Pugh et al., A report on the 2017 native language identification shared task, Proceedings of the 12th Workshop on Innovative Use of NLP for Building Educational Applications, pp.62-75, 2017.

D. Meurers, Learner corpora and natural language processing, 2015.

, The Cambridge Handbook of Learner Corpus Research, pp.537-566

M. Michalke, koRpus: An R Package for Text Analysis. (Version 0.10-2), Available at, 2017.

B. Mons, Data Stewardship for Open Science: Implementing FAIR Principles, 2018.

A. Murakami, Individual variation and the role of L1 in the L2 development of English grammatical morphemes: Insights from learner corpora, 2014.

A. Murakami, Modeling systematicity and individuality in nonlinear second language development: The case of English grammatical morphemes, Language Learning, vol.66, issue.4, pp.834-871, 2016.

K. P. Murphy, Machine Learning. A Probabilistic Perspective. Adaptive Computation and Machine Learning, 2012.
URL : https://hal.archives-ouvertes.fr/hal-02493464

H. T. Ng, S. M. Wu, T. Briscoe, C. Hadiwinoto, R. H. Susanto et al., The CoNLL-2014 shared task on grammatical error correction, Proceedings of the Eighteenth Conference on Computational Natural Language Learning: Shared Task, pp.1-14, 2014.

M. Nissim, L. Abzianidze, K. Evang, R. Van-der-goot, H. Haagsma et al., Sharing is caring: The future of shared tasks, Computational Linguistics, vol.43, issue.4, pp.897-904, 2017.

, Machine Learning for learner English, vol.29

A. O'keeffe and G. Mark, The English grammar profile of learner competence, International Journal of Corpus Linguistics, vol.22, issue.4, pp.457-489, 2017.

E. B. Page, The use of the computer in analyzing student essays". International Review of Education / Internationale Zeitschrift für Erziehungswissenschaft / Revue Internationale de l'Education, vol.14, pp.210-225, 1968.

M. Paquot and L. Plonsky, Quantitative research methods and study quality in learner corpus research, International Journal of Learner Corpus Research, vol.3, issue.1, pp.61-94, 2017.

P. Paroubek, S. Chaudiron, and L. Hirschman, Principles of evaluation in natural language processing, Traitement Automatique des Langues, vol.48, pp.7-31, 2007.
URL : https://hal.archives-ouvertes.fr/hal-00502700

A. Rich, P. O. Popp, D. Halpern, A. Rothe, and T. Gureckis, Modeling second-language learning from a psychological perspective, Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications, pp.223-230, 2018.

E. F. Sang and F. De-meulder, Introduction to the conll-2003 shared task: Language-independent named entity recognition, pp.142-147, 2003.

B. Settles, Data for the 2018 Duolingo shared task on second language acquisition modeling (SLAM), 2018.

B. Settles, C. Brust, E. Gustafson, M. Hagiwara, and N. Madnani, Second language acquisition modeling, Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications, pp.56-65, 2018.

M. D. Shermis, J. Burstein, D. Higgins, and K. Zechner, Automated essay scoring: Writing assessment and instruction, International Encyclopedia of Education, pp.20-26, 2010.

J. Tetreault, J. Burstein, E. Kochmar, C. Leacock, and H. Yannakoudakis, Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications, 2018.

N. Ballier,

J. Thewissen, Accuracy Across Proficiency Levels: A Learner Corpus Approach. Louvain: Presses universitaires de Louvain, 2015.

S. Thrun and L. Pratt, Learning to Learn, 1998.

S. Vajjala and K. Loo, Automatic CEFR level prediction for Estonian learner text, NEALT Proceedings Series, vol.22, pp.113-128, 2014.

E. Volodina, I. Pilán, and D. Alfter, Classification of Swedish learner essays by CEFR levels". CALL Communities and Culture-Short Papers from EURO-CALL, pp.456-461, 2016.

K. Wisniewski, Empirical learner language and the levels of the Common European Framework of Reference, Language Learning, vol.67, issue.S1, pp.232-253, 2017.

H. Yannakoudakis, T. Briscoe, and B. Medlock, A New dataset and method for automatically grading ESOL texts, Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, vol.1, pp.180-189, 2011.

H. Yannakoudakis, E. Kochmar, C. Leacock, N. Madnani, I. Pilán et al., Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, 2019.

, Number of sentences. 2. words: Number of words. 3. letters: Named vector with total number of letters

, length: Average sentence length (number of words per sentence)

. Word,

. Word, Average number of syllables per word 8. sntc.per.word: Number of sentences per word

, Machine Learning for learner English, p.31

, TTR: type to token ratio

, ARI: Automated Readability Index. It takes into account the number of token words divided by the number of syllables and the number of prepositions in the text

, It gives an estimation of the grade required to understand the text. The computation is based on the most frequent 3 000 words, Bormuth: Bormuth readability index

C. , C1: Readability Formulas, taking into account monosyllabic words 13. Coleman.C2: variant also taking into account the number of words divided by the number of sentences

C. , C3 variant also taking into account the proportion of pronouns among words

C. , C4 variant also taking into account the proportion of prepositions among words

C. , Liau : readability index, proportional to the number of letters and sentences

. Dale, readability index, 1995.

. Danielson, . Bryan, . Db1-&-danielson, and . Bryan, DB2 : two readability formulas based on the number of characters

. Dickes, Steiwer: readability for German that takes into account values proportional to the number of words, characters and TTR

, Degrees of Reading Power) measure readibility from Bormuth index. 21. ELF : (Easy Listening Formula): number of polysyllabic words divided by the number of sentences, DRP

. Farr and . Jenkins, Paterson: a simplified version of Flesch, where the number of one syllable words per 100 words replaces the number of syllables per 100 words

, Flesch : the English values for this language-dependent metrics have been used. The index takes into account the number of syllables

. Flesch and . Kincaid, this metric was developed with Vietnam draftees to assess the US school grade corresponding to the difficulty level of a text

, FOG : readability index suggested in the 1950's. It measures the number of years of study (school grade) required to understand a text on its first reading

N. Ballier, It takes into account the number of words per sentence and the proportion of words with three syllables or more

, ) a method implemented with Vietnam draftees

, Fucks : a stylistic feature proposed by W. Fucks. The number of characters divided by the number of words is multiplied by the number of words divided by the number of sentences

. Linsear, Write : readability index that takes into account the number of words of three syllables or more, the number of words and the number of sentences

, LIX : this readability index was first proposed for Swedish, it takes into account the proportion of words of seven letters or more. Texts with a 25 index are supposed to be easy to read

, nWS1 to nWS4 : these readability indices proposed in the 80's for the analysis of German (Neue Wiener Sachtextformeln), take into account -in variable proportions-words of three syllables or more and words of six letters or more

, RIX : adaptation for English of the LIX index. It takes into account the number of six letters or more divided by the number of sentences

, Readability Index based on the square root of the number of polysyllabic words computed at the beginning

, Spache : readability index based on the number of words of a text that is not in Spache reference inventory of words

, Strain : readability index for medias proposed in 2006, which takes into account the number of syllables

. Traenkle and . Bailer, TB1 & Traenkle.Bailer.TB2 : readability indices taking into account the proportion of prepositions (Traenkle.Bailer.TB1) and conjunctions (Traenkle.Bailer.TB2)

, Kuntzsch's Text-Redundanz-Index) readability index initially suggested for German newspapers, it takes into account the number of punctuation symbols and foreign words

, Tuldava: a supposedly language-independent readability index that takes into account the logarithm of the number of words divided by the number of sentences

. Wheeler, Smith: readability index proposed in the 1650s that takes into account words of two syllables ore more

, CTTR : algorithm proposed by Carroll to smooth TTR

. Hd-d-(vocd-d), lexical sophistication index

, Herdan's C : log(V) / log(N), where V is the number of types and N the number of tokens

, Maas & lgV0 : indices of lexical complexity suggested in 1972, which take into account logarithms of types and tokens ' 43. MATTR: (Moving Average of TTR), computed by means of a mobile window

, MSTTR (Mean Segmental Type-Token Ratio): averages TTR over several segments

, MTLD (Measure of Textual lexical sophistication): corrected measure of the TTR 46. Root TTR : rooted square TTR 47. Summer: lexical sophistication index

, TTR.1 : rounded Type-Token ratio

, Yule's K : lexical sophistication index proposed by Yule in 1944