Syntactic complexity analysis of the written text production of the Pázmány Basic English language examination: A pilot study
Karlygash Adamova | PPKE Nyelvtudományi Doktori Iskola
The piloting study is one of the stages of the multi-dimensional analysis of the written text production part designed to investigate of the Pázmány Basic English language examination (BLE) validation process.
The specific research question of this pilot study is whether or not the selected syntactic complexity measures are the predictive patterns of syntactic complexity for writing quality and can investigate the BLE validation process. In this pilot study, validation of the chosen examination is tried to accomplish by the investigation of textual and syntactic variables in the written texts.
The study is concerned with the analysis of the textual and syntactic features of 40 written texts of the students. The methodology of the study contains a mixed-method analysis.
In case of the computational analysis the L2 Syntactic Complexity Analyzer (Lu, 2010) was applied. The eight selected variables are: length of the text, number of sentences, mean length of sentences, number of T-units, number of clauses, mean length of T-units, mean length of clauses, mean of clauses per sentence.
By the manual analysis the texts were examined from different aspects, including Finite Verb Phrases, Perfect Aspect, Passive, Progressive, Subordination, Coordination and Conditional Subordination.
All in all, the study was aimed to test the selected variables in order to prove that the BLE meet all the expectations of the B2+ level (CEFR; Council of Europe, 2001). Therefore, strong correlations between the syntactic and textual variables and the points from the test were expected to be established.
Értékeld a posztert:
Megosztás social media felületen:
Dear Karlygash,
Thank you for the interesting presentation! I found the correlations especially interesting. I have got three questions:
1) Have you found any correlations that are unusually high or unusually low in comparison to similar texts written by native speakers or speakers with a different L1?
2) Which other variables do you recommend to include in similar analyses to find stronger correlation (see conclusion)?
3) Which correlations did you expect to be higher?
Regards,
Balázs
Dear Mr. Balázs Kovács!
Thank you very much for your comments and questions!
Let me answer them one by one, please.
1) First of all, thanks for an interesting question. Comparison with the NS (native-speakers) or L2 English learners with a different L1 is an attractive way of analysis, however, in the case of my study, the aim and objectives are different and thus, another approach was applied. I was intended to look at the written text production of L2 learners with a Hungarian background only. More precisely, the main aim of my study is the investigation of the exam validation process with the help of the selected syntactic complexity measures.
2) As it was illustrated in my poster, altogether 15 variables were selected for this piloting study. Moreover, it is important to highlight that each variable was selected based on previous studies and logical presumptions of correlations between the variables. Unfortunately, the results of this piloting stage did not conform to expected correlations. Therefore, for the next piloting stage, I am planning to implement another automated tool, which includes more syntactic and textual variables. Particularly, I am testing the Multidimensional Analysis Tagger (Nini, 2019), which contains 67 linguistic variables (Biber, 1988).
3) I expected to find significant correlations between grammatical variables and the grammar points, which would mean that there is a relationship between the usage of some linguistic features and the points.