ChatGPT entered the Spanish language and literature exam of the university entrance exams in Catalonia yesterday as if he were just another student. And how has your performance been? Did you study enough? What grade would the examiners have given him if he had been another student?
The first difference with a human is that this chatbot was able to take the assessment in just a few minutes, without the need to travel to a university classroom, without having to prepare for the exam and without facing to the attack of nerves that students experience when they face the first test of selectivity.
Flawless in shape, yes he answered it. Once completed, La Vanguardia has sent it to teacher Montserrat Torra, teacher of Spanish language and literature at high school, now retired, who has accepted the proposal to correct the exam following the established evaluation criteria and that this teacher knows well
A first impression is that ChatGPT has managed to get away with the reading comprehension exercises. “Correct content and well adjusted to what was requested in the statement”, says Torra. However, his comments are sparse and superficial, as well as not very expressive. Let’s say little… humans.
On the other hand, it seems that the chatbot has never enjoyed reading the Madrid writer Antonio Buero Vallejo or the Catalan author of Nada, Carmen Laforet.
It is not that he has left blank the questions that he did not know, as prudent students in his position would have done, no; instead, the artificial intelligence has chosen to “invent the answers (perhaps as other clever students had done)”. If it leaks, it leaks. But, indeed, the answers have not been valid.
“Answer only TWO of the following three questions about La Fundación, by Antonio Buero Vallejo. Use, therefore, no more than thirty words in the first two sectionsâ€. To the first question “How many parts does the Foundation have?” And how many paintings?”, the artificial intelligence answered: “The Foundation has three parts. The number of paintings is not mentioned”. The correct answer is that the work has two parts and that each one has two paintings.
The next: “What is Asel’s secret project?” And he answered: “The secret of Asel is not specified in the text”. Well no Asel plans to get the transfer of the punishment cells to try to escape.
“Which of the following statements is correct? 1) Max commits suicide. 2) Asel prevents Tomás from committing suicide. 3) Berta is a real character. 4) Lino betrays his colleagues. Here the conversational chatbot has chosen not to invent anything, but to be honest in an almost human way: “I can’t answer this question”.
As far as written expression is concerned, the conversational assistant has had no problems getting up to speed. “He answered option A without making any mistakes, while in option B there was a small mistake,” comments Professor Torra.
Artificial intelligence has had difficulties that it has not been able to overcome to distinguish between “a parte” and “aparte”. If I hadn’t failed the made up answers and some mistake as big as a house…
“Reading and writing, a 10! With some grammatical knowledge error”, says professor Montserrat Torra. Once the reading comprehension, written expression and linguistic reflection answers of the exam have been added up, the IA student has obtained a rating of 8 out of 10. Both in option A and B.
