Inventing compelling databases for simulating clinical trials has never been easier. That’s the conclusion of three researchers from the ophthalmology department of the Magna Graecia University of Catanzaro, in Italy, after asking the latest version of ChatGPT to create from scratch the results of a scientific investigation that never existed for support a hypothesis not proven by science. The bogus data, which showed that one surgical technique is better than another when in fact it is not, appeared authentic at first glance.
After the results of the experiment were presented in the journal JAMA Ophthalmology, the researchers warn that it was easy for them to carry out this falsification, which, in their opinion, can have consequences for the integrity of scientific research . “In just a few minutes you can create a database that is not based on real data, and that is also opposed to the available evidence,” said Giuseppe Giannaccare, an eye surgeon and one of the authors of the work, in Nature magazine .
The issue is not limited to malpractice per se, but also has an impact on how the public and the scientific community itself may perceive new developments. “Generative AIs can pose a risk to the credibility of scientific studies,” Ramon López de Mántaras, founder and former director of the CSIC’s Artificial Intelligence Research Institute, told La Vanguardia.
The authors point out that a deep analysis of the database, for now, does allow for the identification of falsehood. In fact, once Nature scrutinized the study results, it identified details that suggested they were fabricated: the names of the alleged clinical trial participants did not match their gender, the ages were suspiciously distributed, and pre- and post-operative results were not correlated.
However, such detailed analyzes are not common in the review process of scientific articles, which means that identifying falsification is not a simple task. In addition, “databases generated by AI can become more and more difficult to distinguish from the real ones as AI technology progresses”, points out Andrea Taloni, co-author of the study.
This is why the Italian team recommends that the control be prior, within the research centers themselves. Taking the data with digital tools to keep an encrypted and immutable copy and registering the clinical trials and protocols with the reference entities are probably the best guarantee of real research.
Taloni also suggests investing in the development of specialized software to identify artificially created data. The idea is that, similar to the tools that are being developed to detect AI-generated texts, “recognize anomalous patterns in false databases”, indicates the Italian expert. Fraud and malpractice in the world of science are not new. Precedents include the manipulation of data to falsely associate vaccines with autism, to the falsification of images and graphs in Alzheimer’s studies. In Spain, 3.6% of the country’s biomedical researchers admit to having manipulated data at some point, according to a recent study. “Before generative AI, there were quite a few scandals of people who invented data or who made false graphs of results”, reflects López de Mántaras. “The problem is that, with generative AI, this is easier to do and, therefore, it can proliferate.”
In other words, tools like ChatGPT do not create a new threat to the scientific world, but rather exacerbate existing problems. “Fraudulent scientists, without any ethical criteria and with bad practices, have tools at their disposal that can now make it easier for them to falsify results”, concludes the expert.
Part of the problem, argues Albert Sabater, director of the Observatory of Ethics in Artificial Intelligence of Catalonia (Oeiac), is that the development of technologies such as the OpenAI chatbot are not transparent. We don’t know how they develop, we don’t know how they get their results, and we also can’t replicate the system, key questions in scientific research. This makes it even more difficult to discern which content is real and which has been artificially created.