Tech giant OpenAI has been promoting its AI-powered transcription tool, Whisper, as highly accurate and robust. However, recent studies have shown that Whisper has a significant flaw – it invents fake statements and hallucinations. These fabrications can include racial commentary, violent rhetoric, and even made-up medical treatments.
This issue is particularly concerning because Whisper is being used in various industries worldwide, including translating interviews, generating text for consumer technologies, and creating subtitles for videos. Despite OpenAI’s warnings against using the tool in high-risk domains, medical centers have been rushing to utilize Whisper-based tools to transcribe patients’ consultations with doctors.
Researchers and engineers have reported coming across Whisper’s hallucinations frequently in their work. For example, a University of Michigan researcher found hallucinations in 8 out of every 10 audio transcriptions he inspected. Another machine learning engineer discovered hallucinations in about half of the over 100 hours of Whisper transcriptions he analyzed.
The implications of these hallucinations could be severe, especially in hospital settings where misdiagnoses could have grave consequences. Experts and advocates have called for AI regulations and for OpenAI to address the flaw in Whisper. The prevalence of hallucinations has raised concerns about the accuracy and reliability of AI-powered transcription tools.
Whisper is integrated into various platforms and services, such as OpenAI’s ChatGPT, Oracle, and Microsoft’s cloud computing platforms. It is widely used for transcribing and translating text into multiple languages. Despite the widespread use of Whisper, the tool’s hallucinations have raised red flags among experts and researchers.
In the healthcare sector, over 30,000 clinicians and 40 health systems have started using a Whisper-based tool developed by Nabla to transcribe doctor-patient interactions. While Nabla is aware of Whisper’s tendency to hallucinate, they are working to mitigate the problem. However, erasing the original audio for data safety reasons could pose challenges in verifying the accuracy of the transcriptions.
Privacy concerns have also been raised regarding AI-generated transcripts of patient consultations. Lawmakers and healthcare providers are grappling with how to ensure patient confidentiality while using AI transcription tools. State and federal privacy laws are in place to protect patient information, but the use of AI tools in healthcare settings raises new privacy considerations.
Overall, the issue of AI-powered transcription tools inventing fake statements underscores the importance of ensuring the accuracy and reliability of such tools, especially in critical domains like healthcare. As technology continues to advance, it is crucial to address potential flaws and mitigate risks to prevent serious consequences in real-world applications.
