LLMs As Bullshit Machines
LLMs don't lie. They are simply indifferent to truth.
Nice paper making the rounds that I'm so glad someone invested the time to write. I've been saying this for a while: LLMs don't lie. They spout bullsh*t.
Paper: Hicks, M.T., Humphries, J. & Slater, J. ChatGPT is bullshit. Ethics Inf Technol 26, 38 (2024). https://doi.org/10.1007/s10676-024-09775-5 (open access)
From the paper, bullsh*t is defined as: "not by an intent to deceive but instead by a reckless disregard for the truth" and "a lack of concern with truth, or an indifference to how things really are". It goes on to distinguish soft bullsh*t (indifference) from hard bullsh*t (intent to deceive). And as discussed in the article, the term "confabulation" is probably better than "hallucination." But bullsh*t is probably an even better term.
I also look at it this way. You should consider everything an LLM says to be a hallucination/confabulation/bullsh*t. Some of the hallucinations are close enough to what we think of as "true" or credible that we think the output is true. But it's just a lucky guess (best case), or you aren't paying enough attention to see the problems (try magnifying hands on most AI-generated pictures with people on social media), or we don't know enough to realize there are potentially harmful falsehoods (worst case). In other words, the degree of truth assigned is a reflection of chance, the reader's attention, and the reader's expertise, and is not related to the process used to create LLM output.
LLMs are indifferent to truth => bullsh*t generators.
Very often the results are close enough to true that they can be useful. But that easily seduces us into automation complacency. Then we start missing the problems when we review outputs before publishing them as our own work.
This paper is worth keeping on hand if you need to explain a fundamental limitation to LLMs. I'm not saying it is impossible to improve this with enough billions of dollars -- just that you need to understand fundamental limitations to use the technology responsibly.
h/t Heather Roff
Video clip with the reference from the Blues Brothers: