This formal warning regarding the disturbing phenomenon of AI Hallucination, straight from the Beast <ahem>Meta Corp:
Its written as a disclaimer for their newly christened Galactica AI, which was spawned in late 2022 with the intention of untangling, organizing, and drawing genuine insights from the massive mountain of every peer-reviewed scientific paper ever published in the history of humanity.
That said, the cautions it states are equally applicable to just about every modern AI platform that uses natural language understanding in both its training data and as its primary interaction modality (i.e. GPT-3, GPT-4, etc.).
Read it and weep: (with equal amounts humor and terror)
A.I. Limitations
You should be aware of the following limitations when using any A.I. spawned from a Deep Learning / Large Language Model (LLM) foundation:
- AIs can & do Hallucinate.There are no guarantees for truthful or reliable output from AIs, even very powerful ones trained on high-quality data like Galactica.
NEVER FOLLOW
ADVICE FROM AN A.I.
WITHOUT INDEPENDENT HUMAN EXPERT VERIFICATION.
- LLM-based AIs are Frequency-Biased. Galactica is good for generating content about well-cited concepts, but does less well for less-cited concepts and ideas, where hallucination is more likely.
- AIs are often 100% Confident Yet Factually Wrong. Some of Galactica’s generated text may appear very authentic and highly-confident, but might be subtly wrong in very important ways. This is particularly the case for highly technical content.
So there it is, from the Horse’s mouth, as it were.
The same horse that has already left the barn?
The same horse that perhaps has gone completely off the reservation?
You tell me.
AI Hallucination. Electric Kool-Aid Acid Test for Machines.
Caveat Sapiens.