Hallucination in large language models usually refers to the model generating unfaithful, fabricated, inconsistent, or nonsensical content. As a term, hallucination has been somewhat generalized to cases when the model makes mistakes. Here, I would like to narrow down the problem of hallucination to be when the model output is fabricated and not grounded by either the provided context or world knowledge.
There are two types of hallucination:
In-context hallucination: The model output should be consistent with the source content in context.

Lilian Weng is a machine learning researcher and writer who shares insights, research findings, and tutorials on machine learning, artificial intelligence, and data science. Through articles, blog posts, and research summaries, Lilian Weng explores topics such as deep learning, natural language processing, and reinforcement learning. Readers can learn about state-of-the-art algorithms, practical applications of machine learning, and trends shaping the field of AI.

Lil’Log

Hallucination in large language models (LLMs) refers to generating unfaithful, fabricated, or nonsensical content not grounded in provided context or world knowledge. The focus is on extrinsic hallucination, emphasizing the need for LLMs to produce factual content and acknowledge when they lack knowledge. Causes of hallucination include issues during pre-training and fine-tuning stages. Various methods, like retrieval-augmented generation (RAG), special sampling methods, and fine-tuning for factuality, are explored to minimize hallucinations. Evaluation benchmarks and different detection approaches are also discussed.

Extrinsic Hallucinations in LLMs