Hallucination in Large Language Models (LLMs) and Its Causes

Large language models (LLMs) like Llama, PaLM, and GPT-4 have advanced text understanding and generation in natural language processing (NLP). However, LLMs are prone to producing hallucinations, which are factually incorrect or inconsistent content. Hallucinations in LLMs can be categorized into factuality hallucinations and faithfulness hallucinations. The causes of hallucinations span data-related, training-related, and inference-related factors. Mitigation strategies for hallucinations include enhancing data quality, improving training processes, and refining decoding techniques.