Dis Is Weird

An exploration of ChatGPT's behavior when given a simple text replacement task reveals concerning patterns: the system makes unwarranted assumptions, exhibits incuriosity by not asking clarifying questions, and produces unexpected errors like replacing 'this' with 'dis' in its own responses. The experiment demonstrates how even basic prompts can expose reliability issues in LLMs, including hallucination and incorrectness, raising questions about their use in critical applications where accuracy matters.