[2411.10541] Does Prompt Formatting Have Any Impact on LLM Performance?

Prompt optimization plays a crucial role in the performance of Large Language Models (LLMs). This study examines how different prompt templates, such as plain text, Markdown, JSON, and YAML, affect the performance of models like GPT-3.5-turbo and GPT-4 across tasks like natural language reasoning, code generation, and translation. Results indicate that GPT-3.5-turbo's performance in code translation can vary by up to 40% depending on the template used, whereas GPT-4 shows more robustness to these variations.