LLMs write worse code if you ask them to return the code wrapped in JSON via a tool function call.

George

Future

Community Picks is a section on daily.dev where our community members share the most interesting and valuable content they've discovered online. From insightful articles to handy tools, every post is a gem curated by our dedicated coomunity. To contribute to Community Picks, you need to have at least 250 reputation points, ensuring that only active and trusted members can share their finds.

Community Picks

Benchmarks show that Language Learning Models (LLMs) produce lower quality code when returning it in a structured JSON response compared to plain text formats. Models struggle with syntax errors related to quoting and escaping code in JSON, impacting their ability to solve coding problems effectively. Despite advancements like OpenAI's strict JSON support, the challenges remain, suggesting a preference for plain text code generation.

LLMs are bad at returning code in JSON