Marc Klingen, co-founder of Langfuse, shares lessons from building a coding agent skill to help developers integrate Langfuse into their projects. Key challenges included outdated pre-training context causing hallucinations, 478 pages of documentation, and non-optimal agent setups. Six main learnings emerged: (1) looking at traces gets you 80% of the way; (2) production signals like a search endpoint help agents navigate docs; (3) exposing an agent sitemap improves documentation discovery; (4) even a basic eval setup is better than none; (5) dynamic content should be referenced rather than duplicated in the skill; (6) auto-research using agents to improve the skill itself is powerful but heavily dependent on defining the right target function. The talk also covers open questions around skill versioning, distribution, and the tension between getting users to a quick 'aha moment' versus a fully optimized setup.
Sort: