A practical guide to observability for AI agents in production, written from first-hand experience. Covers why traditional logging falls short for agents, how to structure session traces, what to log in each trace, and how to name and tag spans for queryability. Recommends tools like Langfuse, LangSmith, Braintrust, and
Table of contents
Why Agent Observability Is Different From Regular LoggingSession Traces: The One Thing You Cannot Live WithoutNaming and Structuring Traces So You Can Actually Find ThingsEvals: The Part Most Developers SkipRunning Evals in Production (Not Just Before Deploy)Cost and Token ObservabilityDebugging Non-Deterministic FailuresPrivacy and Data HandlingThe Minimal Setup That Actually WorksThe Mindset Shift That MattersSort: