BoxLang AI 3.0 ships six built-in middleware classes that address the most common cross-cutting concerns for AI agents: logging, retry with exponential backoff, guardrails for blocking dangerous tool calls, human-in-the-loop approval (CLI and web modes), a flight recorder for record/replay testing, and a max-tool-calls cap to prevent runaway agents. Middleware hooks fire before and after LLM calls, tool invocations, and agent runs, with typed flow control via AiMiddlewareResult. Custom middleware can be written as a struct of closures or a full class. The recommended production stack is logging + retry + guardrails, with additional middleware added based on agent risk profile. The FlightRecorderMiddleware in particular enables deterministic CI testing of non-deterministic agents by capturing and replaying LLM and tool interactions from JSON fixture files.
Table of contents
The Middleware ArchitectureAiMiddlewareResult — Typed Flow ControlLoggingMiddleware — Instant ObservabilityRetryMiddleware — Resilience Without BoilerplateGuardrailMiddleware — Defense in DepthHumanInTheLoopMiddleware — Keeping Humans in ControlFlightRecorderMiddleware — AI Testing SolvedMaxToolCallsMiddleware — Runaway Agent PreventionWriting Your Own MiddlewareComposing MiddlewareWhat's NextSort: