Serving AI models with Open Inference Protocol APIs

Arcjet describes the architecture behind their AI-powered prompt injection detection service. They use Python with FastAPI to wrap inference models, expose them via the Open Inference Protocol v2 (KServe v2) as a standard API contract, host everything on Modal for GPU-backed serverless deployment, and call the models from their core Go backend via a generated OIP client. The layered design separates model inference, API transport, and security decision logic, enabling parallel model backends, token-based billing, fail-open error handling, and easy addition of new model-backed security rules without changing the Go service.