One-Second Voice-to-Voice Latency with Modal, Pipecat, and Open Models

Building a conversational voice AI bot with sub-second response latency using Modal's serverless platform, the Pipecat framework, and open-source models. The implementation achieves ~1 second voice-to-voice latency by combining Parakeet STT, Qwen3 LLM with vLLM, and Kokoro TTS. Key optimizations include using Modal Tunnels to bypass the input plane, WebRTC for client connections, regional pinning to minimize network latency, and independent autoscaling of GPU services. The demo includes a RAG-powered assistant for Modal's documentation with structured outputs and animated avatars.