Learn how Salesforce developed a hybrid on-device and cloud architecture to translate unstructured voice input into structured form data, and much more.

The Salesforce Engineering Blog offers a deep dive into Salesforce technologies, providing technical insights, best practices, and real-world examples. Developers can explore topics such as Salesforce development, architecture, and integrations, gaining  knowledge to build innovative solutions on the Salesforce platform. With contributions from Salesforce engineers and industry experts, the blog serves as a resource for the Salesforce community.

Salesforce Engineering

Salesforce's Field Service Mobile team built a hybrid voice-to-form AI system for field technicians who work in challenging conditions. The architecture keeps speech-to-text on-device using native iOS/Android frameworks to eliminate transcription costs, protect audio privacy, and reduce latency, while sending only text to cloud LLMs for semantic field mapping. Schema-driven metadata is encoded into prompts to handle diverse form structures without hardcoded rules. The system achieves 85% field-level accuracy and completes end-to-end in under 15 seconds. Reliability was validated using a Voice Utterance Library of real technician recordings combined with noise profiles. UX prioritizes simplicity with inline editing, visual field highlights, and no stored audio.

Delivering Low-Latency Voice-to-Form AI in Real-World Field Conditions