Ramp built LLM-powered agents to automate expense approvals, achieving 65% automation rate. Key strategies include transparent reasoning with citations, allowing agents to express uncertainty, using categorical confidence levels instead of numerical scores, collaborative context improvement, and progressive autonomy from suggestions to full automation. The approach emphasizes building user trust through clear explanations, proper guardrails, and continuous evaluation with golden datasets.
Table of contents
Choose problems where LLMs can shineShow your workBuild escape hatchesContext should be collaborativeGive users an autonomy sliderEvals are the new unit testsBuilding agentsSort: