How many of your agent's calls actually need a frontier model?
The article discusses how model routing can reduce costs by offloading less expensive models for certain tasks, with a cost tradeoff formula that determines how many turns should be routed to a frontier model. It highlights that in most cases, 7% of calls can be offloaded without needing a frontier model, and using a routing library like Switchyard helps automate this process. The formula is most effective when there is a significant price difference between models, allowing for substantial cost savings.
More from Langchain
Why managed agents are the next big thing in agent building
Managed Deep Agents gives developers a managed way to build, run, and deploy Deep Agents with built-in runtime, streaming, sandboxes, evals, memory, and auth.
langchain.com··Mads Kristian Nylund·AI InfrastructureAI-agents
LangSmith BYOC on AWS is generally available
LangSmith Bring Your Own Cloud is now generally available on AWS, giving Enterprise teams managed observability, evaluation, and deployment inside their own VPC.
langchain.com··Mads Kristian Nylund·AI DevelopmentAI InfrastructureAI Platforms
What is an AI agent?
Learn what AI agents are, how they work in an LLM loop, and where workflows fit so you can build reliable, production-ready autonomous systems.
langchain.com··Mads Kristian Nylund·AI DevelopmentAI InfrastructureAI-agents