Introducing Rubrics: Build Agents that Evaluate and Correct Their Work
RubricMiddleware enables agents to self-evaluate and iteratively refine their outputs to meet specific criteria, addressing the issue of outputs that pass general checks but fail detailed requirements. It uses a grader sub-agent to provide feedback and ensure all rubric criteria are satisfied, offering greater flexibility than previous implementations. The middleware allows for defining a rubric once and applying it to an agent, with the grader reviewing the agent's output and iterating until all criteria are met or a configured number of attempts are reached.
More from Langchain
Why managed agents are the next big thing in agent building
Managed Deep Agents gives developers a managed way to build, run, and deploy Deep Agents with built-in runtime, streaming, sandboxes, evals, memory, and auth.
langchain.com··Mads Kristian Nylund·AI InfrastructureAI-agents
LangSmith BYOC on AWS is generally available
LangSmith Bring Your Own Cloud is now generally available on AWS, giving Enterprise teams managed observability, evaluation, and deployment inside their own VPC.
langchain.com··Mads Kristian Nylund·AI DevelopmentAI InfrastructureAI Platforms
What is an AI agent?
Learn what AI agents are, how they work in an LLM loop, and where workflows fit so you can build reliable, production-ready autonomous systems.
langchain.com··Mads Kristian Nylund·AI DevelopmentAI InfrastructureAI-agents