Langchain
80 articles · 13 videos
80 articles · 13 videos
Managed Deep Agents gives developers a managed way to build, run, and deploy Deep Agents with built-in runtime, streaming, sandboxes, evals, memory, and auth.
langchain.com··Mads Kristian Nylund·AI InfrastructureAI-agents
LangSmith Bring Your Own Cloud is now generally available on AWS, giving Enterprise teams managed observability, evaluation, and deployment inside their own VPC.
langchain.com··Mads Kristian Nylund·AI DevelopmentAI InfrastructureAI Platforms
Learn what AI agents are, how they work in an LLM loop, and where workflows fit so you can build reliable, production-ready autonomous systems.
langchain.com··Mads Kristian Nylund·AI DevelopmentAI InfrastructureAI-agents
We benchmarked NVIDIA NeMo Switchyard on 145 agent tasks. Only 7% of turns needed a frontier model, and routing cut cost 74% for six points of accuracy.
langchain.com··Mads Kristian Nylund·Open Source AIAI ToolsAI Infrastructure
Run deep agents in production with durable execution, sandboxes, tool access, and LangSmith observability, without building the runtime yourself. Now in private beta
langchain.com··Mads Kristian Nylund·AIAI-agents
Deep Agents, LangChain, and LangGraph each offer distinct approaches to building agents. In this post, we cover the key distinctions between our open source frameworks and when you should reach for each one.
langchain.com··Mads Kristian Nylund·Open Source AIAI InfrastructureAI-agents
Learn how LangChain built an autonomous SRE agent for Kubernetes deployments with Deep Agents, human approval for changes, LangSmith tracing, and evals.
langchain.com··Mads Kristian Nylund·AI ToolsAI InfrastructureAI-agents
Learn how to evaluate voice agents across execution, outcomes, and caller experience using LangSmith traces, code evaluators, LLM judges, and human review.
langchain.com··Mads Kristian Nylund·AI ToolsAI InfrastructureAI Evaluation
Customer Experience (CX) Agents in Production: Lessons from Lyft, Vodafone, and LATAM Airlines
langchain.com··Mads Kristian Nylund·AI ToolsAI-agentsAI Evaluation%20Agents%20in%20Production%20-%20Banner.png)
Learn how Stripe built Kai, a company-wide AI agent on LangChain, LangGraph, and Deep Agents, reaching 5,000 users in roughly 4 weeks.
langchain.com··Mads Kristian Nylund·AI ToolsAI InfrastructureAI Platforms
Learn how LangChain built ReviewBench, a benchmark for evaluating code review agents against real PR feedback from trusted reviewers.
langchain.com··Mads Kristian Nylund·AI DevelopmentAI ToolsAI Evaluation
Introducing LangSmith LLM Gateway: runtime governance for AI agents with spend limits, PII redaction, and trace continuity, built directly into LangSmith.
langchain.com··Mads Kristian Nylund·AI ToolsAI InfrastructureAI Governance
Today we're shipping deep agents v0.7. This release simplifies the base harness, resulting in 65% fewer base input tokens at comparable performance.
langchain.com··Mads Kristian Nylund·AI DevelopmentAI ToolsAI Infrastructure
Learn how Similarweb uses LangSmith to evaluate long-form agent research reports with rubrics, faithfulness checks, traces, and baseline comparisons.
langchain.com··Mads Kristian Nylund·AI ToolsAI InfrastructureAI Evaluation
Learn how LangChain used Hex, dbt, semantic models, and observability to build a trusted data agent and scale self-service analysis by 40x.
langchain.com··Mads Kristian Nylund·AI DevelopmentAI ToolsAI Infrastructure
SmithDB supports full-text search and JSON filtering over agent traces with a median (P50) latency of 400 ms, even though the underlying data consists of large, deeply nested JSON documents stored in object storage.
langchain.com··Mads Kristian Nylund·AI DevelopmentAI InfrastructureAI Evaluation
Learn why companies must own their agent systems, governance, context, and feedback loops to turn generic AI into lasting business advantage.
langchain.com··Mads Kristian Nylund·AI ToolsAI GovernanceAI Ethics
We revamped how we benchmark Deep Agents. Here's the eval setup we run in Harbor across coding, conversation, and retrieval, and how we use it to ship changes.
langchain.com··Mads Kristian Nylund·Open Source AIAI InfrastructureAI Evaluation
LangChain's Eval Engineering Skill inspects your agent's repo and traces, proposes evals through user interviews, and outputs runnable Harbor tasks.
langchain.com··Mads Kristian Nylund·AI DevelopmentAI Tools
Graph engineering isn't a new idea. It's the latest name for a well established approach to building reliable agents. It's the same idea behind loop engineering and harness engineering: building putting model reasoning in the right places, with the right context, at each step. At LangChain, we've been helping people build agents with graphs for 3 years! Here's what we've learned.
langchain.com··Mads Kristian Nylund·AI DevelopmentAIAI-agents.png)
LangSmith now supports tracing for voice agents built with Pipecat, LiveKit, OpenAI Realtime, and Gemini Live. Capture audio, STT and TTS latency, interruptions, tool calls, and more in one trace.
langchain.com··Mads Kristian Nylund·AI ToolsAI InfrastructureAI Observability
Apollo uses Deep Agents and LangSmith to power an AI Assistant that handles prospecting, enrichment, outreach, analytics, and MCP integrations.
langchain.com··Mads Kristian Nylund·AI DevelopmentAI InfrastructureAI Platforms
The gateway is the runtime control plane for enterprise AI, turning policy into enforceable decisions across every model call, tool call, and agent hop.
langchain.com··Mads Kristian Nylund·AI DevelopmentAI InfrastructureAI Governance
Learn how LangChain built IssueBench, a synthetic benchmark for evaluating how well LangSmith Engine identifies, categorizes, and groups issues in agent traces.
langchain.com··Mads Kristian Nylund·AI BenchmarkingAI InfrastructureAI Evaluation
OpenWiki 0.2 generates codebase wikis in the OKF format, helping developers organize repo docs with metadata, changelogs, and agent-friendly retrieval.
langchain.com··Mads Kristian Nylund·Open Source AIAI ToolsAI Infrastructure
For most of computing history, a developer environment meant a physical machine, then a VM, then a container, with each one shared across the work happening on it, each one requiring deliberate setup and teardown. The idea of giving every agent its own isolated computer, booting in under a second and cleaning up when done wasn't practical at scale. Sandboxes give you a safer way to run upgraded workflows, providing you an isolated environment to run tasks that require iteration, verification, and access to the tools a person would normally use, all without requiring human supervision. Every human developer gets a laptop. Every agent can get a computer.
langchain.com··Mads Kristian Nylund·AI SecurityAI InfrastructureAI
Use LangSmith to trace coding agents across Claude Code, Codex, Cursor, Copilot, and more. Inspect tool calls, subagents, errors, costs, and retries.
langchain.com··Mads Kristian Nylund·AI DevelopmentAI ToolsAI Observability
Learn how Deep Agents SDK manages context for long-running AI tasks through offloading, summarization, and filesystem abstraction to prevent context rot.
langchain.com··Mads Kristian Nylund·Open Source AIAI GovernanceAI Ethics
LangChain and Fireworks fine-tuned an open model to mine perceived error signals from production traces, matching frontier model performance at a fraction of the cost.
langchain.com··Mads Kristian Nylund·Open Source AIAI GovernanceAI Ethics
Build targeted evals for Deep Agents that measure real behaviors. Discover data curation, metrics, and testing strategies to improve agent accuracy.
langchain.com··Mads Kristian Nylund·Open Source AIAI ObservabilityAI Evaluation
Learn how LangChain uses LangSmith LLM Gateway to track coding agent spend in real time, set budgets by team and user, and prevent runaway AI costs.
langchain.com··Mads Kristian Nylund·Open Source AIAI InfrastructureAI Governance
The EU AI Act compliance deadline is August 2, 2026. Learn what the EU AI Act requires, and how LangSmith and LangChain OSS products help you meet each requirement.
langchain.com··Mads Kristian Nylund·Open Source AIAI GovernanceAI Ethics.png)
Fleet supports both quick, ad hoc tasks and recurring responsibilities. See how General Purpose Chat and Specialized Agents help teams delegate work.
langchain.com··Mads Kristian Nylund·AI ArchitectureAI InfrastructureAI-agents
A Harvey and LangChain Labs study on making LLM verifiers cheaper and more reliable for legal-agent evaluation and post-training.
langchain.com··Mads Kristian Nylund·Open Source AIAI GovernanceAI Ethics.png)
OpenWiki Brains turns sources like Gmail, Notion, Git, X, Hacker News, and web search into a local wiki that agents can use as fresh, proactive memory.
langchain.com··Mads Kristian Nylund·Open Source AIAI ToolsAI Infrastructure
Learn how leading engineering teams ship AI agents reliably and repeatedly using a four-phase agent development lifecycle: Build, Test, Deploy, and Monitor. Includes guidance on evals, runtimes, observability, and governance at scale.
langchain.com··Mads Kristian Nylund·Agent DevelopmentAI LifecycleAI Governance
AI agents work best when they reflect the knowledge and judgment your team has built over time. Some of that is institutional knowledge that’s already documented and easy for an agent to use as-is. But most great organizations also rely on tacit knowledge that lives inside their employees’ minds.
langchain.com··Mads Kristian Nylund·AI GovernanceAI-agentsAI Evaluation
Multi-agent systems that mirror real engineering teams — not just code faster — can cut debug time by 93% and compress cross-team delivery. Here's the architecture built on LangGraph.
langchain.com··Mads Kristian Nylund·AI ArchitectureSoftware Deliveryagentic-ai.png)
Klarna's AI assistant is revolutionizing the personal shopping experience, including customer service and productivity. See how they used LangGraph and LangSmith to achieve 80% faster customer resolution times.
langchain.com··Mads Kristian Nylund·Open Source AIAI ToolsAI Governance
Learn how agent observability enables effective evaluation of AI agents. Understand tracing, debugging reasoning, and performance insights to iterate and improve agent behavior.
langchain.com··Mads Kristian Nylund·Agent ObservabilityAgent EvaluationAI Evaluation
Agents automate real-world work, but reliable performance requires more than a good model, it requires a carefully designed harness built for specific tasks. This post explores the core agent loop, how stacking and extending loops builds more effective agents, and how to instrument each level with LangChain primitives.
langchain.com··Mads Kristian Nylund·Open Source AIAI ToolsAI Architecture
Rippling uses LangChain Deep Agents and LangSmith to run cross-domain AI across HR, IT, finance, payroll, and global operations.
langchain.com··Mads Kristian Nylund·Open Source AIAI ToolsAI Architecture
Running code execution in an AI agent is harder than it looks. Your agent needs a real computer (filesystem, shell, package manager, persistent state) but handing it access to your infrastructure is dangerous.Think about it this way: you use one laptop. You are n of one. But agents are going to run millions of tasks, and each one needs its own computer to work from. That's the infrastructure shift happening right now. Satya Nadella put it plainly: "Every agent needs a computer." The question is what that computer looks like, and how you give it to them safely.LangSmith Sandboxes are our answer to that. Here's why it matters, and why doing it yourself is harder than it sounds.
langchain.com··Mads Kristian Nylund·AI ArchitectureAI InfrastructureAI Platforms
A technical deep dive into how LangChain built full-text search in SmithDB, from constructing and compacting inverted indexes to routing queries across local SSD and object storage.
langchain.com··Mads Kristian Nylund·Open Source AIAI Infrastructure
Deep Agents' RubricMiddleware adds a self-evaluation loop to your agent runs. Set a rubric, configure a grader, and get reliable outputs on tasks where correctness matters.
langchain.com··Mads Kristian Nylund·AI InfrastructureAI-agentsAI Evaluation
Best practices for evaluating AI agent skills. Define tasks, measure performance, and iterate faster with LangSmith observability and evals.
langchain.com··Mads Kristian Nylund·AI ToolsAI ArchitectureAI Evaluation
Production monitoring for LLM agents requires new observability tools. Learn how to trace, evaluate, and improve AI agents at scale.
langchain.com··Mads Kristian Nylund·Open Source AIAI ToolsAI Observability
A practical checklist for agent evaluation: error analysis, dataset construction, grader design, offline & online evals, and production readiness.
langchain.com··Mads Kristian Nylund·AI EvaluationAgent Technologies
Learn how Deep Agents uses prompt caching to cut LLM token costs by up to 80% across every major model provider - no extra config required.
langchain.com··Mads Kristian Nylund·Open Source AIAI ToolsAI Infrastructure
Lyft used LangGraph and LangSmith to build a self-serve AI agent platform for customer support, cutting agent development from months to weeks.
langchain.com··Mads Kristian Nylund·AI DevelopmentMulti-Agent SystemsAI Platforms
A smaller, in-process alternative to full sandboxes: WASM + QuickJS for isolation, least-privilege capabilities, and snapshot-based durable pauses.
langchain.com··Mads Kristian Nylund·Open Source AIAI ToolsAgent Technologies
Evaluating long-running, stateful agents needs a new kind of runner. Here's how Deep Agents, LangSmith sandboxes, and observability plug into Harbor.
langchain.com··Mads Kristian Nylund·Open Source AIAI ToolsAI Architecture
We tuned an Nemotron 3 Ultra's harness to match Opus 4.8's best agent run at ~8x lower cost, changing only the scaffolding around it.
langchain.com··Mads Kristian Nylund·Open Source AIAI ToolsAI Architecture
Learn how Schneider Electric built enterprise LLMOps foundations with LangSmith to improve observability, evaluation, and deployment for AI products at scale.
langchain.com··Mads Kristian Nylund·LLMOpsAI Platforminfrastructure
How LangChain mines agent traces to find failures, fine-tune judge models cheaper than frontier LLMs, and hill-climb performance with evals.
langchain.com··Mads Kristian Nylund·Open Source AIAI Toolsevaluation
OpenWiki generates and maintains codebase documentation so coding agents can find the repo context they need without loading everything into one instruction file.
langchain.com··Mads Kristian Nylund·developer-toolsOpen Source AIDocumentation Tools
Recursive language models (RLMs) fix context rot by having agents write code that dispatches subagents over context chunks instead of pumping everything in one context window. Deep Agents now implements this through dynamic subagents and a lightweight code interpreter, letting agents programmatically fan out work like grep, map, and reduce over large inputs. We benchmark the approach on OOLONG, a long-context reasoning task, and show it holds up where turn-by-turn agents start to break down.
langchain.com··Mads Kristian Nylund·architectureOpen Source AIAI Tools
Learn why coding agent bills spiral out of control — and how to trace, compare, and govern spend across Claude Code, Cursor, Copilot, and more in one place.
langchain.com··Mads Kristian Nylund·ObservabilityCost ManagementTool Optimization
Dynamic subagents let AI agents orchestrate work at scale using code instead of tool calls. Learn how programmatic orchestration in Deep Agents guarantees coverage, handles fan-out, and unlocks reliable multi-step, complex agent pipelines with common orchestration patterns and live traces.
langchain.com··Mads Kristian Nylund·Open Source AIAI ToolsAI Architecture
Learn how agent engineering combines product thinking, engineering, and data science to build reliable LLM systems. Ship faster with iterative testing.
langchain.com··Mads Kristian Nylund·AI DevelopmentAI InfrastructureAgent Engineering
See how Monte Carlo built its AI Troubleshooting Agent on LangGraph and debugged with LangSmith to help data teams resolve issues faster
langchain.com··Mads Kristian Nylund·AI ToolsAI InfrastructureAI Observability
Open SWE is an open-source, cloud-hosted coding agent that autonomously handles GitHub tasks—planning, coding, testing, and opening PRs.
langchain.com··Mads Kristian Nylund·Open Source AIAI ToolsAI Architecture
Align Evals is a new feature in LangSmith that helps you calibrate your evaluators to better match human preferences.
langchain.com··Mads Kristian Nylund·AI DevelopmentAI ToolsAI Evaluation
Evaluate LLMs with OpenEvals and AgentEvals—pre-built evaluators for LLM-as-judge, structured data, and agent trajectories. Production-ready frameworks.
langchain.com··Mads Kristian Nylund·AI ToolsAI EvaluationAI Platforms
We explore how increasing the number of instructions and tools available to a single ReAct agent affects its performance, benchmarking models like claude-3.5-sonnet, gpt-4o, o1, and o3-mini across two domains of tasks.
langchain.com··Mads Kristian Nylund·AI ToolsAI ArchitectureAI Evaluation
LangSmith's homepage is now organized into Observability, Evaluation, and Prompt Engineering. Learn why we organized the homepage like this. Plus, see our latest Resource Tags updates.
langchain.com··Mads Kristian Nylund·Open Source AIAI ToolsAI Governance
Deep dive into self-improving evaluators in LangSmith, motivated by the rise of LLM-as-a-Judge evaluators plus research on few-shot learning and aligning human preferences.
langchain.com··Mads Kristian Nylund·AI ToolsAI InfrastructureAI Evaluation
How Factory AI uses LangSmith to debug issues and close the product feedback loop, resulting in a 2x improvement in iteration speed.
langchain.com··Mads Kristian Nylund·Open Source AIAI ToolsAI Infrastructure
Learn what pairwise evaluation is, why you might need it for LLM app development, and see an example of how to use it in LangSmith by LangChain.
langchain.com··Mads Kristian Nylund·AI DevelopmentAI ToolsPairwise Evaluation
Evaluate and iterate on LLM applications with confidence using LangSmith's regression testing. Compare experiments, track performance, and identify changes.
langchain.com··Mads Kristian Nylund·AI TestingAI Regression TestingAI Infrastructure
LangSmith is now available in Azure Marketplace. Deploy the DevOps platform for LLM apps in your Azure VPC with full data control and MACC credit support.
langchain.com··Mads Kristian Nylund·Open Source AIAI InfrastructureAI Platforms
langchain.com··Mads Kristian Nylund·Open Source AIAI ToolsAI Infrastructure
langchain.com··Mads Kristian Nylund·AI ToolsAI ArchitectureAI Evaluation
Dosu uses evaluation driven development and LangSmith to build reliable LLM products at scale, monitor production performance, and iterate with confidence.
langchain.com··Mads Kristian Nylund·AI ToolsAI GovernanceAI Evaluation
Build production-ready RAG apps with Pinecone Serverless, LangChain, and LangServe. Deploy scalable AI with usage-based pricing and observability.
langchain.com··Mads Kristian Nylund·Open Source AIAI ToolsAI Infrastructure
Benchmark LLM tool use with 4 test environments. Compare GPT-4, Claude, and open-source models on function calling, planning, and reasoning tasks.
langchain.com··Mads Kristian Nylund·AI ToolsAI BenchmarkingAI Evaluation
Share and compare LLM evaluation results with public LangSmith benchmarks. Test RAG systems, agents, and architectures on community datasets.
langchain.com··Mads Kristian Nylund·AI ArchitectureAI BenchmarksAI Evaluation
Compare LLM test runs side-by-side with LangSmith's Test Run Comparisons. Manually inspect data, filter results, and gain insights faster.
langchain.com··Mads Kristian Nylund·AI ToolsAI GovernanceAI Ethics
Evaluate and compare fine-tuned open source LLMs using LangSmith. Test multiple models, automate evaluations, and choose the best performing AI.
langchain.com··Mads Kristian Nylund·Open Source AIAI ToolsAI Evaluation
Compare GPT-4, Claude, and open-source LLMs on structured data extraction from chat logs. Benchmark results, evaluation metrics, and dataset creation insights.
langchain.com··Mads Kristian Nylund·AI ToolsAI ArchitectureAI Evaluation
Managed Deep Agents gives developers a managed way to build, run, and deploy Deep Agents with built-in runtime, streaming, sandboxes, evals, memory, and auth.
langchain.com··Mads Kristian NylundLangSmith Bring Your Own Cloud is now generally available on AWS, giving Enterprise teams managed observability, evaluation, and deployment inside their own VPC.
langchain.com··Mads Kristian NylundLearn what AI agents are, how they work in an LLM loop, and where workflows fit so you can build reliable, production-ready autonomous systems.
langchain.com··Mads Kristian NylundWe benchmarked NVIDIA NeMo Switchyard on 145 agent tasks. Only 7% of turns needed a frontier model, and routing cut cost 74% for six points of accuracy.
langchain.com··Mads Kristian NylundRun deep agents in production with durable execution, sandboxes, tool access, and LangSmith observability, without building the runtime yourself. Now in private beta
langchain.com··Mads Kristian NylundDeep Agents, LangChain, and LangGraph each offer distinct approaches to building agents. In this post, we cover the key distinctions between our open source frameworks and when you should reach for each one.
langchain.com··Mads Kristian NylundLearn how LangChain built an autonomous SRE agent for Kubernetes deployments with Deep Agents, human approval for changes, LangSmith tracing, and evals.
langchain.com··Mads Kristian NylundLearn how to evaluate voice agents across execution, outcomes, and caller experience using LangSmith traces, code evaluators, LLM judges, and human review.
langchain.com··Mads Kristian NylundCustomer Experience (CX) Agents in Production: Lessons from Lyft, Vodafone, and LATAM Airlines
langchain.com··Mads Kristian NylundLearn how Stripe built Kai, a company-wide AI agent on LangChain, LangGraph, and Deep Agents, reaching 5,000 users in roughly 4 weeks.
langchain.com··Mads Kristian NylundLearn how LangChain built ReviewBench, a benchmark for evaluating code review agents against real PR feedback from trusted reviewers.
langchain.com··Mads Kristian NylundIntroducing LangSmith LLM Gateway: runtime governance for AI agents with spend limits, PII redaction, and trace continuity, built directly into LangSmith.
langchain.com··Mads Kristian NylundToday we're shipping deep agents v0.7. This release simplifies the base harness, resulting in 65% fewer base input tokens at comparable performance.
langchain.com··Mads Kristian NylundLearn how Similarweb uses LangSmith to evaluate long-form agent research reports with rubrics, faithfulness checks, traces, and baseline comparisons.
langchain.com··Mads Kristian NylundLearn how LangChain used Hex, dbt, semantic models, and observability to build a trusted data agent and scale self-service analysis by 40x.
langchain.com··Mads Kristian NylundSmithDB supports full-text search and JSON filtering over agent traces with a median (P50) latency of 400 ms, even though the underlying data consists of large, deeply nested JSON documents stored in object storage.
langchain.com··Mads Kristian NylundLearn why companies must own their agent systems, governance, context, and feedback loops to turn generic AI into lasting business advantage.
langchain.com··Mads Kristian NylundWe revamped how we benchmark Deep Agents. Here's the eval setup we run in Harbor across coding, conversation, and retrieval, and how we use it to ship changes.
langchain.com··Mads Kristian NylundLangChain's Eval Engineering Skill inspects your agent's repo and traces, proposes evals through user interviews, and outputs runnable Harbor tasks.
langchain.com··Mads Kristian NylundGraph engineering isn't a new idea. It's the latest name for a well established approach to building reliable agents. It's the same idea behind loop engineering and harness engineering: building putting model reasoning in the right places, with the right context, at each step. At LangChain, we've been helping people build agents with graphs for 3 years! Here's what we've learned.
langchain.com··Mads Kristian NylundLangSmith now supports tracing for voice agents built with Pipecat, LiveKit, OpenAI Realtime, and Gemini Live. Capture audio, STT and TTS latency, interruptions, tool calls, and more in one trace.
langchain.com··Mads Kristian NylundApollo uses Deep Agents and LangSmith to power an AI Assistant that handles prospecting, enrichment, outreach, analytics, and MCP integrations.
langchain.com··Mads Kristian NylundThe gateway is the runtime control plane for enterprise AI, turning policy into enforceable decisions across every model call, tool call, and agent hop.
langchain.com··Mads Kristian NylundLearn how LangChain built IssueBench, a synthetic benchmark for evaluating how well LangSmith Engine identifies, categorizes, and groups issues in agent traces.
langchain.com··Mads Kristian NylundOpenWiki 0.2 generates codebase wikis in the OKF format, helping developers organize repo docs with metadata, changelogs, and agent-friendly retrieval.
langchain.com··Mads Kristian NylundFor most of computing history, a developer environment meant a physical machine, then a VM, then a container, with each one shared across the work happening on it, each one requiring deliberate setup and teardown. The idea of giving every agent its own isolated computer, booting in under a second and cleaning up when done wasn't practical at scale. Sandboxes give you a safer way to run upgraded workflows, providing you an isolated environment to run tasks that require iteration, verification, and access to the tools a person would normally use, all without requiring human supervision. Every human developer gets a laptop. Every agent can get a computer.
langchain.com··Mads Kristian NylundUse LangSmith to trace coding agents across Claude Code, Codex, Cursor, Copilot, and more. Inspect tool calls, subagents, errors, costs, and retries.
langchain.com··Mads Kristian NylundLearn how Deep Agents SDK manages context for long-running AI tasks through offloading, summarization, and filesystem abstraction to prevent context rot.
langchain.com··Mads Kristian NylundLangChain and Fireworks fine-tuned an open model to mine perceived error signals from production traces, matching frontier model performance at a fraction of the cost.
langchain.com··Mads Kristian NylundBuild targeted evals for Deep Agents that measure real behaviors. Discover data curation, metrics, and testing strategies to improve agent accuracy.
langchain.com··Mads Kristian NylundLearn how LangChain uses LangSmith LLM Gateway to track coding agent spend in real time, set budgets by team and user, and prevent runaway AI costs.
langchain.com··Mads Kristian NylundThe EU AI Act compliance deadline is August 2, 2026. Learn what the EU AI Act requires, and how LangSmith and LangChain OSS products help you meet each requirement.
langchain.com··Mads Kristian NylundFleet supports both quick, ad hoc tasks and recurring responsibilities. See how General Purpose Chat and Specialized Agents help teams delegate work.
langchain.com··Mads Kristian NylundA Harvey and LangChain Labs study on making LLM verifiers cheaper and more reliable for legal-agent evaluation and post-training.
langchain.com··Mads Kristian NylundOpenWiki Brains turns sources like Gmail, Notion, Git, X, Hacker News, and web search into a local wiki that agents can use as fresh, proactive memory.
langchain.com··Mads Kristian NylundLearn how leading engineering teams ship AI agents reliably and repeatedly using a four-phase agent development lifecycle: Build, Test, Deploy, and Monitor. Includes guidance on evals, runtimes, observability, and governance at scale.
langchain.com··Mads Kristian NylundAI agents work best when they reflect the knowledge and judgment your team has built over time. Some of that is institutional knowledge that’s already documented and easy for an agent to use as-is. But most great organizations also rely on tacit knowledge that lives inside their employees’ minds.
langchain.com··Mads Kristian NylundMulti-agent systems that mirror real engineering teams — not just code faster — can cut debug time by 93% and compress cross-team delivery. Here's the architecture built on LangGraph.
langchain.com··Mads Kristian NylundKlarna's AI assistant is revolutionizing the personal shopping experience, including customer service and productivity. See how they used LangGraph and LangSmith to achieve 80% faster customer resolution times.
langchain.com··Mads Kristian NylundLearn how agent observability enables effective evaluation of AI agents. Understand tracing, debugging reasoning, and performance insights to iterate and improve agent behavior.
langchain.com··Mads Kristian NylundAgents automate real-world work, but reliable performance requires more than a good model, it requires a carefully designed harness built for specific tasks. This post explores the core agent loop, how stacking and extending loops builds more effective agents, and how to instrument each level with LangChain primitives.
langchain.com··Mads Kristian NylundRippling uses LangChain Deep Agents and LangSmith to run cross-domain AI across HR, IT, finance, payroll, and global operations.
langchain.com··Mads Kristian NylundRunning code execution in an AI agent is harder than it looks. Your agent needs a real computer (filesystem, shell, package manager, persistent state) but handing it access to your infrastructure is dangerous.Think about it this way: you use one laptop. You are n of one. But agents are going to run millions of tasks, and each one needs its own computer to work from. That's the infrastructure shift happening right now. Satya Nadella put it plainly: "Every agent needs a computer." The question is what that computer looks like, and how you give it to them safely.LangSmith Sandboxes are our answer to that. Here's why it matters, and why doing it yourself is harder than it sounds.
langchain.com··Mads Kristian NylundA technical deep dive into how LangChain built full-text search in SmithDB, from constructing and compacting inverted indexes to routing queries across local SSD and object storage.
langchain.com··Mads Kristian NylundDeep Agents' RubricMiddleware adds a self-evaluation loop to your agent runs. Set a rubric, configure a grader, and get reliable outputs on tasks where correctness matters.
langchain.com··Mads Kristian NylundBest practices for evaluating AI agent skills. Define tasks, measure performance, and iterate faster with LangSmith observability and evals.
langchain.com··Mads Kristian NylundProduction monitoring for LLM agents requires new observability tools. Learn how to trace, evaluate, and improve AI agents at scale.
langchain.com··Mads Kristian NylundA practical checklist for agent evaluation: error analysis, dataset construction, grader design, offline & online evals, and production readiness.
langchain.com··Mads Kristian NylundLearn how Deep Agents uses prompt caching to cut LLM token costs by up to 80% across every major model provider - no extra config required.
langchain.com··Mads Kristian NylundLyft used LangGraph and LangSmith to build a self-serve AI agent platform for customer support, cutting agent development from months to weeks.
langchain.com··Mads Kristian NylundA smaller, in-process alternative to full sandboxes: WASM + QuickJS for isolation, least-privilege capabilities, and snapshot-based durable pauses.
langchain.com··Mads Kristian NylundEvaluating long-running, stateful agents needs a new kind of runner. Here's how Deep Agents, LangSmith sandboxes, and observability plug into Harbor.
langchain.com··Mads Kristian NylundWe tuned an Nemotron 3 Ultra's harness to match Opus 4.8's best agent run at ~8x lower cost, changing only the scaffolding around it.
langchain.com··Mads Kristian NylundLearn how Schneider Electric built enterprise LLMOps foundations with LangSmith to improve observability, evaluation, and deployment for AI products at scale.
langchain.com··Mads Kristian NylundHow LangChain mines agent traces to find failures, fine-tune judge models cheaper than frontier LLMs, and hill-climb performance with evals.
langchain.com··Mads Kristian NylundOpenWiki generates and maintains codebase documentation so coding agents can find the repo context they need without loading everything into one instruction file.
langchain.com··Mads Kristian NylundRecursive language models (RLMs) fix context rot by having agents write code that dispatches subagents over context chunks instead of pumping everything in one context window. Deep Agents now implements this through dynamic subagents and a lightweight code interpreter, letting agents programmatically fan out work like grep, map, and reduce over large inputs. We benchmark the approach on OOLONG, a long-context reasoning task, and show it holds up where turn-by-turn agents start to break down.
langchain.com··Mads Kristian NylundLearn why coding agent bills spiral out of control — and how to trace, compare, and govern spend across Claude Code, Cursor, Copilot, and more in one place.
langchain.com··Mads Kristian NylundDynamic subagents let AI agents orchestrate work at scale using code instead of tool calls. Learn how programmatic orchestration in Deep Agents guarantees coverage, handles fan-out, and unlocks reliable multi-step, complex agent pipelines with common orchestration patterns and live traces.
langchain.com··Mads Kristian NylundLearn how agent engineering combines product thinking, engineering, and data science to build reliable LLM systems. Ship faster with iterative testing.
langchain.com··Mads Kristian NylundSee how Monte Carlo built its AI Troubleshooting Agent on LangGraph and debugged with LangSmith to help data teams resolve issues faster
langchain.com··Mads Kristian NylundOpen SWE is an open-source, cloud-hosted coding agent that autonomously handles GitHub tasks—planning, coding, testing, and opening PRs.
langchain.com··Mads Kristian NylundAlign Evals is a new feature in LangSmith that helps you calibrate your evaluators to better match human preferences.
langchain.com··Mads Kristian NylundEvaluate LLMs with OpenEvals and AgentEvals—pre-built evaluators for LLM-as-judge, structured data, and agent trajectories. Production-ready frameworks.
langchain.com··Mads Kristian NylundWe explore how increasing the number of instructions and tools available to a single ReAct agent affects its performance, benchmarking models like claude-3.5-sonnet, gpt-4o, o1, and o3-mini across two domains of tasks.
langchain.com··Mads Kristian NylundLangSmith's homepage is now organized into Observability, Evaluation, and Prompt Engineering. Learn why we organized the homepage like this. Plus, see our latest Resource Tags updates.
langchain.com··Mads Kristian NylundDeep dive into self-improving evaluators in LangSmith, motivated by the rise of LLM-as-a-Judge evaluators plus research on few-shot learning and aligning human preferences.
langchain.com··Mads Kristian NylundHow Factory AI uses LangSmith to debug issues and close the product feedback loop, resulting in a 2x improvement in iteration speed.
langchain.com··Mads Kristian NylundLearn what pairwise evaluation is, why you might need it for LLM app development, and see an example of how to use it in LangSmith by LangChain.
langchain.com··Mads Kristian NylundEvaluate and iterate on LLM applications with confidence using LangSmith's regression testing. Compare experiments, track performance, and identify changes.
langchain.com··Mads Kristian NylundLangSmith is now available in Azure Marketplace. Deploy the DevOps platform for LLM apps in your Azure VPC with full data control and MACC credit support.
langchain.com··Mads Kristian Nylundlangchain.com··Mads Kristian Nylundlangchain.com··Mads Kristian NylundDosu uses evaluation driven development and LangSmith to build reliable LLM products at scale, monitor production performance, and iterate with confidence.
langchain.com··Mads Kristian NylundBuild production-ready RAG apps with Pinecone Serverless, LangChain, and LangServe. Deploy scalable AI with usage-based pricing and observability.
langchain.com··Mads Kristian NylundBenchmark LLM tool use with 4 test environments. Compare GPT-4, Claude, and open-source models on function calling, planning, and reasoning tasks.
langchain.com··Mads Kristian NylundShare and compare LLM evaluation results with public LangSmith benchmarks. Test RAG systems, agents, and architectures on community datasets.
langchain.com··Mads Kristian NylundCompare LLM test runs side-by-side with LangSmith's Test Run Comparisons. Manually inspect data, filter results, and gain insights faster.
langchain.com··Mads Kristian NylundEvaluate and compare fine-tuned open source LLMs using LangSmith. Test multiple models, automate evaluations, and choose the best performing AI.
langchain.com··Mads Kristian NylundCompare GPT-4, Claude, and open-source LLMs on structured data extraction from chat logs. Benchmark results, evaluation metrics, and dataset creation insights.
langchain.com··Mads Kristian Nylund