Skip to content

Conversation

@ixchio
Copy link

@ixchio ixchio commented Dec 6, 2025

Add built-in health-aware middleware for LangChain v1 agents that monitors tool/model health and provides automatic retry, fallback, and telemetry capabilities.

Fixes #34193

Features

  • HealthStats: Rolling window statistics tracking per-tool/model (error rate, P95 latency)
  • HealthPolicy: Configurable degradation thresholds
  • HealthMiddleware: Auto retry with exponential backoff, model fallback, tool disabling
  • Telemetry: Pluggable emitter with create_langsmith_emitter() helper
  • Persistence: Optional store integration for cross-run stats via store param
  • Provider tool preservation: Keep provider dict tools active even when degraded

Usage

from langchain.agents.middleware import HealthMiddleware, HealthPolicy

health = HealthMiddleware(
    policy=HealthPolicy(max_error_rate=0.2, consecutive_failures=3),
    fallback_model="openai:gpt-4o-mini",
    disable_tools=True,
)

agent = create_agent(model="openai:gpt-4o", tools=[...], middleware=[health])

Testing
35 unit tests covering HealthStats, HealthPolicy, HealthMiddleware
All 534 middleware tests pass

@github-actions github-actions bot added langchain `langchain` package issues & PRs feature For PRs that implement a new feature; NOT A FEATURE REQUEST labels Dec 6, 2025
@sydney-runkle sydney-runkle self-assigned this Dec 12, 2025
@sydney-runkle
Copy link
Collaborator

Thanks for your contribution!

At this time, we're only adding middlewares that we've seen significant demand for and have the capacity to maintain.

Cool implementation though, we're definitely seeing lots of middlewares that add resilience to agentic workflows!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature For PRs that implement a new feature; NOT A FEATURE REQUEST langchain `langchain` package issues & PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add built-in health-aware middleware for tools/models with auto retry/fallback

2 participants