-
Notifications
You must be signed in to change notification settings - Fork 9
Description
Is your feature request related to a problem? Please describe.
Currently, /llm/call only supports single LLM calls. For multi-step AI workflows (e.g., STT → LLM → TTS), developers must manually chain multiple API calls, handle intermediate results, and manage errors across requests. This increases latency and complexity.
Describe the solution you'd like
Extend the existing LLM service layer to support chained execution through a new /llm/chain endpoint. This reuses the core LLM execution components (models, config resolution, provider registry, observability, job tracking) and orchestrates multiple sequential blocks at the service layer, where each block's output becomes the next block's input. This maintains architectural consistency by sharing the same infrastructure (ConfigBlob, QueryParams, provider execution, langfuse integration) while adding orchestration logic for multi-step workflows.
Describe alternatives you've considered
Check for the langchain, haystack or other agent frameworks how they handle the multi-step llm-execution
Additional context
Use a well-defined API contract for llm/chain that is flexible and closely aligned with the existing llm/call API contract. The new contract should support all functionalities currently available in llm/call, while additionally enabling prompt interpolation. This will allow users to modify, extend, or wrap additional context around previous outputs when invoking chained LLM calls.
Sub-issues
Metadata
Metadata
Assignees
Labels
Type
Projects
Status