update livekit inference model to match latest #993
Conversation
🦋 Changeset detectedLatest commit: e416a5e The changes in this PR will be included in the next version bump. This PR includes changesets to release 18 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
📝 WalkthroughWalkthroughUpdates inference model type definitions for LiveKit Agents across LLM, STT, and TTS: adds new provider/model variants (GPT‑5.1/5.2, Moonshot, DeepSeek, Deepgram Aura, AssemblyAI, ElevenLabs STT, Inworld), removes deprecated models, and adjusts union/option typings. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
📜 Recent review detailsConfiguration used: Organization UI Review profile: CHILL Plan: Pro 📒 Files selected for processing (3)
🧰 Additional context used📓 Path-based instructions (3)**/*.{ts,tsx,js,jsx}📄 CodeRabbit inference engine (.cursor/rules/agent-core.mdc)
Files:
**/*.{ts,tsx}?(test|example|spec)📄 CodeRabbit inference engine (.cursor/rules/agent-core.mdc)
Files:
**/*.{ts,tsx}?(test|example)📄 CodeRabbit inference engine (.cursor/rules/agent-core.mdc)
Files:
🧠 Learnings (3)📓 Common learnings📚 Learning: 2026-01-16T14:33:39.551ZApplied to files:
📚 Learning: 2026-01-16T14:33:39.551ZApplied to files:
🧬 Code graph analysis (2)agents/src/inference/stt.ts (1)
agents/src/inference/tts.ts (2)
🪛 Biome (2.1.2)agents/src/inference/tts.ts[error] 58-58: An empty interface is equivalent to {}. Safe fix: Use a type alias instead. (lint/suspicious/noEmptyInterface) [error] 60-60: An empty interface is equivalent to {}. Safe fix: Use a type alias instead. (lint/suspicious/noEmptyInterface) [error] 62-62: An empty interface is equivalent to {}. Safe fix: Use a type alias instead. (lint/suspicious/noEmptyInterface) 🔇 Additional comments (10)
✏️ Tip: You can disable this entire section by setting Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Fix all issues with AI agents
In `@agents/src/inference/llm.ts`:
- Around line 85-90: The LLMModels union type is split across multiple lines
causing Prettier to fail; update the declaration of LLMModels so the entire
union is on a single line (e.g., collapse the multi-line union of OpenAIModels,
GoogleModels, MoonshotModels, DeepSeekModels, and AnyString into one line) to
conform with the repo's Prettier rules and re-run formatting/linting for the
file containing the LLMModels type.
In `@agents/src/inference/stt.ts`:
- Around line 36-40: Prettier flags the AssemblyaiModels union formatting;
update the type declarations so each union member is on its own line —
specifically reformat the AssemblyaiModels type (and keep consistent style with
CartesiaModels and ElevenlabsSTTModels) so the two members
'assemblyai/universal-streaming' and
'assemblyai/universal-streaming-multilingual' are split across separate lines to
satisfy Prettier.
In `@agents/src/inference/tts.ts`:
- Around line 58-62: Replace the three empty interface declarations
DeepgramTTSOptions, RimeOptions, and InworldOptions with equivalent type aliases
(e.g. "type DeepgramTTSOptions = {}") so they are not flagged as empty
interfaces by CI; update the declarations for DeepgramTTSOptions, RimeOptions,
and InworldOptions in the tts module accordingly, keeping names unchanged and
preserving exported visibility.
- Around line 64-72: Prettier flagged formatting on the _TTSModels / TTSModels
union; reformat these type declarations to match the project's Prettier rules
(consistent line breaks and pipe placement) — specifically update the _TTSModels
and export type TTSModels declarations (symbols: _TTSModels and TTSModels) so
the union members are arranged and indented according to the repo style, then
run Prettier (or your editor's format) to ensure CI passes.
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (4)
.changeset/kind-coins-add.mdagents/src/inference/llm.tsagents/src/inference/stt.tsagents/src/inference/tts.ts
🧰 Additional context used
📓 Path-based instructions (3)
**/*.{ts,tsx,js,jsx}
📄 CodeRabbit inference engine (.cursor/rules/agent-core.mdc)
Add SPDX-FileCopyrightText and SPDX-License-Identifier headers to all newly added files with '// SPDX-FileCopyrightText: 2025 LiveKit, Inc.' and '// SPDX-License-Identifier: Apache-2.0'
Files:
agents/src/inference/llm.tsagents/src/inference/stt.tsagents/src/inference/tts.ts
**/*.{ts,tsx}?(test|example|spec)
📄 CodeRabbit inference engine (.cursor/rules/agent-core.mdc)
When testing inference LLM, always use full model names from
agents/src/inference/models.ts(e.g., 'openai/gpt-4o-mini' instead of 'gpt-4o-mini')
Files:
agents/src/inference/llm.tsagents/src/inference/stt.tsagents/src/inference/tts.ts
**/*.{ts,tsx}?(test|example)
📄 CodeRabbit inference engine (.cursor/rules/agent-core.mdc)
Initialize logger before using any LLM functionality with
initializeLogger({ pretty: true })from '@livekit/agents'
Files:
agents/src/inference/llm.tsagents/src/inference/stt.tsagents/src/inference/tts.ts
🧠 Learnings (2)
📓 Common learnings
Learnt from: CR
Repo: livekit/agents-js PR: 0
File: .cursor/rules/agent-core.mdc:0-0
Timestamp: 2026-01-16T14:33:39.551Z
Learning: Applies to **/*.{ts,tsx}?(test|example|spec) : When testing inference LLM, always use full model names from `agents/src/inference/models.ts` (e.g., 'openai/gpt-4o-mini' instead of 'gpt-4o-mini')
📚 Learning: 2026-01-16T14:33:39.551Z
Learnt from: CR
Repo: livekit/agents-js PR: 0
File: .cursor/rules/agent-core.mdc:0-0
Timestamp: 2026-01-16T14:33:39.551Z
Learning: Applies to **/*.{ts,tsx}?(test|example|spec) : When testing inference LLM, always use full model names from `agents/src/inference/models.ts` (e.g., 'openai/gpt-4o-mini' instead of 'gpt-4o-mini')
Applied to files:
agents/src/inference/llm.tsagents/src/inference/stt.tsagents/src/inference/tts.ts
🧬 Code graph analysis (2)
agents/src/inference/stt.ts (1)
agents/src/inference/tts.ts (1)
CartesiaModels(25-29)
agents/src/inference/tts.ts (3)
agents/src/inference/utils.ts (1)
AnyString(8-8)plugins/deepgram/src/tts.ts (1)
TTSOptions(21-29)plugins/cartesia/src/tts.ts (1)
TTSOptions(30-45)
🪛 Biome (2.1.2)
agents/src/inference/tts.ts
[error] 58-58: An empty interface is equivalent to {}.
Safe fix: Use a type alias instead.
(lint/suspicious/noEmptyInterface)
[error] 60-60: An empty interface is equivalent to {}.
Safe fix: Use a type alias instead.
(lint/suspicious/noEmptyInterface)
[error] 62-62: An empty interface is equivalent to {}.
Safe fix: Use a type alias instead.
(lint/suspicious/noEmptyInterface)
🪛 GitHub Actions: Build
agents/src/inference/llm.ts
[error] 85-85: Prettier: Replace ⏎··|·OpenAIModels⏎··|·GoogleModels⏎··|·MoonshotModels⏎··|·DeepSeekModels with OpenAIModels|GoogleModels|MoonshotModels|DeepSeekModels [Error/prettier/prettier]
agents/src/inference/stt.ts
[error] 38-38: Prettier: Replace ·'assemblyai/universal-streaming' with ⏎··|·'assemblyai/universal-streaming'⏎· [Error/prettier/prettier]
agents/src/inference/tts.ts
[warning] 58-58: An empty interface is equivalent to {}. [Warning/@typescript-eslint/no-empty-interface]
[warning] 60-60: An empty interface is equivalent to {}. [Warning/@typescript-eslint/no-empty-interface]
[warning] 62-62: An empty interface is equivalent to {}. [Warning/@typescript-eslint/no-empty-interface]
[error] 64-64: Prettier: Replace CartesiaModels|DeepgramTTSModels|ElevenlabsModels|RimeModels with CartesiaModels|DeepgramTTSModels|ElevenlabsModels|RimeModels [Error/prettier/prettier]
🔇 Additional comments (9)
agents/src/inference/llm.ts (3)
19-32: LGTM!The new OpenAI GPT-5.x model variants follow the established naming convention and integrate correctly into the existing type union.
34-41: LGTM!The Google models have been updated from preview to stable variants (
gemini-3-pro,gemini-3-flash), which aligns with the PR objective of matching the latest inference models.
43-45: LGTM!New MoonshotModels and expanded DeepSeekModels types are well-defined and follow the
provider/modelnaming convention.agents/src/inference/stt.ts (2)
27-34: LGTM!DeepgramModels appropriately expanded with
flux-generalandnova-2-conversationalaivariants, following the established naming convention.
80-84: LGTM!The
_STTModelsunion correctly includes all the new model types (ElevenlabsSTTModels, AssemblyaiModels) alongside existing ones.agents/src/inference/tts.ts (3)
25-31: LGTM!CartesiaModels and DeepgramTTSModels are well-defined. The Cartesia sonic variants and Deepgram aura models follow the established naming convention.
40-46: LGTM!InworldModels and updated RimeModels types are correctly defined with the
provider/modelnaming convention.
76-86: LGTM!The
TTSOptions<TModel>conditional type correctly maps each model type to its corresponding options interface, with proper nesting and fallback toRecord<string, unknown>..changeset/kind-coins-add.md (1)
1-5: LGTM!The changeset correctly identifies this as a patch release for
@livekit/agents. A patch bump is appropriate since the changes are additive (new model types) without breaking existing functionality.
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
lukasIO
left a comment
lukasIO
left a comment
There was a problem hiding this comment.
coderabbitai is right about the formatter still needing to run.
Looks good to me.
A general question: should we surface the fact that some of these are deprecated now? Right now users don't get any info on this, wondering if we should collect the deprecated ones in an Array and log a warning if they're still being used?
we should have a better strategy of handling models.. it seems relying on the SDK isn't the best solution. (someone using an older version of the model wouldn't be able to see the warning). ideally these are coming from the server side in our protocol.. the ability to return an error. |
Summary by CodeRabbit
New Features
Deprecations
✏️ Tip: You can customize this high-level summary in your review settings.