feat: add LiteLLM AI gateway adapter#729
Merged
tombeckenham merged 4 commits intoJul 20, 2026
Merged
Conversation
Contributor
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughThe OpenAI-Compatible adapter documentation now recognizes LiteLLM, explains self-hosted LiteLLM Proxy configuration and model routing, and refreshes the adapter metadata timestamp. ChangesOpenAI-Compatible LiteLLM Documentation
Estimated code review effort: 1 (Trivial) | ~3 minutes Suggested reviewers: 🚥 Pre-merge checks | ✅ 3 | ❌ 2❌ Failed checks (2 warnings)
✅ Passed checks (3 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
🧹 Nitpick comments (3)
packages/ai-litellm/package.json (1)
52-52: ⚡ Quick winInconsistent workspace protocol for internal peer dependency.
The guideline specifies using
workspace:*for internal package dependencies, but this peer dependency usesworkspace:^. For consistency with the regular dependencies (lines 56-57) and the coding guidelines, consider usingworkspace:*here as well.📝 Suggested change for consistency
- "`@tanstack/ai`": "workspace:^", + "`@tanstack/ai`": "workspace:*",As per coding guidelines, "Use the
workspace:*protocol for internal package dependencies in package.json".🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/ai-litellm/package.json` at line 52, The peer dependency entry "`@tanstack/ai`": "workspace:^" is using the wrong workspace protocol; update this dependency to use the internal workspace protocol "workspace:*" so it matches the other internal dependencies and the project guideline (look for the "`@tanstack/ai`" key in packages/ai-litellm/package.json and replace the version string).Source: Coding guidelines
packages/ai-litellm/src/adapters/text.ts (1)
24-26: 💤 Low valueConsider using
unknowninstead ofanyfor provider options.Line 26 uses
Record<string, any>as a type parameter. While this provides maximum flexibility, usingRecord<string, unknown>would provide better type safety by requiring explicit type assertions when accessing provider options, without losing functionality.💡 Suggested type safety improvement
export class LiteLLMTextAdapter extends OpenAIBaseChatCompletionsTextAdapter< string, - Record<string, any>, + Record<string, unknown>, readonly [], Record<string, never>, readonly []🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/ai-litellm/src/adapters/text.ts` around lines 24 - 26, The generic parameter on LiteLLMTextAdapter uses Record<string, any> which weakens type safety; update the adapter declaration that extends OpenAIBaseChatCompletionsTextAdapter (the type parameter currently written as Record<string, any>) to use Record<string, unknown> so provider options are typed as unknown and require explicit assertions or narrowing when accessed.packages/ai-litellm/src/utils/client.ts (1)
15-21: 💤 Low valueConsider narrowing the catch clause or re-throwing unexpected errors.
The empty catch clause catches all errors from
getApiKeyFromEnv, not just the expected missing environment variable error. While this is unlikely to cause issues in practice (sincegetApiKeyFromEnvhas a single failure mode), catching all errors without inspection can mask unexpected failures.Consider either checking the error type before re-throwing, or documenting that all errors from
getApiKeyFromEnvare treated as missing API key errors.🛡️ Alternative approach with error inspection
export function getLiteLLMApiKeyFromEnv(): string { try { return getApiKeyFromEnv('LITELLM_API_KEY') - } catch { + } catch (error) { + // getApiKeyFromEnv only throws when env var is missing, but be explicit throw new Error( 'LITELLM_API_KEY is required. Please set it in your environment variables or use createLitellmText() with an explicit API key.', ) } }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/ai-litellm/src/utils/client.ts` around lines 15 - 21, The current catch block around getApiKeyFromEnv('LITELLM_API_KEY') swallows all errors; change it to catch the error as a variable, inspect it (e.g., check instanceof or error.message content that indicates "missing API key" from getApiKeyFromEnv), and only convert that specific case to the user-facing Error about LITELLM_API_KEY; for any other unexpected error re-throw the original error so failures in getApiKeyFromEnv are not masked. Ensure references to getApiKeyFromEnv and the existing message mentioning createLitellmText() remain unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@packages/ai-litellm/package.json`:
- Line 52: The peer dependency entry "`@tanstack/ai`": "workspace:^" is using the
wrong workspace protocol; update this dependency to use the internal workspace
protocol "workspace:*" so it matches the other internal dependencies and the
project guideline (look for the "`@tanstack/ai`" key in
packages/ai-litellm/package.json and replace the version string).
In `@packages/ai-litellm/src/adapters/text.ts`:
- Around line 24-26: The generic parameter on LiteLLMTextAdapter uses
Record<string, any> which weakens type safety; update the adapter declaration
that extends OpenAIBaseChatCompletionsTextAdapter (the type parameter currently
written as Record<string, any>) to use Record<string, unknown> so provider
options are typed as unknown and require explicit assertions or narrowing when
accessed.
In `@packages/ai-litellm/src/utils/client.ts`:
- Around line 15-21: The current catch block around
getApiKeyFromEnv('LITELLM_API_KEY') swallows all errors; change it to catch the
error as a variable, inspect it (e.g., check instanceof or error.message content
that indicates "missing API key" from getApiKeyFromEnv), and only convert that
specific case to the user-facing Error about LITELLM_API_KEY; for any other
unexpected error re-throw the original error so failures in getApiKeyFromEnv are
not masked. Ensure references to getApiKeyFromEnv and the existing message
mentioning createLitellmText() remain unchanged.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 60ddb427-349a-40ad-b1f4-0a7e5b16d8db
📒 Files selected for processing (7)
packages/ai-litellm/package.jsonpackages/ai-litellm/src/adapters/text.tspackages/ai-litellm/src/index.tspackages/ai-litellm/src/utils/client.tspackages/ai-litellm/src/utils/index.tspackages/ai-litellm/tsconfig.jsonpackages/ai-litellm/vite.config.ts
Contributor
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
testing/e2e/src/lib/feature-support.ts (1)
11-149:⚠️ Potential issue | 🔴 CriticalFix LiteLLM E2E wiring:
provider: 'litellm'is marked supported, but the harness has no LiteLLM adapter.
testing/e2e/src/lib/feature-support.ts+testing/e2e/tests/test-matrix.tsincludelitellmfor the added feature keys, buttesting/e2e/src/lib/providers.ts’screateTextAdapter()has nolitellmentry indefaultModelsor thefactoriesmap;testing/e2e/src/routes/api.chat.tsalways callscreateTextAdapter(provider, ...), so this will fail at runtime forprovider === 'litellm'.testing/e2e/src/lib/llmock-server.ts’sLLMockrecord config doesn’t includelitellm, so recording/fixture generation for LiteLLM can’t work either.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@testing/e2e/src/lib/feature-support.ts` around lines 11 - 149, The test harness marks provider 'litellm' supported but the runtime wiring is missing; add a LiteLLM adapter and fixtures: update createTextAdapter (in testing/e2e/src/lib/providers.ts) by adding 'litellm' to defaultModels and the factories map (implement or reference a LiteLLM TextAdapter factory) so createTextAdapter(provider, ...) in routes/api.chat.ts can instantiate it, and update the LLMock record config (in testing/e2e/src/lib/llmock-server.ts) to include a 'litellm' entry so recording/fixture generation works for LiteLLM; ensure the adapter follows the same interface used by other providers and reuse existing adapter patterns for naming and behavior.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Outside diff comments:
In `@testing/e2e/src/lib/feature-support.ts`:
- Around line 11-149: The test harness marks provider 'litellm' supported but
the runtime wiring is missing; add a LiteLLM adapter and fixtures: update
createTextAdapter (in testing/e2e/src/lib/providers.ts) by adding 'litellm' to
defaultModels and the factories map (implement or reference a LiteLLM
TextAdapter factory) so createTextAdapter(provider, ...) in routes/api.chat.ts
can instantiate it, and update the LLMock record config (in
testing/e2e/src/lib/llmock-server.ts) to include a 'litellm' entry so
recording/fixture generation works for LiteLLM; ensure the adapter follows the
same interface used by other providers and reuse existing adapter patterns for
naming and behavior.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: f15d9991-7dff-4f86-98ea-d2a7d90976cb
📒 Files selected for processing (4)
.changeset/add-litellm-adapter.mdtesting/e2e/src/lib/feature-support.tstesting/e2e/src/lib/types.tstesting/e2e/tests/test-matrix.ts
✅ Files skipped from review due to trivial changes (1)
- testing/e2e/tests/test-matrix.ts
…ider # Conflicts: # testing/e2e/src/lib/feature-support.ts # testing/e2e/src/lib/types.ts # testing/e2e/tests/test-matrix.ts
LiteLLM exposes an OpenAI Chat Completions endpoint, so it is already usable through the existing @tanstack/ai-openai/compatible adapter (added in TanStack#676) — no dedicated package needed. Add a "LiteLLM Proxy" section to the OpenAI-Compatible adapter docs instead. Removes the @tanstack/ai-litellm package, its changeset, and the E2E matrix wiring introduced by this PR: the package duplicated the generic adapter (and with weaker typing — model was `string`), did not type-check or build (wrong message-metadata generic), had no unit tests, and its E2E matrix entries were never wired into providers.ts so they could not run. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
|
Sorry for the delay in looking at this. We now support LiteLLM through the |
tombeckenham
self-requested a review
July 20, 2026 07:35
|
View your CI Pipeline Execution ↗ for commit c55c423
☁️ Nx Cloud last updated this comment at |
tombeckenham
approved these changes
Jul 20, 2026
tombeckenham
left a comment
tombeckenham
left a comment
Contributor
There was a problem hiding this comment.
Altered to docs only to mention LiteLLM
@tanstack/ai
@tanstack/ai-acp
@tanstack/ai-angular
@tanstack/ai-anthropic
@tanstack/ai-bedrock
@tanstack/ai-claude-code
@tanstack/ai-client
@tanstack/ai-code-mode
@tanstack/ai-code-mode-skills
@tanstack/ai-codex
@tanstack/ai-devtools-core
@tanstack/ai-elevenlabs
@tanstack/ai-event-client
@tanstack/ai-fal
@tanstack/ai-gemini
@tanstack/ai-grok
@tanstack/ai-grok-build
@tanstack/ai-groq
@tanstack/ai-isolate-cloudflare
@tanstack/ai-isolate-node
@tanstack/ai-isolate-quickjs
@tanstack/ai-mcp
@tanstack/ai-mistral
@tanstack/ai-ollama
@tanstack/ai-openai
@tanstack/ai-opencode
@tanstack/ai-openrouter
@tanstack/ai-preact
@tanstack/ai-react
@tanstack/ai-react-ui
@tanstack/ai-sandbox
@tanstack/ai-sandbox-cloudflare
@tanstack/ai-sandbox-daytona
@tanstack/ai-sandbox-docker
@tanstack/ai-sandbox-local-process
@tanstack/ai-sandbox-sprites
@tanstack/ai-sandbox-vercel
@tanstack/ai-solid
@tanstack/ai-solid-ui
@tanstack/ai-svelte
@tanstack/ai-utils
@tanstack/ai-vue
@tanstack/ai-vue-ui
@tanstack/openai-base
@tanstack/preact-ai-devtools
@tanstack/react-ai-devtools
@tanstack/solid-ai-devtools
commit: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🎯 Changes
Adds
@tanstack/ai-litellm, a tree-shakeable adapter for the LiteLLM AI gateway proxy. Gives users access to 100+ LLM providers (OpenAI, Anthropic, Google, Azure, AWS Bedrock, Ollama, Groq, Mistral, and more) through a single adapter.Follows the same pattern as
ai-groqandai-grok: extendsOpenAIBaseChatCompletionsTextAdapterwith abaseURLoverride pointing at the LiteLLM proxy. New package atpackages/ai-litellm/withLiteLLMTextAdapter,createLitellmText(), andlitellmText()factory functions. ReadsLITELLM_API_KEYfrom env, defaults proxy URL tohttp://localhost:4000/v1.LiteLLM added to E2E feature-support matrix and test-matrix for all OpenAI-compatible features (chat, tool-calling, structured output, etc.).
Example usage:
✅ Checklist
pnpm run test:pr. (pnpm installblocked by upstreamchokidar@4.0.3trust downgrade in the svelte example, unrelated to this change.)🚀 Release Impact
Summary by CodeRabbit