llm-cache-router

svalench/llm-cache-router

A lightweight, async-first Python library combining semantic caching, multi-provider routing (OpenAI, Anthropic, Gemini, Ollama, MiniMax, Qwen) and cost tracking. Cuts LLM spend 30-70% via vector-similarity cache with in-memory/Redis/Qdrant backends

Python MIT

View on GitHub → Find it in the atlas

Install

pip install llm-cache-router

Repository reader

Explore before you clone

Read the project’s README, skills and supporting Markdown here. Rendered view is shown first; source is one click away when you need the exact text.

Fetched from GitHub
Open on GitHub
Loading the project’s README…

Public files are loaded on demand from the repository. The atlas does not copy or modify them.

Screenshot

Screenshot or social card for llm-cache-router
GitHub's social card — no screenshot was found for this project, fetched from source rather than copied into this repository.

Platform support

Derived from the project's own README, install route, CI configuration and release assets — not from running it.
PlatformRuns hereOn what evidence
WindowsLikelyInferred from the language or runtime. Nothing in the project says so.
WSL2LikelyInferred from the language or runtime. Nothing in the project says so.
macOSLikelyInferred from the language or runtime. Nothing in the project says so.
LinuxLikelyInferred from the language or runtime. Nothing in the project says so.
DockerYesStated in the project's own README, install route, CI config or release assets.

Where it came from

Named by 1 of the 39 lists the atlas merges.

  • LLMOps (InftyAI)

How it is classified

One topic, 4 harnesses. Each link is a prerendered page of everything else in that slice.

Harnesses & Runtime InfraClaude / AnthropicCodex / OpenAIGemini / GoogleLocal / Ollama

Related in Harnesses & Runtime Infra

Closest first — projects that plug into the same harnesses, then the ones the most lists agreed on.

  • LobeChatlobehub/lobehubLobe Chat - an open-source, modern-design AI chat framework. Supports Multi AI Providers( OpenAI / Claude 3 / Gemini / Ollama / DeepSeek / Qwen)…
  • Llm-Interfacesamestrin/llm-interfaceA simple NPM interface for seamlessly interacting with 36 Large Language Model (LLM) providers, including OpenAI, Anthropic, Google Gemin…
  • OntheiaOntheia/ontheiaSelf-hosted AI agent platform with multi-provider LLM support (Claude, OpenAI, Gemini, Ollama), MCP-native tool integration, visual workflow…
  • opencodexlidge-jun/opencodex(8.5k ⭐) - A local provider proxy and dashboard that routes Codex CLI, the Codex app, SDKs, and Claude Code to multiple LLM providers
  • Simplerllmhassancs91/SimplerLLMSimplify interactions with Large Language Models
  • zen-mcp-serverBeehiveInnovations/pal-mcp-server(11.6k ⭐) - The power of Claude Code + Gemini / OpenAI / Grok / OpenRouter / Ollama / Custom Model working as one
  • ccmanagerkbwo/ccmanagerCoding agent session manager supporting Claude Code, Gemini CLI, Codex, Cursor, Copilot, Cline, OpenCode, Kimi CLI. Smart auto-approval via Haiku…
  • CLI Proxy APIrouter-for-me/CLIProxyAPI(51.2k ⭐) - API service that wraps Antigravity, Codex, Claude Code, and Grok Build behind OpenAI-, Gemini-, Claude-, and Codex-compatible endpoints