Observability & Evals tools for Local / Ollama

An open-source GPU cluster manager for running and managing LLMs

AI gateway exposing a unified OpenAI-compatible API across OpenAI, Anthropic, Gemini, Groq, xAI, Ollama and other providers, with routing, usage tracking, rate limits, and guardrails

Docker Compose stack for local AI with Ollama, a LiteLLM gateway, RAG, voice services, MCP tools, persistent data, and health checks

CLI/TUI tool for managing localization files (.resx, JSON, Android, iOS) with LLM-powered translation via Ollama, validation, and code scanning for unused/missing keys

Self-hosted, no-code platform for building and running AI agents on a visual canvas, with built-in local inference (llama.cpp, MLX, vLLM) or any OpenAI-compatible API. Durable crash-resumable runs, per-run trace waterfalls with OTLP export, MCP tools, RAG on pgvector, and human-in-the-loop approvals

Finds which part of an agent's context caused a bad decision by rerunning it with pieces removed, checks candidate fixes the same way, and writes a pytest regression test. Runs locally with OpenAI, Anthropic, LangGraph and Ollama

LLM deployment planner and benchmarking CLI: given a model and a GPU it answers will it fit, will it hit your SLO, and what will it cost. Searches model x quantization x backend x GPU count against VRAM, quality, latency, cost, energy and an opt-in safety gate; models tensor/pipeline parallelism, MoE active parameters, FP8, KV-quant and self-host-vs-API break-even. Every number is labeled measured

Speed benchmarking a 7B LLM on different gcloud VMs (using llama.cpp)