Harnesses & Runtime Infra

DeepSeek's official open-source agent harness (August 2026) built on an "everything is a plugin" architecture powered by Cordis: models, tools, skills, UI surfaces, and even the desktop shell are plugins composed through dependency injection rather than a monolith with extension points, with an accompanying paper on spatiotemporal composability. The clearest first-party reference for treating harn

An Open Source Machine Learning Framework for Everyone

Ollama is a tool for running large language models locally, offering easy setup for macOS, Windows, Linux, and Docker, along with a library of models and quickstart guides for customization and integration github | github profile

User-friendly AI Interface (Supports Ollama, OpenAI API, ...)

C/C++ inference. CPU, GPU, Apple Silicon. Foundation of local AI

Minimal terminal coding harness built around "lazy skills": each capability keeps only a one-line description in active context, loading full instructions and tool schemas only when invoked. Keeps the system prompt under 1,000 tokens versus 7,000–10,000 for typical agents, making it a concrete reference for context-minimal harness design

Tensors and Dynamic neural networks in Python with strong GPU acceleration

68x cheaper. Strong reasoning. V4 Lite in preview

A high-throughput and memory-efficient inference and serving engine for LLMs

Lobe Chat - an open-source, modern-design AI chat framework. Supports Multi AI Providers( OpenAI / Claude 3 / Gemini / Ollama / DeepSeek / Qwen), Knowledge Base (file upload / knowledge management / RAG ), Multi-Modals (Plugins/Artifacts) and Thinking. One-click FREE deployment of your private ChatGPT/ Claude / DeepSeek application

Run VS Code on any machine anywhere and access it in the browser

OSS local chat. Consumer hardware

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

Moby is an open-source project created by Docker to enable and accelerate software containerization

Machine Learning in Python

The all-in-one Desktop & Docker AI application with built-in RAG, AI agents, No-code agent builder, MCP compatibility, and more

Keras is a deep learning API written in Python, running on top of the machine learning platform TensorFlow

(51.2k ⭐) - API service that wraps Antigravity, Codex, Claude Code, and Grok Build behind OpenAI-, Gemini-, Claude-, and Codex-compatible endpoints

Port of OpenAI's Whisper model in C/C++

Drop-in OpenAI API replacement. No GPU required

Self-hosted multi-model chat. All major providers

Jan is an open-source, development-stage ChatGPT alternative that operates fully offline on diverse hardware platforms, supporting universal architectures from PCs to multi-GPU clusters github | github profile

The Cloud-Native API Gateway and AI Gateway

Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective

LLM training and inference system whose ColossalChat application implements LLM with RLHF

Open-source, community-driven terminal coding agent harness written in Rust with a model-agnostic runtime, OS-level sandbox (Seatbelt/Landlock/seccomp/bwrap), resumable fleets via an append-only ledger, and a /model command that lets you switch providers mid-task. A strong example of treating the harness — not the model — as the product surface, with clear separation between interactive TUI, headl

An Open Bilingual Pre-Trained Model, quantization of ChatGLM-130B, can run on consumer-level GPUs

FastChat is a platform for training, serving, and evaluating large language model chatbots, featuring an open-source distributed multi-model system, API compatibility, and a dataset for LLM conversations github | demo | discord | twitter

SGLang is a fast serving framework for large language models and vision language models

Autograd and XLA for high-performance machine learning research

DeepSeek-native coding agent harness engineered around prefix-cache stability as a loop invariant: immutable-prefix / append-only-log / volatile-scratch partitioning achieves 99.82% cache-hit rates and ~5× cost reduction on long sessions. The most detailed public case study of designing an entire agent loop to preserve a provider-specific economic property rather than treating caching as an aftert

A fast open framework for deep learning

You like pytorch? You like micrograd? You love tinygrad! ❤️

(23.7k ⭐) - Routes Claude Code, Codex, Cursor, Cline, and other coding agents to dozens of model providers with automatic fallback

Deep learning framework to train, deploy, and ship AI products Lightning fast

Code and documentation to train Stanford's Alpaca models, and generate the data

The Modular Platform (includes MAX & Mojo)

FastGPT is a knowledge-based platform built on the LLMs, offers a comprehensive suite of out-of-the-box capabilities such as data processing, RAG retrieval, and visual AI workflow orchestration, letting you easily develop and deploy complex question-answering systems without the need for extensive setup or configuration

MLX: An array framework for Apple silicon

Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library

A central community library by Hugging Face for AI in robotics — end-to-end learning tools, data pipelines, and support for training/deploying VLA models

An open-source framework for the end-to-end machine learning lifecycle, helping developers track experiments, evaluate models/prompts, deploy models, and add observability with tracing

NEW (Apr 2026) Agentic focus. 1M ctx. Repo-level coding. MCP-native

SpaceXAI's open-source terminal coding agent harness: a Rust-based fullscreen TUI with an extensible tool runtime, MCP/skills/hooks support, and headless/embedded modes via ACP. A useful first-party counterpoint to Claude Code and Codex CLI for studying how a new model provider structures the loop, checkpoints workspace state, and exposes the same harness through interactive and scripted surfaces

(1.5k ⭐) - Menu bar and terminal app for choosing model providers for Claude Code, Codex, and other coding agents

Mozilla project that packages LLMs into single executable files that run anywhere

fast inference engine for whisper in C++ using CTranslate2

Machine Learning Framework from Industrial Practice

The easiest way to automate your data

verl: Volcano Engine Reinforcement Learning for LLMs

ncnn is a high-performance neural network inference framework optimized for the mobile platform

Universal LLM Deployment Engine with ML Compilation

State-of-the-art Parameter-Efficient Fine-Tuning

Build multimodal AI services via cloud native technologies · Model Serving · Generative AI · Neural Search · Cloud Native

Minimalist ML framework for Rust

Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler

Train transformer language models with reinforcement learning

Universal AI IDE account manager for Codex, GitHub Copilot, Windsurf, Kiro, Cursor, Gemini CLI. Multi-account switching, quota monitoring

High-performance In-browser LLM Inference Engine

A fast, distributed, high performance gradient boosting (GBT, GBDT, GBRT, GBM or MART) framework based on decision tree algorithms, used for ranking, classification and many other machine learning tasks

Instruct-tune LLaMA on consumer hardware

(8.5k ⭐) - A local provider proxy and dashboard that routes Codex CLI, the Codex app, SDKs, and Claude Code to multiple LLM providers

The Cloud-Native API Gateway and AI Gateway with extensive plugin system and AI capabilities

Workflow engine for Kubernetes

Alibaba's open-source coding model, leading choice for self-hosted/local deployment in 2026

State-of-the-art Machine Learning for the web. Run 🤗 Transformers directly in your browser, with no need for a server!

Meta's open-source LLM optimized for code generation and completion

A compact, inspectable open-source agent harness from HKUDS (April 2026) featuring a built-in personal agent (ohmo), auto-compaction with session preservation, MCP HTTP transport, and multimodal gateway support. Excellent reference for understanding how a small, modular harness can support multi-day sessions without manual context management

Use PEFT or Full-parameter to finetune 450+ LLMs (Qwen2.5, InternLM3, GLM4, Llama3.3, Mistral, Yi1.5, Baichuan2, DeepSeek-R1, ...) and 150+ MLLMs (Qwen2.5-VL, Qwen2-Audio, Llama3.2-Vision, Llava, InternVL2.5, MiniCPM-V-2.6, GLM4v, Xcomposer2.5, Yi-VL, DeepSeek-VL2, Phi3.5-Vision, GOT-OCR2, ...)

Machine Learning Toolkit for Kubernetes

StableLM: Stability AI Language Models

ChatGLM2-6B is the second-generation version of the open-source bilingual (Chinese-English) chat model ChatGLM-6B

A hyperparameter optimization framework

Inference engine for TensorRT on Nvidia GPUs

Distributed training framework for TensorFlow, Keras, PyTorch, and Apache MXNet

Open-source VLA models from Physical Intelligence, including π₀ and π₀.5 — flow-based vision-language-action models pretrained on large-scale robot data with fine-tuning support

An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning

Open deep learning compiler stack for cpu, gpu and specialized accelerators

The Jupyter notebook is a web-based notebook environment for interactive computing

A blazing fast AI Gateway with integrated guardrails. Route to 200+ LLMs, 50+ AI Guardrails with 1 fast & friendly API

serving the OpenAI CLIP model

A tool designed to streamline the fine-tuning of various AI models, offering support for multiple configurations and architectures

Open-source framework for building AI SRE agents with 60+ observability and remediation tool integrations plus a synthetic incident evaluation environment. The clearest open-source reference for turning infrastructure incident response into a trainable, evaluable agent harness rather than a one-off chatbot

An open platform for operating large language models (LLMs) in production. Fine-tune, serve, deploy, and monitor any LLMs with ease

10x Faster Long-Context LLM By Smart KV Cache Optimizations

BISHENG is an open LLM devops platform for next generation Enterprise AI applications. Powerful and comprehensive features include - GenAI…

(11.6k ⭐) - The power of Claude Code + Gemini / OpenAI / Grok / OpenRouter / Ollama / Custom Model working as one

a toolbox built on top of TensorFlow that allows to train and test deep learning models without the need to write code

Fast State-of-the-Art Tokenizers optimized for Research and Production

The Triton Inference Server provides an optimized cloud and edge inferencing solution

OpenVINO™ is an open source toolkit for optimizing and deploying AI inference

Efficient finetuning approach that reduces memory usage enough to finetune a 65B parameter model on a single 48GB GPU while preserving full 16-bit finetuning task performance

Kedro is an open-source Python framework for creating reproducible, maintainable and modular data science code

Large Language Model Text Generation Inference

Databricks’ Dolly, a large language model trained on the Databricks Machine Learning Platform

AutoML for Image, Text, and Tabular Data

Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

Build and manage real-life data science projects with ease!

An Easy-to-use, Scalable and High-performance RLHF Framework (70B+ PPO Full Tuning & Iterative DPO & LoRA & RingAttention & RFT)