CIAgent

suniel12/ciagent

Pytest-native regression testing for AI agents — golden-trace diffing, cost guardrails, multi-run stability scoring with flip attribution, LLM-judge auditing, and one-command import of production traces (OTel/Langfuse/LangSmith) into CI tests

Python Apache-2.0

View on GitHub → Find it in the atlas

Install

uvx ciagent test --mock --runs 3

Repository reader

Explore before you clone

Read the project’s README, skills and supporting Markdown here. Rendered view is shown first; source is one click away when you need the exact text.

Fetched from GitHub
Open on GitHub
Loading the project’s README…

Public files are loaded on demand from the repository. The atlas does not copy or modify them.

Screenshot

Screenshot or social card for CIAgent
The project's own screenshot, fetched from source rather than copied into this repository.

Platform support

Derived from the project's own README, install route, CI configuration and release assets — not from running it.
PlatformRuns hereOn what evidence
WindowsLikelyInferred from the language or runtime. Nothing in the project says so.
WSL2LikelyInferred from the language or runtime. Nothing in the project says so.
macOSLikelyInferred from the language or runtime. Nothing in the project says so.
LinuxLikelyInferred from the language or runtime. Nothing in the project says so.
DockerNoNo evidence of support anywhere in the project.

Where it came from

Named by 1 of the 39 lists the atlas merges.

  • AI Agents (Jenqyang)

How it is classified

One topic, 1 harness. Each link is a prerendered page of everything else in that slice.

Observability & EvalsLangChain / LangGraph

Related in Observability & Evals

Closest first — projects that plug into the same harnesses, then the ones the most lists agreed on.

  • Langfuselangfuse/langfuseThe most widely adopted self-hostable LLM observability platform: traces every agent step, manages prompt versions, and runs evals in one tool…
  • HeliconeHelicone/heliconeOpen-source LLM observability proxy (YC W23) with the largest open-source pricing database (300+ models). One-line proxy integration provides cost…
  • Opikcomet-ml/opikComet's open-source AI observability and evaluation platform: deep tracing of LLM calls, conversation logging, and agent activity, plus built-in…
  • AgentOpsAgentOps-AI/agentopsOpen-source agent engineering platform (YC W24) with session replay, cost tracking, and failure detection across 10+ frameworks including CrewAI…
  • EvalViewhidai25/eval-viewRegression testing for AI agents. Snapshot behavior, detect tool-call and output regressions, with golden-baseline diffing and LLM-as-judge scoring…
  • Terminal-Benchharbor-framework/terminal-bench-1The de facto benchmark and execution harness for terminal agents: real sandboxed shell tasks verified by deterministic test scripts, now at 2.x on…
  • Agent-Solarpanels-TutorialPairrot-Lore/agent-solarpanels-tutorialThis repository contains an AI agent built with LangGraph to calculate energy savings for solar panels. The project is designed for use w…
  • AgentPrismevilmartians/agent-prismOpen-source React component library (Evil Martians) that transforms OpenTelemetry trace data into interactive visualizations: tree view…