Skip to main content
Open Skills
agent-regression-evalsby Alireza RezvaniDevelopmentGitHub stars: 28.8k

Agent Regression Evals plugin: what it installs and how to add it

Tests agent, prompt, model and tool changes for regressions using repeated trials, trajectory checks and statistical merge gates.

From the Marketplace: claude-code-skills marketplace

Install

> /plugin marketplace add alirezarezvani/claude-skills> /plugin install agent-regression-evals@claude-code-skills

Run both lines inside Claude Code: the first adds the marketplace, the second installs the plugin.

What is inside

  • 1skill
  • 2commands
  • 1agent

About this plugin

The plugin turns repeated agent evaluation results into a merge decision. It checks whether tool-call paths match the expected rules, identifies flaky cases across trials, and compares baseline and candidate results using per-case averages, confidence intervals and an exact McNemar test. The gate can block changes for significant drops, critical-case regressions, missing cases or invalid quarantine entries.

Run the supplied slash commands with JSONL result files and a cases file, or invoke the deterministic Python scripts directly. The scripts support sample data and JSON output, and report pass@k, pass^k, Wilson intervals and trajectory deviations.

What's inside

  • 1 skill: agent-regression-evals
  • 2 commands: /cs-agent-evals and /cs-eval-gate
  • 1 agent: cs-eval-gatekeeper
  • 3 references and a CI job example

More from claude-code-skills

Related plugins

Skills, not plugins

Prefer a single skill for development work? 932 skills in the same category, each installed on its own.

Development skills