Continuously evaluate and optimize your AI agent: build Harbor-format evals from your code, requirements, and traces, then get harness improvements as pull requests.
Ask your coding agent in plain language. RELAI turns the behavior you expect into repeatable agent tests, runs your agent against them in isolated containers, and proposes changes that are checked against every earlier test before you merge them.
/plugin marketplace add relai-ai/coding-agent-plugins
/plugin install relai@relai
codex plugin marketplace add relai-ai/coding-agent-plugins
codex plugin add relai@relaiStart a new Codex session to load the skills.
This marketplace used to be named relai-claude for Claude Code and
relai-codex for Codex. Updating it doesn't rename an existing install, so
remove the old one and add the marketplace again.
In Claude Code:
/plugin marketplace remove relai-claude
/plugin marketplace add relai-ai/coding-agent-plugins
/plugin install relai@relai
For Codex:
codex plugin remove relai@relai-codex
codex plugin marketplace remove relai-codex
codex plugin marketplace add relai-ai/coding-agent-plugins
codex plugin add relai@relaiSkip this if you installed the plugin with relai setup --agent. On that
machine, relai-claude and relai-codex are the RELAI CLI's own marketplaces.
Install through the RELAI CLI with relai setup --agent cursor or
relai setup --agent copilot (see below). Cursor Teams and Enterprise admins
can also import this repository as a team marketplace under
Dashboard → Plugins & MCPs → Add Marketplace → Import from Repo.
The plugin runs the RELAI CLI on macOS 15+ (Apple silicon) or Linux (x86_64 and ARM64). If you installed the plugin from a marketplace, install the CLI and sign in:
curl -fsSL https://api.relai.ai/install.sh | bash
relai auth loginDon't also install the plugin through the CLI. Skip relai setup --agent and
relai plugins install, and answer no when interactive setup, including the
first-run setup, offers to configure the plugin: a CLI install adds a second
copy, and your coding agent then loads every RELAI skill twice. To switch to
the copy the CLI installs, first uninstall this one:
/plugin uninstall relai@relai in Claude Code or
codex plugin remove relai@relai for Codex.
To start from the CLI instead, let setup install the plugin that matches your CLI version:
curl -fsSL https://api.relai.ai/install.sh | bash
relai setup --agent <claude|codex|cursor|copilot>RELAI also needs Git, uv, and a running Docker daemon with Compose. See the
installation guide.
Open your agent's repository in your coding agent and ask:
Use RELAI to check whether this agent project is ready. Keep the check read-only and explain any blockers.
Then initialize it:
Use RELAI to initialize this agent project. Keep provider keys out of chat and Git.
- Evals in your repository. Agent tests and benchmarks use the open-source Harbor format and are versioned next to your code.
- EvalsWiki. Your agent's requirements, risks, coverage, and open questions
live as Markdown files in
.relai/evals-wiki/. - Simulation. RELAI runs your agent against each scenario in an isolated Harbor environment and scores the result with code evaluators and LLM judges.
- Harness optimization. RELAI changes prompts, context, tools, workflows, memory, and agent logic for your quality and cost objective, then opens a pull request or leaves the change on a local branch.
- Your existing tracing. Ingest traces exported from Arize, Braintrust, LangSmith, Langfuse, Helicone, or your own logs to keep evals current.
- Terminal-Bench 2.0: pass rate rose from 62.5% to 79.2% after the first optimization round. Read the article
- AutomationBench: 3 to 10 pass-rate points gained and 22–44% lower cost per successful task across three frontier models. Read the article
Do I need traces or an existing eval suite to start? No. RELAI starts from your code, requirements, and existing tests. Traces add evidence as they arrive.
Does RELAI replace my observability platform? No. Keep Arize, Braintrust, LangSmith, Langfuse, or your own stack for monitoring. RELAI uses their traces to keep evals current.
How does RELAI avoid regressions? Earlier evals stay in the loop. Every candidate change is checked against all accumulated coverage, so a new fix can't quietly break what already worked.
Where do evals and changes live? In your repository: EvalsWiki, agent tests, and harness changes. RELAI hosts run history, metrics, and evidence.
Which agents does it work with? RELAI is framework-agnostic. It works best with a Git-tracked Python, TypeScript, JavaScript, or Go project that has a runnable local entry point. See compatibility.
| Host | Package | Catalog |
|---|---|---|
| Claude Code | plugins/claude/relai/ |
.claude-plugin/marketplace.json |
| Codex | plugins/codex/relai/ |
.agents/plugins/marketplace.json |
| Cursor | plugins/cursor/relai/ |
.cursor-plugin/marketplace.json |
| GitHub Copilot | copilot/skills/ |
Published with gh skill publish |
The OpenAI submission archive is artifacts/relai-openai-vX.Y.Z.zip. These
files are generated by the RELAI CLI release; see
CONTRIBUTING.md before proposing changes.
Copyright 2026 RELAI Inc.
Licensed under the Apache License, Version 2.0. The license covers the files in this repository, not the RELAI CLI or RELAI's service, and grants no rights to the RELAI name or trademarks.