cicada0007

Relay Bench — AI skill for Claude Code

AI community

Open, pre-registered benchmark of Relay against Claude Code, Codex, Hermes and Prime Agent on the same model.

How to install Relay Bench

This entry records only its repository, not the path inside it, so there is no exact command to give. Open cicada0007/relay-bench and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

What Relay Bench does

Open, pre-registered benchmark of Relay against Claude Code, Codex, Hermes and Prime Agent on the same model.

Alternatives in AI

Folkbench — AI API 中转站评测排行与切换工具：实测可用率、延迟与价格 · Open-source AI API relay benchmark, rankings & switcher for Claude Code / Co 420 ★
Crux — Taking the pi coding agent to Claude Code-level performance: 0.539 → 0.773 pass@1 on Terminal-Bench 2.1 with t 187 ★
People Search Bench — The first open benchmark for evaluating AI-powered people search agents 128 ★

README

relay-bench

An open benchmark of [Relay](https://relayevals.com), a coding-agent harness built by Relay, against the tools people use directly: Claude Code, Codex, Hermes and Prime Agent, each on the same model as Relay.

**Status:** round 1 is complete. Its numbers are in [results/](results/) and on [relayevals.com/ai-coding-agent-benchmark](https://relayevals.com/ai-coding-agent-benchmark).

PREREGISTRATION.md fixes the tests, measures, statistics and the words we may use about the results, before the first paid run. Its commit date is the record.
Tasks, hidden judges, the runner and the cost meter arrive here as they are built.
Results, with every run's record and transcript, will be published here and on relayevals.com, losses included.

Anyone will be able to re-run each test on their own API key with one command. The cheapest test is planned to cost about $30 to repeat.

relay-bench is built and maintained by Relay. Questions: arnav@relayevals.com.

Released under the [MIT License](LICENSE).