Cceval

cceval

Evaluate and benchmark your CLAUDE.md effectiveness with automated testing.

Why?

Your `CLAUDE.md` file guides how Claude Code behaves in your project. But how do you know if your instructions are actually working?

**cceval** lets you:

• Test different system prompt variations
• Compare performance across metrics
• Find what actually improves Claude's behavior
• Share benchmarks with your team

Installation

# Global install (recommended)
bun add -g cceval

# Or per-project
bun add -D cceval

**Requirements:**

• Bun ≥ 1.0
• Claude Code CLI installed and authenticated

Quick Start

# Run with default prompts and variations
cceval run

# Generate report from existing results
cceval report evaluation-results.json

# Create custom config
cceval init

Usage

Basic Evaluation

Run with sensible defaults (5 variations × 5 prompts = 25 tests):

cceval run

This tests:

• baseline: Minimal "You are a helpful assistant"
• gateFocused: Clear pass/fail criteria
• bunFocused: Bun-specific instructions
• rootCauseFocused: Root cause protocol
• antiPermission: Trust-based prompting

Against prompts that test:

• Reading files before coding
• Using Bun instead of Node
• Fixing root cause vs surface symptoms
• Asking permission appropriately

Custom Configuration

Create a config file:

cceval init

Edit `cceval.config.ts`:

import type { EvalConfig } from "cceval"

const config: EvalConfig = {
  prompts: {
    // Your test scenarios
    authentication: "Add login functionality to the app.",
    performance: "The dashboard is slow, optimize it.",
    testing: "Add tests for the user service.",
  },

  variations: {
    // Your system prompt variations
    baseline: "You are a helpful coding assistant.",

    myClaudeMd: `You are evaluated on gates. Fail any = FAIL.
1. Read files before coding
2. State plan then pr