ariel-frischer

RA.Aid Swe Bench — Development skill for Claude Code

Development community

Streamlined interface for running the RA-AID agent on the SWE Bench dataset.

How to install RA.Aid Swe Bench

This entry records only its repository, not the path inside it, so there is no exact command to give. Open ariel-frischer/RA.Aid-swe-bench and copy the folder into ~/.claude/skills/, or the file into ~/.claude/agents/.

What RA.Aid Swe Bench does

Streamlined interface for running the RA-AID agent on the SWE Bench dataset.

Alternatives in Development

Add Dataset — Renamed → /portaljs-add-dataset 2.3k ★
Claude Engineer — A version of command line interface(CLI) that leverages the power of Claude Opus(3-4.7) 954 ★
Harnessrouter — HarnessRouter Community Edition: the self-hosted, Apache-2.0 edition of the unified interface for agent harnes 618 ★

README

SWE-bench RA-AID

Run SWE Bench Lite dataset with the RA-AID agent and evaluate your results!

Description

Streamlined interface for running the RA-AID agent on the SWE Bench dataset. It's designed to make it easier to test and evaluate the RA-AID agent's performance on software engineering tasks.

Requirements

Python >=3.9, <3.13
Poetry for this project's dependency management
RA-AID ^0.12.0 (must have ra-aid cli path working in the running shell)
uv for fast dependency installation for each attempt

Environment Variables

Depending on your chosen model in `config.py`, you'll need to set appropriate API keys:

OpenAI models: OPENAI_API_KEY
Anthropic models: ANTHROPIC_API_KEY
OpenRouter models: OPENROUTER_API_KEY

Set them in your SHELL, .env support is not implemented yet.

⚠️ Important Notes

**Parallel Processing**: The `MAX_THREADS` setting in `config.py` determines how many `ra-aid` instances run in parallel. Be cautious with high values as this can:
- Significantly increase API costs
- Potentially trigger rate limits
- Cause memory/CPU issues
**Cost Warning**: Different models have varying pricing. Running multiple instances in parallel with expensive models can quickly accumulate significant costs. Monitor your usage carefully!

Installation

git clone https://github.com/ariel-frischer/RA.Aid-swe-bench
cd swe-lite-ra-aid

poetry install

# Before running predictions some instances may need legacy python versions
# Install required legacy Python versions (<3.7) using pyenv
make install-pythons

Python Version Management

As instances are running, different repositories need different python versions, even for the same repo `environment_setup_commit` may be different and so we need to accurate install all needed python versions. Using `uv` is preferred during runtime i