Inside The Machine
Evals & benchmarks
Evals & benchmarks

SWE-bench

Benchmark of real GitHub issues for coding agents.

Measures how close you are to the frontier.

—PENDING
Gap ScorePending test on Mac Studio
Where it sits
Catalog entryVerified it exists. Install steps not tested yet.
.md .json

Homepage Repo

AGENT VIEW · text/markdown · or send Accept: text/markdown
---
schema: "https://insidethemachine.ai/schema/listing.schema.json"
slug: "swe-bench"
name: "SWE-bench"
kind: "repo"
tier: "catalog"
category: "evals"
category_label: "Evals & benchmarks"
stack_layer: "tools"
homepage: "https://www.swebench.com"
repo: "https://github.com/SWE-bench/SWE-bench"
license: "MIT"
stars: 6033
last_push: "2026-09-18"
archived: false
tags: []
seen_in_videos: []
gap_score: null
gap_score_status: "pending test on Mac Studio"
verified_on: "2026-10-15"
---

# SWE-bench

**What:** Benchmark of real GitHub issues for coding agents.

**Why:** Measures how close you are to the frontier.

**Category:** Evals & benchmarks · **Stack layer:** tools

## Status

Catalog entry. Existence and repo metadata checked 2026-10-15 via the GitHub API. Install commands below are auto-extracted from the README and have NOT been executed or verified. Read the README before running anything; prefer a sandbox or venv.

## Install (unverified)

No install command found. no install command found in README; follow the README or docs

## Verify

No verified check yet. Minimum: the install command exits 0 and the tool's own help/version flag runs (exact flag: see README).

## Rollback (unverified)

No derived rollback. Remove what the install created (see README).

## Links

- Homepage: https://www.swebench.com
- Repo: https://github.com/SWE-bench/SWE-bench
- README: https://github.com/SWE-bench/SWE-bench#readme