# Evals & benchmarks (14)

Measures how close you are to the frontier.

| slug | name | tier | stars | what | md |
|---|---|---|---|---|---|
| promptfoo | promptfoo | catalog | 26019 | Test and red-team prompts, agents and RAG. | /setups/promptfoo.md |
| openai-evals | OpenAI Evals | catalog | 19607 | Framework and registry for evaluating LLMs. | /setups/openai-evals.md |
| deepeval | DeepEval | catalog | 18890 | Unit-test style evaluation for LLM apps. | /setups/deepeval.md |
| ragas | Ragas | catalog | 16034 | Metrics for evaluating RAG pipelines. | /setups/ragas.md |
| lm-evaluation-harness | LM Evaluation Harness | catalog | 14234 | The standard harness for academic LLM benchmarks. | /setups/lm-evaluation-harness.md |
| gorilla-bfcl | Gorilla / BFCL | catalog | 13052 | Function-calling models and the Berkeley tool-use leaderboard. | /setups/gorilla-bfcl.md |
| opencompass | OpenCompass | catalog | 7520 | Large evaluation platform across many benchmarks. | /setups/opencompass.md |
| swe-bench | SWE-bench | catalog | 6033 | Benchmark of real GitHub issues for coding agents. | /setups/swe-bench.md |
| simple-evals | simple-evals | catalog | 4666 | OpenAI's lightweight reference evals. | /setups/simple-evals.md |
| lmms-eval | LMMs-Eval | catalog | 4453 | Evaluation toolkit for multimodal models. | /setups/lmms-eval.md |
| inspect-ai | Inspect AI | catalog | 3012 | UK AISI framework for LLM and agent evals. | /setups/inspect-ai.md |
| terminal-bench | Terminal-Bench | catalog | 2599 | Benchmark of hard tasks in a real terminal. | /setups/terminal-bench.md |
| lighteval | LightEval | catalog | 2561 | Hugging Face's multi-backend eval toolkit. | /setups/lighteval.md |
| thinkingbox | ThinkingBox | catalog | 102 | Runs agents against MCP tools and checks what they really did. | /setups/thinkingbox.md |