Benchmarks. GGUF. VRAM. What's it good at.
Every score is cited. Click a source name in any benchmark table to jump here.
Software engineering benchmark from Princeton NLP. Measures % of real GitHub issues a model can solve. Agent-dependent — we take best score per model.
VerifiedHuggingFace evaluation harness. Provides IFEval, GPQA Diamond, and MMLU-PRO scores. Note: evaluates without chain-of-thought, so reasoning models may score lower than expected.
VerifiedContamination-free benchmarks from Abacus.AI. Questions are refreshed monthly so models can't memorize answers. Provides coding and reasoning scores.
VerifiedBerkeley Function Calling Leaderboard. Tests tool use and function calling ability. We use the overall accuracy after stripping FC/Prompt suffixes.
VerifiedTAU-bench Retail and Airline scores. Measures agent ability in realistic customer service scenarios.
VerifiedHuman preference ELO ratings from blind A/B comparisons. Independent of benchmarks — captures real user satisfaction. Data via community API mirror.
VerifiedQuality index: composite intelligence score aggregating 10 benchmarks (agents, coding, general, scientific reasoning). Attribution required per API terms.
VerifiedBenchmark scores reported by the model creator in the HF README. Parsed from HTML tables, markdown tables, or inline text. These are self-reported — not independently verified.
ReportedScores pulled from comparison tables on other models' cards. Higher risk of misattribution — the parser checks model names but unusual layouts may need manual review.
ReportedModel metadata: parameter counts, license, architecture details, GGUF quantization files and sizes. Used for VRAM estimation.
MetadataVerified — confirmed by an independent leaderboard or evaluation harness
Reported — from the model creator's own benchmarks
Missing — no data available from any source
The pipeline never overwrites higher-confidence data with lower.