bug-hunt-bench

Name: bug-hunt-bench Review
Item: bug-hunt-bench
Rating: 2.3
Author: Aaron Matthews

Skill Development Poor

Works inClaude Code

Poor Scanned — metadata only

Quantifies which AI models can reliably identify and fix production bugs, reducing uncertainty in delegating code review and remediation tasks to frontier LL…

Bug Hunt Bench: 105 real bugs in two production repos, frontier coding models (GPT-6, Claude, Grok, Gemini, DeepSeek...) find and fix them in their own CLI, graded blind. Live leaderboard + every receipt.

Ask Claude to identify and fix bugs in real production codebases using the benchmark CLI.
Generate a report comparing Claude's bug detection performance against other frontier models.
Find which bug categories Claude struggles with most across the 105 real production bugs.

170 starsFreeQuick setup

Below standard — Significant caveats apply. Not recommended without careful review of the security and evaluation sections.

Why it matters

Quantifies which AI models can reliably identify and fix production bugs, reducing uncertainty in delegating code review and remediation tasks to frontier LLMs. Provides objective evidence for build-vs-buy decisions on AI-assisted debugging.

Best for

Engineering leaders evaluating frontier coding models for bug detection and repair workflows in production systems.

Works in

Claude Code Claude Cowork Claude Chat

Source

https://github.com/phuryn/bug-hunt-bench

By phuryn

How to Get It

Option 1: Claude Desktop App (Code Mode)Click the + button next to the prompt box → Plugins → Add plugin. Search and click Install. Skills work in Claude Code only.

Option 2: Paste into Claude CodeCopy the command below and paste it into your conversation. Claude will install it.

Command

claude plugins install phuryn/bug-hunt-bench

Tip: Paste this into a Claude Code conversation. Verify command matches your Claude Code version.

Auto-generated from the tool's public listing — not hands-on verified. Cross-check against the source repo's README before running.

First thing to try

After installing, paste this into Claude:

Identify and fix bugs in real production codebases using the benchmark CLI

CostFree

Trust Signals Auto-scanned

Stars170Contributors2Last updated2026-10-27Known CVEsNone foundSources: GitHub Advisory Database + OSV.dev · Scanned 2026-10-27 · scanner v1

Community Pulse New

No community discussions found yet. This doesn't mean the tool isn't good — it may be new or serve a niche use case.

Reviewer notes

Auto-scanned review. These are observations, not a security certification.

Scored from trust signals (evidence-eval-v1): 109 GitHub stars; contributors unknown; last commit 0d ago; license no license.

Things to check

Scanned, not hands-on tested — this entry was auto-scanned from public metadata (GitHub metrics, license, security flags). No reviewer has run it, and no tool-specific limitations have been documented yet.

How to evaluate tools before deploying →

Data shown here comes from public APIs and automated scanning. Reviewer notes reflect one person's experience. This is not a security certification or legal recommendation. Always evaluate tools according to your own organization's policies.

Evaluation Scores

Evaluation

Ease of Use

1/5

Versatility

3/5

Reliability

3/5

Security

3/5

Overall score2.30 / 5.00 PoorEvaluatedSep 2026

Scored from trust signals (evidence-eval-v1): 109 GitHub stars; contributors unknown; last commit 0d ago; license no license.

← Back to the Claude Observatory