Developer Benchmarks Uncensored Security AI Models on GitHub

A developer posting as JoasASantos has published a GitHub repository that benchmarks AI models marketed for offensive security work, focusing on builds distributed without the refusal behaviors that block hacking-related requests in mainstream assistants. The project is an early attempt to give penetration testers, red teamers and capture-the-flag players a side-by-side read on how these uncensored models perform on security tasks. It arrived on GitHub this week with little fanfare, and its details remain sparse.

The niche it targets has expanded over the past year. Fine-tunes and merges built on open-weight bases let anyone strip safety alignment, and a subset of those builds are pitched specifically at security practitioners who want a model that will write exploit code, parse tool output or draft payloads without pushback. Closed-model vendors have moved in the opposite direction, tightening policies on offensive use after high-profile misuse incidents. That divergence is the gap the benchmark tries to measure.

What the repository actually contains is still thin. The listing does not yet show a full scoring rubric, the exact roster of models tested, or the hardware used for runs, and the summary available to VintuxAI is limited to the link and its submission, so claims about which model wins cannot be independently verified. Buyers in this corner of the market currently lean on scattered forum posts and vendor demos rather than repeatable tests, which is the gap a comparison table would fill if it holds up.

Getting such a benchmark right is genuinely difficult. Security tasks range from reading a stack trace to chaining a multi-step web exploit, and a model that aces a multiple-choice quiz on CVEs may still flounder against a live target. Refusal rates also shift with prompt phrasing, so a single pass can mislead. Without a published methodology, results risk becoming marketing ammunition rather than engineering signal.

The next test is whether the author adds a transparent scoring method and reproducible commands, and whether the security community responds with its own runs. If the repo stays a link dump, it will fade like most weekend projects. If it grows into a maintained leaderboard, it could become the reference point for a market that model makers mostly prefer not to discuss in public.