What the engine measures, and how it spots a duplicate
Six categories carry the score: structure and size, readability and complexity, duplication, dead code and leftovers, consistency and style, and repo hygiene. Every threshold is the production one, unchanged for the study, so a repo scanned here gets the number a paying customer would get.
Duplication is the category most likely to be measured wrong, so it's worth saying how ours works. The engine hashes 8-line normalized windows, which means whitespace and formatting differences don't hide a copy, and a renamed variable doesn't either. A hit is the same normalized window appearing twice. That's why cross-file duplication (46.4% of repos) reads as a separate figure from in-file repetition (41.2%): they're the same detector, split by whether the second copy lives in another file.
Scoring runs on the production model too: six weighted category subscores, severity deductions of 40 for critical, 22 for high, 10 for medium and 4 for low, then a weighted average and a letter grade.
Which repos got in, and which didn't
The population is repos whose own name, description, or README calls them AI- or vibe-coded. June 2026 turned up 107 of them; a second sweep in July took the candidate list to 577. Nothing was hand-picked after that. The engine dropped 28 (archive over 40MB compressed, or the repo wouldn't load), which leaves n = 549.
Taking a repo at its word is the whole selection method, and it's also the study's biggest weakness. Some of these are real applications. Others are conference demos and single-file toys: 85 have 15 or fewer scannable files and 29 have five or fewer, against a median repo of 69 files. Nothing was excluded on size, because a small vibe-coded repo is still vibe-coded output. It does mean a near-empty repo collects an A for having nothing in it to flag, which is why the report leads on means and the grade spread instead of the median.
Where the counting stops
Two caps apply: 10 findings per rule, 300 per repo. A repo with thirty oversized files reports ten of them. So the per-repo averages (mean 21.4 findings, median 6) understate the worst end of the distribution and should be read as floors. Hit rates dodge the problem entirely, since "did this repo have at least one X" doesn't care how many X it had, and that's why the headline figures are all hit rates.
Three more things to know before citing this. The engine reads code, it never runs it, and no tests were executed, so a good grade describes a tidy repo and says nothing about whether the software works. Provenance is unverified. And this is a single snapshot from July 2026; repos get tidied and abandoned, so a re-run in six months lands somewhere else. What survives all of that is solid: the hit rates, the bimodal grade distribution, and a per-repo result row behind every figure on the page.
how it works
- 01
Build the candidate list
Public GitHub search for repos self-describing as AI- or vibe-coded in name, description, or README. 107 found June 2026, 470 more in July, 577 total, no hand-picking.
- 02
Stream each archive through memory
Default-branch archive pulled by quality·vibes's own fetcher, nothing written to disk, 90 seconds per repo. 28 dropped as too large or unreachable, leaving 549.
- 03
Run production rules, unmodified
All six hygiene categories at production thresholds, including 8-line normalized-window duplication hashing. No Claude review, so the run reproduces exactly.
- 04
Score as a customer would be scored
Weighted category subscores, severity deductions (critical −40, high −22, medium −10, low −4), weighted average, letter grade.
- 05
Aggregate, then report the spread
Hit rates as share of repos with at least one finding, which the per-rule caps can't distort, alongside means, medians, and the full A-to-F distribution.
frequently asked
How does the engine know two blocks of code are duplicates?
It hashes 8-line normalized windows, so reformatting or renaming a variable doesn't disguise a copy. Two windows with the same hash are a duplication finding. The split between cross-file duplication (46.4%) and in-file repetition (41.2%) comes from one detector, divided by where the second copy sits.
Why run without the AI pass?
So the study can be checked. Rules-only output is deterministic, and anyone with the same repo list gets the same findings. It also biases the results low: the Claude review in a real quality·vibes scan adds findings on top of these, so treat every number as a minimum.
Didn't the tiny repos skew this?
They skewed the median, and we left them in and said so. 85 of the 549 have 15 or fewer files, and a repo that small scores A because there's nothing in it to catch. That's the source of the bimodal shape in the grade distribution, and the reason the report quotes means and hit rates first.
Will there be a second run?
Yes, on the same engine against a fresh list, since re-running costs us very little. July 2026 is version one. A second run only earns a write-up if the numbers actually move.
Last updated June 11, 2026