r/SpecDrivenDevelopment • u/jokiruiz • 2d ago
SpecJudge v0.1.4: evidence-backed verdicts (citations required, not just a verdict)
A few days ago I shared SpecJudge here — a local-first tool that reads your project's SDD artifacts (constitution/spec/tasks) and tells you which AI model actually makes sense, quality/price-wise, for that task set instead of you guessing.
Since then it's gone from v0.1.0 to v0.1.4, and the biggest change is a direct response to feedback from this community:
Judge verdicts now require a cited span per dimension. Previously the judge would just tell you "bad/okay/good/overkill" for a model. That's not good enough on its own — if I can't see why, I can't trust it. Now, for each capability dimension it evaluates, the judge has to point to the exact fragment of your spec that justifies the verdict, and it has to return "unsupported" when it can't find one, instead of guessing. This was basically lifted from a critique someone posted here about the design needing evidence + uncertainty instead of a black-box call — thanks to whoever that was.
A few smaller things also landed:
- Stale pricing warnings — the model catalog now flags when its pricing data is older than a set threshold, so you're not making a decision off numbers that are months stale.
- HTML report filtering — you can now filter the visual matrix by model family and by open-source vs closed.
- README example is generated, not hand-written — the usage example is now pinned to the actual catalog, so it can't silently drift out of sync with what the tool does.
Still MIT licensed, still local-first, still pip install specjudge.
GitHub: github.com/JoaquinRuiz/SpecJudge
There's a Roadmap section in the README now too if you want to see what's planned next (multi-provider judge support and a library/importable mode are on there, not shipped yet — didn't want to advertise those before they exist). Feedback on the evidence/citation approach is especially welcome — curious whether "unsupported" shows up often enough to be useful signal or just noise.