Practical AI/Brief
Judge-Guided Revision Improves AI Patent Drafts, Narrows Gap Between Cheap and Expensive Models
A new arXiv paper evaluates an LLM judge for patent drafting. It finds that iterative judge feedback improves judge-assessed quality and enables a low-reasoning agent to approach a more expensive high-reasoning agent.
BriefPublished 15 September 20261 min read1 linked source · 4 checked facts
The paper introduces Vibe Patenting, an end-to-end testbed where a separately invoked LLM judge evaluates generated patent drafts and provides structured feedback for revision. Across multiple inventions and drafting-agent configurations, judge-guided revision consistently improved judge-assessed quality, while unguided revision tended to saturate.
The useful detail: iterative judge feedback allowed a low-reasoning agent to approach the performance of a substantially more expensive high-reasoning agent. Stronger models and increased reasoning generally improved quality, and domain-specific agentic workflows added further gains. However, validation against a professional patent attorney found meaningful but strongly metric-dependent agreement and systematic calibration differences.
Our view
This points to a practical way to get better patent drafts from cheaper models, though the judge's reliability depends on the metric and differs from a human attorney's evaluation.
What the reporting says: Judge-guided revision consistently improves judge-assessed quality, while unguided revision tends to saturate, and that iterative judge feedback enables a low-reasoning agent to approach the performance of a substantially more expensive high-reasoning agent.