Science Explained/Brief

Clinical calculator AI: code solver lifts 32B model, not 7B, preprint finds

A preprint tests whether language models do clinical arithmetic better by writing Python for a restricted executor to run instead of calculating directly. It reports a gain for a 32B model but not a 7B one, and says verified formulas remain essential.

BriefPublished 12 September 20261 min read1 linked source · 5 checked factsRevision 5

Correction note. Source material changed; article revised (revision 2). Previous headline: Program-Solve interface helps 32B clinical model but not 7B in preprint benchmark

dishes of a radio telescope array at night — Illustration: National Radio Astronomy Observatory at night - panoramio · Diarrhea · CC BY 3.0 licence

A preprint posted to arXiv on 12 September tests whether language models handle clinical arithmetic better when they write case-specific Python for a restricted executor to run rather than calculating directly. The authors evaluated the approach on MedCalc-Bench Verified, a set of 1,100 cases spanning 55 calculators.

With formulas and variables supplied, the solver lifted Qwen2.5-32B-AWQ from 83.47% to 90.53%, a paired gain of 7.05 points with a 95% interval of [0.47, 14.60]. For Qwen2.5-7B the gain was 3.29 points, from 72.02% to 75.31%, with an interval of [-3.49, 10.38] that includes zero.

A hand-written 22-calculator library was exact on its 440 supported cases but abstained elsewhere, scoring 40.0% overall. The authors audited the benchmark's formulas against current clinical guidelines and flagged 16 of 55 for version, use or coefficient concerns.

What the reporting says

Each line below is stated by the page named above it, and only by that page.

Stated by arXiv

The study evaluates a Program-Solve interface on MedCalc-Bench Verified, which contains 1,100 cases and 55 calculators.
At 32B, the solver achieved 90.53% versus 83.47% for direct arithmetic, a paired +7.05 points with a 95% interval of [0.47, 14.60].
At 7B, the solver achieved 75.31% versus 72.02%, a paired +3.29 points with a 95% calculator-cluster interval of [-3.49, 10.38].
The hand-written 22-calculator library was exact on its 440 supported cases but abstained elsewhere, yielding 40.0% overall.
The benchmark's formulas were audited against current clinical guidelines, and 16 of 55 were flagged for version, use or coefficient concerns.

Our view

This is preliminary evidence that program-solvers may help larger clinical language models, but with only two models tested and 16 benchmark formulas flagged, the finding is far from settled.

What the reporting says: The preprint reports that handing off to the solver is not a reliable advantage at 7B but is one at 32B, and is not a substitute for verified formulas or reliable variable extraction.

Sources

arXivPublished 12 September 2026 · checked 14 September 2026

Found through monitored feeds: arXiv. A feed address is how the desk finds a story, not the evidence for it — the sources listed above are the pages the facts were taken from.

How this story was checked

This story was written from the 1 page listed above, all read on 14 September 2026.
Every figure and identifier in the text was required to appear in that source text before publication; any that did not was removed rather than checked by memory.
Of the 5 reported statements recorded for this story, 5 are each traceable to a linked page and are shown above. The rest were dropped.
Quantities traced to the source text: yes; identifiers traced: yes.
Where the reporting was silent, the text says so instead of filling the gap. This is a reported story, not a review of the source.

Clinical calculator AI: code solver lifts 32B model, not 7B, preprint finds

What the reporting says

Our view

Sources

Related reading

More from Science Explained

Preprint proposes intent-driven privacy filtering for LLM prompts

Preprint proposes audit protocol for AI update admission, tested only in simulation

Probabilistic Focal Search preprint cuts node expansions when FOCAL admission stalls

How this story was checked

What the reporting says

Our view

Sources

Related reading

Program-solve interface improves clinical math for 32B model, not 7B

More from Science Explained

Preprint proposes intent-driven privacy filtering for LLM prompts

Preprint proposes audit protocol for AI update admission, tested only in simulation

Probabilistic Focal Search preprint cuts node expansions when FOCAL admission stalls

How this story was checked