This product was not featured by Product Hunt yet. It will not be visible on their landing page and won't be ranked (cannot win product of the day regardless of upvotes).
Plumb re-runs a paper's own code on your compute, re-derives each claim from the actual run, and returns a per-claim verdict: REPRODUCED, DIVERGED, or UNVERIFIED. Execution decides, never an LLM. Open source; the honest number is published with it.
Hi Product Hunt — I'm Aliz, and I built Plumb.
Every paper you read is a pile of numbers someone else's code printed. Peer review does not re-run that code. LLM judges guess whether the numbers look right, and measurably cannot do it (frontier models hit ~6% precision on errata-worthy errors). Plumb does the third thing: it re-runs the paper's own artifacts on your machine and re-derives each claim from what the run actually produced. Execution decides — never a model's opinion.
Try the 60-second demo (offline once cloned, no environment build, no API key):
git clone https://github.com/haqaliz/plumb && cd plumb
uv run plumb verify --from-record fixtures/gate/rcai
The honest number: across 3 real papers and 133 claims, precision and recall are 1/15 against owner labels. One republishable discrepancy out of fifteen flagged — because fourteen were the artifact's own run-to-run noise and were refuted on review. The corpus records the refutations instead of hiding the flags. A verifier reporting 15/15 there would be guessing.
What it does not do (yet): live mode needs a hand-written --bindings file; there is no binding proposer. Most papers ship no runnable code, and figure-only results are unreachable. When Plumb cannot decide, it says UNVERIFIED — it never dresses that up as REPRODUCED.
Point me at a paper with runnable code and I'll probe it during the launch.
No comment highlights available yet. Please check back later!
About plumb on Product Hunt
“Run the paper. Check its numbers.”
plumb was submitted on Product Hunt and earned 0 upvotes and 1 comments, placing #397 on the daily leaderboard. Plumb re-runs a paper's own code on your compute, re-derives each claim from the actual run, and returns a per-claim verdict: REPRODUCED, DIVERGED, or UNVERIFIED. Execution decides, never an LLM. Open source; the honest number is published with it.
plumb was featured in Open Source (69k followers), Developer Tools (521.5k followers), GitHub (41.4k followers) and Science (1.4k followers) on Product Hunt. Together, these topics include over 139.4k products, making this a competitive space to launch in.
Who hunted plumb?
plumb was hunted by Ali Haqiqi. A “hunter” on Product Hunt is the community member who submits a product to the platform — uploading the images, the link, and tagging the makers behind it. Hunters typically write the first comment explaining why a product is worth attention, and their followers are notified the moment they post. Around 79% of featured launches on Product Hunt are self-hunted by their makers, but a well-known hunter still acts as a signal of quality to the rest of the community. See the full all-time top hunters leaderboard to discover who is shaping the Product Hunt ecosystem.
Want to see how plumb stacked up against nearby launches in real time? Check out the live launch dashboard for upvote speed charts, proximity comparisons, and more analytics.
Ali Haqiqi