This product was not featured by Product Hunt yet.
It will not be visible on their landing page and won't be ranked (cannot win product of the day regardless of upvotes).
Product upvotes vs the next 3
Product comments vs the next 3
Product upvote speed vs the next 3
Product upvotes and comments
Product vs the next 3
plumb
Run the paper. Check its numbers.
Plumb re-runs a paper's own code on your compute, re-derives each claim from the actual run, and returns a per-claim verdict: REPRODUCED, DIVERGED, or UNVERIFIED. Execution decides, never an LLM. Open source; the honest number is published with it.
Top comment
Hi Product Hunt — I'm Aliz, and I built Plumb. Every paper you read is a pile of numbers someone else's code printed. Peer review does not re-run that code. LLM judges guess whether the numbers look right, and measurably cannot do it (frontier models hit ~6% precision on errata-worthy errors). Plumb does the third thing: it re-runs the paper's own artifacts on your machine and re-derives each claim from what the run actually produced. Execution decides — never a model's opinion. Try the 60-second demo (offline once cloned, no environment build, no API key): git clone https://github.com/haqaliz/plumb && cd plumb uv run plumb verify --from-record fixtures/gate/rcai The honest number: across 3 real papers and 133 claims, precision and recall are 1/15 against owner labels. One republishable discrepancy out of fifteen flagged — because fourteen were the artifact's own run-to-run noise and were refuted on review. The corpus records the refutations instead of hiding the flags. A verifier reporting 15/15 there would be guessing. What it does not do (yet): live mode needs a hand-written --bindings file; there is no binding proposer. Most papers ship no runnable code, and figure-only results are unreachable. When Plumb cannot decide, it says UNVERIFIED — it never dresses that up as REPRODUCED. Point me at a paper with runnable code and I'll probe it during the launch.
About plumb on Product Hunt
“Run the paper. Check its numbers.”
plumb was submitted on Product Hunt and earned 0 upvotes and 1 comments, placing #397 on the daily leaderboard. Plumb re-runs a paper's own code on your compute, re-derives each claim from the actual run, and returns a per-claim verdict: REPRODUCED, DIVERGED, or UNVERIFIED. Execution decides, never an LLM. Open source; the honest number is published with it.
On the analytics side, plumb competes within Open Source, Developer Tools, GitHub and Science — topics that collectively have 633.4k followers on Product Hunt. The dashboard above tracks how plumb performed against the three products that launched closest to it on the same day.
Who hunted plumb?
plumb was hunted by Ali Haqiqi. A “hunter” on Product Hunt is the community member who submits a product to the platform — uploading the images, the link, and tagging the makers behind it. Hunters typically write the first comment explaining why a product is worth attention, and their followers are notified the moment they post. Around 79% of featured launches on Product Hunt are self-hunted by their makers, but a well-known hunter still acts as a signal of quality to the rest of the community. See the full all-time top hunters leaderboard to discover who is shaping the Product Hunt ecosystem.
For a complete overview of plumb including community comment highlights and product details, visit the product overview.