LoRA keeps the base model frozen: read, never written. So Soup keeps it in system RAM and streams it into the GPU one decoder layer at a time. Peak VRAM becomes one layer instead of the whole model. Measured on an RTX 3050 Laptop 4 GB: Llama-3.1-8B trains at 119.6 tok/s in 3.32 GB peak. One YAML, one command. SFT, DPO, GRPO, KTO, plus eval, gating and export. Apache-2.0. Every number is published, including the ones I measured and threw away.
I built Soup because I have a 4 GB laptop and wanted to fine-tune models that
do not fit in it.
The idea is simple. During LoRA the base model is frozen. It is read, never
written. So it does not have to live in the GPU, it only has to arrive before
the matmul that uses it. It sits in system RAM and streams in one decoder layer
at a time.
The hard part was not speed. It was proving it is correct. Streaming fails
silently: cut the autograd path and the loss still goes down, because the upper
layers keep learning. So every release compares a streamed run against a
resident one and requires the logits to match exactly.
Last week someone lent me 8 H100s for three days. That protocol found a bug in
my own released code: above a certain layer size the gradients are silently
wrong while the loss curve looks healthy. I published it, with a reproducer.
Everything is Apache-2.0 and every measurement is in the repo, including the
ones that turned out wrong.
Happy to answer anything.
About Soup CLI on Product Hunt
“Fine-tune an 8B LLM on a 4 GB laptop GPU”
Soup CLI launched on Product Hunt on August 9th, 2026 and earned 100 upvotes and 3 comments, placing #15 on the daily leaderboard. LoRA keeps the base model frozen: read, never written. So Soup keeps it in system RAM and streams it into the GPU one decoder layer at a time. Peak VRAM becomes one layer instead of the whole model. Measured on an RTX 3050 Laptop 4 GB: Llama-3.1-8B trains at 119.6 tok/s in 3.32 GB peak. One YAML, one command. SFT, DPO, GRPO, KTO, plus eval, gating and export. Apache-2.0. Every number is published, including the ones I measured and threw away.
On the analytics side, Soup CLI competes within Open Source, Developer Tools, Artificial Intelligence and GitHub — topics that collectively have 1.1M followers on Product Hunt. The dashboard above tracks how Soup CLI performed against the three products that launched closest to it on the same day.
Who hunted Soup CLI?
Soup CLI was hunted by Alpamys Makazhan. A “hunter” on Product Hunt is the community member who submits a product to the platform — uploading the images, the link, and tagging the makers behind it. Hunters typically write the first comment explaining why a product is worth attention, and their followers are notified the moment they post. Around 79% of featured launches on Product Hunt are self-hunted by their makers, but a well-known hunter still acts as a signal of quality to the rest of the community. See the full all-time top hunters leaderboard to discover who is shaping the Product Hunt ecosystem.
For a complete overview of Soup CLI including community comment highlights and product details, visit the product overview.
Alpamys Makazhan