LoRA keeps the base model frozen: read, never written. So Soup keeps it in system RAM and streams it into the GPU one decoder layer at a time. Peak VRAM becomes one layer instead of the whole model. Measured on an RTX 3050 Laptop 4 GB: Llama-3.1-8B trains at 119.6 tok/s in 3.32 GB peak. One YAML, one command. SFT, DPO, GRPO, KTO, plus eval, gating and export. Apache-2.0. Every number is published, including the ones I measured and threw away.
I built Soup because I have a 4 GB laptop and wanted to fine-tune models that
do not fit in it.
The idea is simple. During LoRA the base model is frozen. It is read, never
written. So it does not have to live in the GPU, it only has to arrive before
the matmul that uses it. It sits in system RAM and streams in one decoder layer
at a time.
The hard part was not speed. It was proving it is correct. Streaming fails
silently: cut the autograd path and the loss still goes down, because the upper
layers keep learning. So every release compares a streamed run against a
resident one and requires the logits to match exactly.
Last week someone lent me 8 H100s for three days. That protocol found a bug in
my own released code: above a certain layer size the gradients are silently
wrong while the loss curve looks healthy. I published it, with a reproducer.
Everything is Apache-2.0 and every measurement is in the repo, including the
ones that turned out wrong.
Happy to answer anything.
Publishing the ones that turned out wrong is the detail that stands out. Ran into a smaller version of that this month, checked AI Overview traffic on three real sites expecting some signal and got zero across the board, and the honest move was publishing zero instead of only writing up the wins. Question on the streaming approach: does the correctness check, streamed logits matching a resident run, hold up against an already 4-bit quantized base model, or is that validated against full precision only so far?
@makazhanalpamys The layer-streaming idea is neat, but the correctness protocol is the part that won me over — requiring streamed-run logits to exactly match a resident run, because "cut the autograd path and the loss still goes down" is exactly the kind of silent failure most tools never check for.
And publishing the H100-found bug (gradients silently wrong above a layer size while the loss curve looks healthy) with a reproducer, in your own released code, is rarer than the optimization itself. That's how benchmarks earn trust.
119.6 tok/s in 3.32 GB on a 3050 laptop is a genuinely useful floor for people who want to iterate locally before paying for cloud GPUs 👌
About Soup CLI on Product Hunt
“Fine-tune an 8B LLM on a 4 GB laptop GPU”
Soup CLI launched on Product Hunt on August 9th, 2026 and earned 100 upvotes and 3 comments, placing #15 on the daily leaderboard. LoRA keeps the base model frozen: read, never written. So Soup keeps it in system RAM and streams it into the GPU one decoder layer at a time. Peak VRAM becomes one layer instead of the whole model. Measured on an RTX 3050 Laptop 4 GB: Llama-3.1-8B trains at 119.6 tok/s in 3.32 GB peak. One YAML, one command. SFT, DPO, GRPO, KTO, plus eval, gating and export. Apache-2.0. Every number is published, including the ones I measured and threw away.
Soup CLI was featured in Open Source (68.7k followers), Developer Tools (517.4k followers), Artificial Intelligence (475.9k followers) and GitHub (41.4k followers) on Product Hunt. Together, these topics include over 233.7k products, making this a competitive space to launch in.
Who hunted Soup CLI?
Soup CLI was hunted by Alpamys Makazhan. A “hunter” on Product Hunt is the community member who submits a product to the platform — uploading the images, the link, and tagging the makers behind it. Hunters typically write the first comment explaining why a product is worth attention, and their followers are notified the moment they post. Around 79% of featured launches on Product Hunt are self-hunted by their makers, but a well-known hunter still acts as a signal of quality to the rest of the community. See the full all-time top hunters leaderboard to discover who is shaping the Product Hunt ecosystem.
Want to see how Soup CLI stacked up against nearby launches in real time? Check out the live launch dashboard for upvote speed charts, proximity comparisons, and more analytics.
Alpamys Makazhan