oMLX turns your Mac into a full LLM inference server, run from the menu bar. It serves text, vision, OCR, embedding and reranker models with continuous batching, plus a RAM+SSD tiered KV cache that survives restarts, so Claude Code and Cursor respond in about 5s instead of 90s. OpenAI and Anthropic compatible APIs drop straight in. Native Swift, not Electron. Apache 2.0, open source.
Kept hitting the same wall with local models: the agent loops back, the whole conversation recomputes, 90 seconds of nothing.
oMLX writes the KV cache to SSD. Old context comes back in milliseconds, even after a restart. Claude Code on a local model stops feeling like dial-up.
Jun has been shipping this almost daily since February. 21k stars, and his Show HN still barely got seen. Felt wrong.
If you run models on a Mac: what does your stack look like? Curious what people pair this with.
The number that would sell me isn't 90 to 5, it's what happens when the cache is wrong. A KV cache that survives a restart also survives me swapping the model or editing the system prompt, and a stale prefix doesn't crash, it just answers a slightly different question than the one on screen. If the cache key includes the model hash and the full prefix, put that on the page, because that's the thing a dev has to trust before leaving it running for a week. Tiered RAM plus SSD is the right shape though, most local setups throw the whole thing away and pretend prefill is free.
my stack right now is just LM Studio for casual local chat, which never bothered me because that's a one-off question and answer. the thing that actually annoyed me was agent coding loops - watching Claude Code re-read the whole conversation on every follow-up turn. that's a different use case than most "run a model locally" tools are built for, so the KV cache surviving a restart is the part that's actually interesting here, more than raw tokens/sec.
Recently got my Mac that has enough RAM to run local LLMs and honestly – oMLX was the best solution so far to optimize for context and speed at the same time on Apple silicon. Also huge shoutout to the devs, who are constantly shipping updates at a crazy pace. Love to see it on PH, thanks for hunting!
Cache surviving a restart is the detail I'd have skipped and then regretted. How much SSD does the tiered cache use in practice?
About oMLX on Product Hunt
“Mac LLM server that cuts agent wait times from 90s to 5s”
oMLX launched on Product Hunt on August 30th, 2026 and earned 89 upvotes and 17 comments, placing #13 on the daily leaderboard. oMLX turns your Mac into a full LLM inference server, run from the menu bar. It serves text, vision, OCR, embedding and reranker models with continuous batching, plus a RAM+SSD tiered KV cache that survives restarts, so Claude Code and Cursor respond in about 5s instead of 90s. OpenAI and Anthropic compatible APIs drop straight in. Native Swift, not Electron. Apache 2.0, open source.
oMLX was featured in Open Source (68.8k followers), Developer Tools (518.4k followers), Artificial Intelligence (477.3k followers) and GitHub (41.4k followers) on Product Hunt. Together, these topics include over 241.6k products, making this a competitive space to launch in.
Who hunted oMLX?
oMLX was hunted by Rabnoor Singh. A “hunter” on Product Hunt is the community member who submits a product to the platform — uploading the images, the link, and tagging the makers behind it. Hunters typically write the first comment explaining why a product is worth attention, and their followers are notified the moment they post. Around 79% of featured launches on Product Hunt are self-hunted by their makers, but a well-known hunter still acts as a signal of quality to the rest of the community. See the full all-time top hunters leaderboard to discover who is shaping the Product Hunt ecosystem.
Want to see how oMLX stacked up against nearby launches in real time? Check out the live launch dashboard for upvote speed charts, proximity comparisons, and more analytics.
Rabnoor Singh