This product was not featured by Product Hunt yet. It will not be visible on their landing page and won't be ranked (cannot win product of the day regardless of upvotes).
Product upvotes vs the next 3
Waiting for data. Loading
Product comments vs the next 3
Waiting for data. Loading
Product upvote speed vs the next 3
Waiting for data. Loading
Product upvotes and comments
Waiting for data. Loading
Product vs the next 3
Loading
Veil
A web scraper that uses the lightest method that works
Veil is an open-source Python web scraper that tries the cheapest method first and only escalates when a site blocks it so you're not running a full browser for every page. • 3 tiers: plain HTTP → TLS impersonation → stealth browser • Resumable concurrent crawler (checkpoint, JSONL, dedupe) • Proxy rotation, rate limiting, robots.txt by default • Honest README: documents what it can't beat Python, MIT, tested. A learning project built in the open.
hey PH 👋
i'm still pretty new to python and built this mostly to learn, so honestly any
feedback helps.
the idea: most scrapers either send a plain http request and get blocked, or spin
up a whole headless browser for every page (slow + expensive). veil tries the cheap
stuff first and only moves up when it actually gets blocked:
plain http → TLS fingerprint impersonation → stealth browser
it's also got a crawler that resumes if it crashes, rotates proxies, and respects
robots.txt by default.
the part i'm actually proud of: i tested it against real sites and wrote down what
it CAN'T do. it gets past JS challenges, but if a site throws a 403 before any
javascript runs, that's an IP reputation thing and no code fixes it — you just need
residential proxies. that's in the readme, i didn't wanna pretend it beats
everything.
it's open source (MIT), and there's a CONTRIBUTING guide + good-first-issues if
anyone wants to build with me. would love to hear what i got wrong 🙏
About Veil on Product Hunt
“A web scraper that uses the lightest method that works”
Veil was submitted on Product Hunt and earned 4 upvotes and 1 comments, placing #153 on the daily leaderboard. Veil is an open-source Python web scraper that tries the cheapest method first and only escalates when a site blocks it so you're not running a full browser for every page. • 3 tiers: plain HTTP → TLS impersonation → stealth browser • Resumable concurrent crawler (checkpoint, JSONL, dedupe) • Proxy rotation, rate limiting, robots.txt by default • Honest README: documents what it can't beat Python, MIT, tested. A learning project built in the open.
On the analytics side, Veil competes within Open Source, Developer Tools and GitHub — topics that collectively have 625.3k followers on Product Hunt. The dashboard above tracks how Veil performed against the three products that launched closest to it on the same day.
Who hunted Veil?
Veil was hunted by Aditya Panda. A “hunter” on Product Hunt is the community member who submits a product to the platform — uploading the images, the link, and tagging the makers behind it. Hunters typically write the first comment explaining why a product is worth attention, and their followers are notified the moment they post. Around 79% of featured launches on Product Hunt are self-hunted by their makers, but a well-known hunter still acts as a signal of quality to the rest of the community. See the full all-time top hunters leaderboard to discover who is shaping the Product Hunt ecosystem.
For a complete overview of Veil including community comment highlights and product details, visit the product overview.
Aditya Panda