NiceTryGPT

Make CTFs harder to shortcut with LLMs, not humans

Open Source

Developer Tools

GitHub

Security

Visit WebsiteSee on Product HuntGithub ⧉

Hey Product Hunt! 👋 I built NiceTryGPT after noticing how easily some CTF challenges can be shortcut by an LLM simply because the intended vulnerability is exposed through very recognizable patterns. The idea behind the project is simple: **Increase uncertainty, not complexity.** NiceTryGPT is an open-source Agent Skill for CTF authors. It first solves an existing authorized challenge end-to-end, identifies the cheapest LLM shortcut, then applies the smallest useful transformation and verifies the challenge again. The important constraint is that the human experience should remain essentially the same. So NiceTryGPT tries to preserve: • the same vulnerability class • the same learning objective • the same prerequisite knowledge • the same success/flag semantics • roughly the same human difficulty The current v0.2.0 includes small working examples for IDOR, path traversal and SQL injection, together with end-to-end tests. I'm deliberately not claiming that this makes a challenge “AI-proof.” The goal is narrower: reduce shallow pattern-matching shortcuts while keeping CTFs fair and educational for humans. The project is open source, and the next major milestone is independent multi-model evaluation. I'd especially love feedback from CTF authors, players and security researchers: **What kinds of CTF shortcuts have you seen LLMs exploit too easily, and which transformations would still feel fair to a human player?** Thanks for checking it out! ☕🤖