agents-cli

The CLI your coding agent uses to ship agents

Developer Tools

Artificial Intelligence

GitHub

Visit WebsiteSee on Product HuntGithub ⧉

Agents being the primary user of products is definitely the direction we are heading in. Curious to know how you do evals on the CLI? How do you validate if its suitable to be used by agents or run into the same confusion they might have had before?

the eval run step is the part that's usually missing from these agent-scaffolding tools. everyone ships the "build me an agent in an afternoon" demo but scoring it against a dataset before deploy is what actually tells you if it's ready, not just that it compiles and responds to a prompt

Agent-first output with a human interactive mode is the right call, and rarer than it should be. The subtlety I keep hitting is partial failure. A single top-level 'deploy: ok' is easy for a model to over-trust, so we ended up emitting per-resource state, this bucket created, that Cloud Run revision rolled back, because 'ok' was hiding that one of three resources failed and the agent moved on. Does the agent-mode output break status out per step and resource, or is it one overall result the model has to interpret?

What does a typical agent that one would build with this look like? Any common examples?

How does the evaluation step actually work in practice? Like, does it spin up a real signal in the cloud or do I need to wire up my own eval set before it can tell me if my agent is any good?

The gap between a slick demo and something that actually holds up in real use is where most of my side projects quietly die. Good to see real attention going into that messy middle bit, Elia.

The “CLI built for coding agents” framing feels right. The part I’d want to see is how the tool keeps the agent inside the intended change boundary.

When a scaffold/eval/deploy flow touches infra files, tests, Docker, telemetry, and CI, a coding agent can accidentally turn a product task into a repo-wide cleanup. A small plan/diff contract before each step — what files may change, what success signal matters, and how to roll back — would make me trust the loop much more.

Curious if agents-cli exposes that kind of step-level boundary, or if it’s mostly handled through the coding agent’s prompt/skill layer today?

The "agent-first output, human-friendly only in interactive mode" philosophy is the detail that makes this click for me, @elia_secchi — I spend a lot of time on agent reliability, and the number of loops that break because a tool emits prose an agent misreads as success is higher than anyone admits.

Loved your reply to Tyler too: wiring the secret reference but making a human add the actual secret is exactly the right seam to keep a coding agent from minting prod creds. Nice work 👌

Treating evals and deploy as first-class commands is the part that matters. A demo agent is easy; a shippable agent needs a repeatable failure loop, logs someone can read, and a human checkpoint before it touches production. That makes the CLI feel more like release engineering than scaffolding.

Congrats on the launch Elia! The "demo in an afternoon, production in weeks" framing is painfully accurate, and the detail that barely any of those weeks are the actual agent matches what I see too.

The headless part is the bit I'm most curious about: when Claude Code is driving the CLI end to end, how does auth work once the scaffolded agent needs real credentials for its tools and MCP servers? That's usually where the self-driving loop stalls and a human gets pulled back in. I live in that corner of the stack (launching an MCP thing myself today), so genuinely curious how you handle it.

To your question: I'd ship a changelog agent. It reads the PRs merged since the last tag, drafts release notes plus the announcement post, and files the result as a PR for review. Curious what an eval signal looks like for something that fuzzy, "good release notes" is a hard thing to score.

The 'driven by your coding agent' framing is the interesting bet here. When we let a coding agent run our deploy CLI unattended, the thing that bit us wasn't the deploy logic, it was output format. The agent would read a human-formatted stderr, miss that the deploy half-failed, and cheerfully report success. Are the CLI's outputs, eval scores, deploy status, errors, structured for a model to consume, like JSON with explicit exit states, or the same prose a person reads? That one choice decided whether our agent could actually close the loop.

the headless self-optimize loop is the part I'd want to understand before trusting it. if the coding agent is scoring its own agent against eval criteria and iterating unsupervised until the number goes up, what stops it from overfitting to the eval dataset or gaming the specific metric instead of actually improving behavior on real traffic? that's a known failure mode any time the thing being optimized also controls the optimization loop. do you have guardrails around eval set size/diversity or a human checkpoint before it ships to Cloud Run/GKE, or is it fully autonomous end to end