Skill Creator: write a skill, then prove it beats no skill
Writing a SKILL.md is the easy part; knowing whether it actually improves the agent is harder. Skill Creator walks you through the loop: it pins down the intent, drafts the skill, runs test prompts with and without it, grades the outputs and opens a review page where you leave notes for the next round.
Turns a workflow you already did in chat into a skill, pulling the tools, steps and your corrections from the conversation.
Spawns paired subagent runs for every test prompt, one with the skill and one baseline, launched together so they finish together.
Aggregates results with scripts.aggregate_benchmark into pass rate, time and tokens, each with mean and standard deviation.
Opens eval-viewer/generate_review.py with an Outputs tab for per-case feedback and a Benchmark tab for the numbers.
Ships a separate description improver that rewrites the skill's description so it triggers on the right requests more reliably.
When to use it
You keep pasting the same instructions into every session and want them captured once as a reusable skill.
An existing skill fires only some of the time and its description needs rewriting around real user phrasing.
Before sharing a skill with your team you want evidence that outputs improve compared with running without it.
Ask your agent
Turn what we just did into a skill for writing release notes
Run my invoice-check skill on three test prompts and compare against a no-skill baseline
My skill doesn't trigger when I say "reconcile invoices", so fix its description
Questions
How do I install Skill Creator?
Run npx skills add anthropics/skills@skill-creator. In Claude Code you can also add the repo as a plugin marketplace with /plugin marketplace add anthropics/skills.
Do I have to write evals to use Skill Creator?
No. It suggests test cases for skills with verifiable output, such as file transforms or data extraction, but for subjective skills like writing style you can skip them and iterate on feedback.
Does Skill Creator work in a headless environment?
Yes. The review viewer has a --static option that writes a standalone HTML file instead of starting a server, and feedback then downloads as feedback.json.
