Architectural best practice: Prefer remote MCP tools
When designing skills, our guiding principle is: Reference remote Model Context Protocol (MCP) tools whenever possible, falling back to CLI or API calls only when necessary. Remote MCP servers are best suited for Agentic workloads by providing tools, while also offering built-in auth and IAM governance.
Public export
We build and evaluate our skills internally first to make sure they work and are properly validated. Once ready to go public, we use automated export rules to publish to GitHub. This keeps public repos clean while stripping out internal assets, ownership information, and evaluation suites.
Automated checks on check-in
Before any skill enters the repository, it must pass an automated CI/CD pipeline:
-
Linters: We validate frontmatter metadata, line counts, directory layout, and strict naming conventions.
-
Link Checkers: We test every URL using link-checking tools to eliminate 404s and hallucinated links before merge.
-
AI-Assisted Checklists: We use automated validation checks to verify that instructions follow required structural patterns and guardrails.
Continuous evals (on submit & weekly)
Documentation and APIs evolve, and so do LLM models and agent harnesses. A skill that works today might break tomorrow if an underlying API, model, or agent harness changes.
To set an initial quality bar and prevent degradation, we run continuous evaluations:
-
On-submit evaluations: Authors must provide explicit evaluation prompt suites and scoring rubrics. Every new skill that we launch is first evaluated internally to ensure its accuracy and efficiency.
-
Weekly quality checks: We run continuous, scheduled evaluation jobs against the full skill library to catch regressions early.
Skill authors must supply multiple evaluation test cases, each containing a prompt and a set of expectations. With each evaluation suite, we compare the performance of agents with and without each skill.
And look at two main dimensions:
-
Accuracy – response quality and task completion rate
-
Efficiency – number of consumed tokens and time for completion
Moreover we run our evals multiple times against different agent frameworks to obtain statistically significant results.
Finally the 2×2 matrix proves whether a skill delivers a measurable accuracy and efficiency uplift.
