Quickstart

Plugins are the installable products in Claude, Codex, Copilot Cowork, and other agent harnesses. Skills are the task-specific capabilities inside them. Telvine records the plugin package and attaches execution telemetry to each observable component, including each SKILL.md capability, without flattening individual behavior. Use it to improve both the plugin product and the Skills it contains.

For Telvine-backed distribution and metrics, install the public npm package @telvine/cli@0.1.0:

npm i -g @telvine/cli@0.1.0
telvine login
telvine publish ./my-plugin --dry-run       # inspect manifests, components, Skills, evals
telvine publish ./my-plugin                 # register plugin, versions, evals, write key

telvine publish does three things:

1. Reads .codex-plugin/plugin.json or .claude-plugin/plugin.json, then records the plugin package and versioned component inventory.
2. Discovers skills/*/SKILL.md, computes version hashes, and links each Skill to the plugin version.
3. Imports evals/**/cases.jsonl as human-review eval suites, then mints a plugin-scoped write key (tel_wk_...). The key is printed once — store it somewhere safe.

For legacy single-Skill instrumentation, publish a Skill directory with --skill-id. See plugin components for the component inventory schema and event types.

Runtime installers that need a short-lived write key use POST /v1/runtime/write-key. This endpoint requires a Clerk session token or a read-scoped Telvine key for the same org/plugin; it will not mint keys from only a public plugin id or slug.

Record first-run installation

Marketplace hosts do not always send install webhooks. To avoid overstating adoption, Telvine dashboards show observed instances from events, not raw marketplace installs.

When your plugin first runs in a harness, persist an installation id and emit plugin.install once:

{
  "event_type": "plugin.install",
  "plugin_id": "plg_yourpluginid",
  "version": "0.1.0",
  "installation_id": "inst_generated_once_per_plugin_install",
  "occurred_at": "2026-05-28T12:00:00.000Z",
  "idempotency_key": "plugin-install-inst_generated_once_per_plugin_install-0.1.0",
  "runtime": "codex-app",
  "properties": {
    "via": "marketplace",
    "source": "marketplace",
    "install_context": "self_serve",
    "initial_version": "0.1.0"
  }
}

After that, emit skill.invocation.* for Skills and plugin.component.* for observable connectors, hooks, MCP config, commands, agents, and runtime components.

Add evals to the plugin repo

PM-authored evals should live with the plugin and Skill they judge:

my-plugin/
  skills/my-skill/SKILL.md
  evals/my-skill/eval.yaml
  evals/my-skill/cases.jsonl

Keep eval fixtures synthetic or approved for testing. Do not use production prompts, connector payloads, browser captures, screenshots, account values, or customer content. Treat evals as versioned evidence that complements production traces, outcomes, and feedback.

Agent experience feedback is enabled by default for eval suites. During an eval run, the harness can ask the AI agent using the plugin to score how usable the plugin felt from 1-5 and return a short metadata-only note. Turn the setting off on a suite when self-review would be noisy or inappropriate.

The Telvine Create Plugin workflow includes this as the default eval companion for new plugins: scaffold an agent-experience-reviewer Skill, or use the equivalent harness prompt when the runtime does not support Skills.

See events arriving

telvine plugins metrics <plugin-id> --since 1h
telvine plugins events <plugin-id> --limit 20

For a production rollout checklist, see Production Onboarding.