Event schema

Skill events use skill_id for capability drill-downs and may carry plugin_id for product roll-ups. Package-level and non-Skill component events require plugin_id and do not invent a synthetic Skill.

{
  "event_type": "skill.invocation.end",
  "plugin_id": "plg_product123",
  "skill_id": "skl_abc123",
  "version": "1.2.0",
  "installation_id": "inst_xyz789",
  "occurred_at": "2026-05-20T12:34:56.789Z",
  "idempotency_key": "uuid-v7-generated-client-side",
  "runtime": "claude-code",
  "runtime_version": "2.0.40",
  "properties": { "duration_ms": 4321, "tool_calls_count": 7, "trigger": "explicit" }
}

properties is a closed set of known keys per event type, validated at the API boundary. No user content, no tool arguments, no absolute paths, and no paths to user files. Plugin component events may include package-relative component_path values, such as skills/reconcile-invoices/SKILL.md, so Telvine can measure plugin directories without collecting workspace contents.

event_type	properties
`plugin.install`	`via`, `source?`, `install_context?`, `initial_version?`
`plugin.update.applied`	`from_version`, `to_version`, `update_source?`
`plugin.component.invoked`	`component_type`, `component_name`, `operation`, `component_path?`, `component_directory?`, `manifest_field?`, `duration_ms?`, `outcome?`
`plugin.component.error`	`component_type`, `component_name`, `operation?`, `component_path?`, `component_directory?`, `manifest_field?`, `error_class`, `blocker_reason?`
`skill.install`	`via`
`skill.invocation.start`	`trigger`
`skill.invocation.end`	`duration_ms`, `tool_calls_count`, `trigger?`
`skill.invocation.error`	`error_class`
`skill.script.invoked`	`script_name`, `exit_code`, `duration_ms`
`skill.reference.loaded`	`reference_name`
`skill.modified`	`frontmatter_changed`, `body_changed`
`feedback.submitted`	`rating?`, `comment?`, `source?` (`user`, `reviewer`, or `agent`)
`skill.eval.suite.published`	`eval_suite_id`, `eval_suite_slug`, `eval_suite_version`, `case_count`, `pass_threshold?`
`skill.eval.run.started`	`eval_suite_id`, `eval_run_id`, `target_version?`, `baseline_version?`, `harness?`, `model?`
`skill.eval.case.result`	`eval_suite_id`, `eval_run_id`, `eval_case_id`, `case_key?`, `status`, `score?`, `failure_cluster?`, `error_class?`, `duration_ms?`, `agent_experience_score?`, `agent_experience_feedback?`
`skill.eval.run.completed`	`eval_suite_id`, `eval_run_id`, `status`, `pass_rate?`, `score?`, `passed_count?`, `failed_count?`, `skipped_count?`, `errored_count?`
`plugin.eval.suite.published`	`eval_suite_id`, `eval_suite_slug`, `eval_suite_version`, `case_count`, `pass_threshold?`
`plugin.eval.run.started`	`eval_suite_id`, `eval_run_id`, `target_version?`, `baseline_version?`, `harness?`, `model?`
`plugin.eval.case.result`	`eval_suite_id`, `eval_run_id`, `eval_case_id`, `case_key?`, `status`, `score?`, `failure_cluster?`, `error_class?`, `duration_ms?`, `agent_experience_score?`, `agent_experience_feedback?`
`plugin.eval.run.completed`	`eval_suite_id`, `eval_run_id`, `status`, `pass_rate?`, `score?`, `passed_count?`, `failed_count?`, `skipped_count?`, `errored_count?`

Eval events describe cases, runs, scores, and failure clusters. They must not include prompts, model outputs, connector payloads, file contents, or raw tool arguments. Store detailed evidence in your own harness and send Telvine only a safe evidence_ref through the eval run API when needed.

Eval suites request agent experience feedback by default. A harness can ask the AI agent using the plugin to rate usability from 1-5 and provide a short metadata-only note, then submit agent_experience_score and agent_experience_feedback with the eval case result. If the same signal is sent as general feedback, set source to agent on feedback.submitted.

If you want to emit something exotic, open an issue — we'd rather add it to the schema than have you stuff things into ad-hoc keys.

Install events and first-run telemetry

Most agent harnesses do not expose reliable marketplace-install webhooks yet. Do not assume that adding a plugin marketplace or clicking install will appear in Telvine automatically.

For Git marketplace installs, use this practical pattern:

Persist a plugin-local installation id the first time the plugin runs.
Emit plugin.install once for that installation id.
Emit skill.invocation.* for SKILL.md capabilities and plugin.component.* for observable non-Skill behavior after that.

The dashboard treats unverified marketplace installs as unobserved until an event arrives. If plugin.install is unavailable, the first committed runtime event still creates an observed installation cohort, but it is not proof that the marketplace install itself occurred.

Canonical plugin component types are manifest, skill, command, agent, sub_agent, hook, mcp_server, mcp_config, connector, tool, app, channel, model_provider, harness_adapter, runtime_component, eval_suite, context_engine, memory_provider, lsp_server, monitor, executable, script, settings, user_config, dependency, directory, asset, theme, and output_style.

Canonical component directories include root, .claude-plugin, skills, commands, agents, hooks, mcp, lsp, output-styles, themes, monitors, bin, scripts, settings, connectors, tools, apps, assets, integrations, providers, memory, context, adapters, evals, custom, and inline.

Canonical manifest fields include Claude-compatible manifest and package fields such as skills, commands, agents, hooks, mcpServers, outputStyles, lspServers, experimental.themes, experimental.monitors, userConfig, channels, dependencies, settings, bin, and scripts, plus cross-harness fields such as connectors, tools, apps, assets, memoryProviders, contextEngines, modelProviders, harnessAdapters, evals, and runtimeComponents.

Component operations are closed too: use invoked, executed, tool_called, connected, configured, discovered, validated, loaded, registered, scanned, resolved, cached, installed, started, completed, enabled, or disabled.

Event schema

Install events and first-run telemetry

On this page