Start free

Eval-and-improve for agent plugins and Skills

Make agent plugins and Skills measurably better

Telvine is the eval-and-improve control plane for agent plugins and Skills. Bring your own harness and models; Telvine turns production traces, outcomes, feedback, evals, and promotion gates into better plugin and Skill releases.

Start freeRead the docs →
npm i -g @telvine/cli@0.1.0
telvine login
telvine init
telvine publish ./my-plugin

A live look

One plugin and Skill improvement loop, every harness, measured end to end

Telvine keeps the plugin and Skill model stable while each harness gets the package shape it expects: Codex, Microsoft 365 Copilot Cowork, Claude, and whatever comes next.

Versioning and promotion

Promote the plugin version with the strongest output evidence

Plugin owners need more than install counts. Telvine keeps every plugin release, Skill version, component inventory, eval suite, production outcome, and feedback signal tied together so teams can prove which version is ready for users.

Compare a candidate against stable, inspect regressions, and promote only when the version clears the gates that matter: eval pass rate, output quality, error rate, latency, and real workflow completion.

stablev1.4.0current customer release
candidatev1.5.0ready to promote
Eval pass rate94%passes gate
Output rating4.54+21%
Error rate2.3%-81%
Workflow completion88%+14%
✓Promote to stableEvidence beats guesswork before the new plugin output reaches customers.

Use your current tools

Bring your own harness and models. Keep your analytics stack.

SaaS teams should not rebuild the same plugin and Skill separately for Codex, Copilot Cowork, Claude, Cursor, and the next agent harness. Telvine gives teams one plugin and Skill workspace, then maps that product into the package shape each platform expects.

Your existing analytics tools still matter. Telvine produces plugin-native and Skill-native traces, outcome events, feedback, eval signals, and version evidence, then forwards clean telemetry to PostHog, Mixpanel, Amplitude, Datadog, warehouses, and internal dashboards.

Agent harnesses are the new operating systems. Plugins are the apps.

Every computing wave reorganizes work around a new operating system, and a new unit of software runs on top of it. Mainframes had jobs. PCs had applications. Phones had apps. The browser had web apps.

The agent harness - the runtime that wraps a model with its loop, tools, and context, like Claude Cowork, Claude Code, or Codex - is the next operating system. The product distributed on it is the Agent Plugin: an installable package of Skills, tools, connectors, hooks, MCP servers, and other capabilities that does real work.

Every mature software ecosystem eventually grows observability, evals, versioning, and release control. Telvine is that layer for agent plugins and Skills: the place you define the product and its capabilities once, ship them across harnesses, measure what happened in production, run evals, and improve each version with evidence.

What is an Agent Plugin?

An Agent Plugin is the installable package users add to an agent harness. A Plugin may contain one or more Skills, where each SKILL.md describes a capability an agent loads on demand, alongside tools, connectors, hooks or other resources.

Plugins are the app-level product. Skills and components are the execution-level surfaces. Telvine keeps the app-level product consistent across harnesses, then measures each Skill, attaches eval evidence, and forwards clean events wherever you already work.

Everything you need to improve plugins and Skills across harnesses

Observe production plugin and Skill runs

Capture metadata-only traces, outcomes, latency, component usage, and feedback across Codex, Claude, Copilot Cowork, and custom harnesses.

Evaluate plugin and Skill quality

Keep PM-defined eval scenarios, rubrics, pass thresholds, and promotion gates attached to the Skill versions they are meant to judge.

Version, compare, and promote releases

Compare candidates against stable versions on eval pass rate, output outcomes, error rate, latency, and feedback before promotion.

Keep your stack

Forward normalized events to PostHog, Mixpanel, Amplitude, Datadog, your warehouse, or internal dashboards while Telvine manages the plugin and Skill improvement record.

Who Telvine is for

SaaS teams shipping MCP servers and agent plugins who need to prove their plugins and Skills work across many agent surfaces. Platform and AI teams standardizing packaging, rollout, telemetry, evals, and promotion gates across an org. Agencies and consultancies proving whether the plugins and Skills they build actually improve in production.

Pricing

CLI included. Pay for the plugin and Skill improvement layer.

Plans scale with tracked plugin packages, observable Skills and components, metadata event ingestion, eval and version-comparison workflows, dashboard collaborators, and governance requirements. The CLI is free to install and run in agents, CI, and local development; the paid surface is the hosted workspace where teams measure, compare, and improve plugin versions.

Free

For trying Telvine on one plugin.

  • 1 tracked plugin
  • Up to 3 tracked Skills or components
  • Metadata events for local and design-partner testing
  • 1 dashboard user
  • CLI included

Builder

For builders shipping their first plugin portfolio.

  • 3-10 tracked plugins
  • Portfolio dashboards for Skills and components
  • Higher metadata event allowance
  • Eval suites and version comparison workflows
  • 1-3 dashboard users
  • CLI included

Team

For teams operating plugins across customers or departments.

  • 10+ tracked plugins
  • Production event allowance with overage path
  • Shared eval, feedback, and promotion workflows
  • Webhook and CSV export workflows
  • 5-10 dashboard users
  • Extra seats available

Enterprise

For organizations with security, scale, and procurement needs.

  • Contracted plugin, Skill, and event volume
  • Custom retention and data handling review
  • SSO, audit, and governance controls
  • Security and procurement review
  • Dedicated support

Learn the playbook

Vendor-neutral guides on building, measuring, evaluating, and improving Agent Plugins and their Skills.

Frequently asked questions

What is an Agent Plugin?
An Agent Plugin is an installable package of capabilities for an agent harness. It can contain Skills, tools, connectors, hooks, or agent configuration. Plugins are the apps people install; Skills are task-specific capabilities inside them.
What is Telvine?
Telvine is the eval-and-improve control plane for agent plugins and Skills. Bring your own harness and models; Telvine turns production traces, outcomes, feedback, evals, version comparisons, and promotion gates into better plugin and Skill releases.
Which agents and runtimes does Telvine support?
Any agent that can run a Plugin capability and emit an event. Telvine's model is runtime-agnostic and maps one plugin product to adapters for Codex, Microsoft 365 Copilot Cowork, Claude Cowork, Claude Code, and SKILL.md-based harnesses.
How do I instrument a Plugin?
Publish the plugin directory with the CLI, import its evals, and emit metadata-only Skill or component events from observable runtime boundaries. Telvine captures invocation metadata, errors, feedback, outcomes, latency, and version signals while your existing analytics stack remains the system of record.
Can PMs include evals for Skills?
Yes. PM-authored eval suites can live with the plugin as versioned assets today, and Telvine treats evals as part of the improvement loop: define scenarios, expected outcomes, rubric criteria, pass thresholds, and promotion gates without using production user content.
How does Telvine help plugin owners ship better output?
Each plugin and Skill version carries its component inventory, eval results, production outcomes, feedback, latency, and error evidence. Owners can compare versions, catch regressions, and promote only the release with the strongest proof.
Does Telvine collect prompts or user content?
No. The event envelope is a closed schema of typed metadata — counts, durations, enums, ratings. It never carries prompts, file contents, or other user content, so you can instrument Plugin capabilities without exposing what they operate on.
Why does this matter now?
Teams are shipping plugins and Skills into agent harnesses without a reliable way to know whether a new version actually helped. Existing tools can store events; Telvine gives teams the plugin-native and Skill-native traces, eval signals, and version evidence to improve what agents do.
Does Telvine replace PostHog, Mixpanel, Amplitude, or my warehouse?
No. Telvine is the instrumentation and normalization layer in front of those tools. Use Telvine to produce safe plugin events, then forward or export them into the stack your team already uses.

Make every plugin and Skill version better than the last.

Start with one plugin workspace, one component inventory, one telemetry contract, and one improvement loop.

Start freeRead the docs →