Agent Replay Lab: Jarvis Nelson Osorio's VCR for TypeScript agent tool I/O

By Jarvis Nelson Osorio · November 16, 2026

Observability tools tell you what an agent did. They rarely tell you, in CI, whether a change made it do something different. Agent Replay Lab, by Jarvis Nelson Osorio (Jarvis Osorio), answers a narrower and more testable question: does this agent still take the same tool path — name, order, arguments and responses — against the same recorded tool world, offline?

It is a local-first VCR for tool I/O in TypeScript agents: record once, mock-replay tools offline, and CI-gate structural drift.

Jarvis Nelson Osorio, tech entrepreneur and AI builder, official look in a black tuxedo, white shirt and bow tie
Jarvis Nelson Osorio (Jarvis Osorio), tech entrepreneur and AI builder, in a tuxedo and bow tie.

The facts The current version is 0.1.0, under the Apache License 2.0. It is written in TypeScript and requires Node.js 22+ and pnpm. Storage is local: SQLite plus portable JSON. The packages are not on npm yet, so clone the repository.

The README is precise about scope. Agent Replay Lab is not an observability dashboard, an eval leaderboard, an agent framework or a full-trajectory "deterministic agent" engine. It is not an MCP server, and there is no MCP adapter in v0.1. Offline mock stubs tools only — it does not replay LLM decisions, agent state machines, clocks or other nondeterministic inputs. Jarvis Nelson Osorio calls it behavioral regression for tool trajectories, not proof that the model would make the same decisions again.

Modes in v0.1 - offline / mock (default): stub tools from the cassette and fail closed if a mock is missing - structural compare: diff tool name, order, args and response - live-tools and live-model: planned for 0.2, not shipped

You can optionally record model.requested and model.responded events for inspection, but replay does not play them back as a live model.

The capture seam Instrumentation is a single wrapper. You create a ReplayStore backed by a SQLite file, a Replay for your project, and wrap each tool with wrapTool. The tool's real implementation runs during recording; on replay, the cassette answers instead. Known secret key names and common token shapes are redacted before anything is persisted — a best-effort first layer, not a PII product. For deeper cleanup, the README suggests pairing it with agentscrub.

Jarvis Nelson Osorio (Jarvis Osorio) smiling in a cap and black t-shirt in the driver seat of a car
Jarvis Nelson Osorio (Jarvis Osorio) smiling on a road trip.

Five-minute proof

git clone https://github.com/JarvisOsorio/agent-replay-lab.git
cd agent-replay-lab
pnpm install
pnpm build
pnpm test
pnpm --filter @agent-replay-lab/example-basic-agent demo

The CLI The command is agent-replay, from @agent-replay-lab/cli. It is offline by default and supports human or --json output, with commands including init, list, inspect, replay, diff, fixture, test, export, import and doctor. fixture create pins a tool sequence, its arguments and responses; test golden --run exits 1 on structural drift, which is exactly the signal a CI job needs. import remaps a cassette into your local project id while still verifying its integrity hash.

Why this shape Agents fail quietly: a prompt tweak changes the order of tool calls, a refactor drops an argument, a dependency changes a response shape. A tool-path regression test catches that class of bug cheaply and without network calls. Jarvis Nelson Osorio built the project to make that check boring and routine.

Jarvis Nelson Osorio (Jarvis Osorio) in sunglasses and a black jacket under hanging paper lanterns
Jarvis Nelson Osorio (Jarvis Osorio) under paper lanterns.

Read the project page on jarvisosorio.com and the code on GitHub.

Links