Samuel Arendt

Emulate sits between fragile mocks and expensive staging

Vercel Labs released Emulate, a local stateful implementation of external APIs for CI and no-network sandboxes. Running npx emulate starts local GitHub, Vercel, and Google endpoints. Creating a repository, adding collaborators, opening issues, merging a pull request, and deleting related state follow modeled transitions rather than fixed mock responses.

The project includes cursor pagination, cascade deletion, scoped tokens, seeded organizations, and strict credential checks. It can validate GitHub App JWTs with RS256 and deliver signed webhooks with HMAC-SHA256. Configuration begins with defaults and can grow through YAML or seed JSON.

AI-agent sandboxes are a strong use case. Letting an agent touch production GitHub or Vercel data during a test is unsafe, while a shallow mock often misses state transitions. A local emulator gives the agent a system it can change and inspect repeatedly without network access or external cost.

What local state covers and what still needs the real service

npx emulate starts Vercel on 4000, GitHub on 4001, and Google on 4002. The project README now lists other services in the same style, each on the next port, with seed data and reset(). The stated target is CI with no network and an agent sandbox that cannot point at production.

Cascade delete, cursor pagination, branch protection, and merges that honor rules cover the behavior your code assumes. Real rate limits, incidents, a new field in a payload, and the endpoint the emulator does not implement yet stay outside.

Keep at least one canary, job, or periodic test talking to the external service on a controlled schedule. The emulator cuts cost and the risk of real data. The public API remains the contract the end user hits.

Seed, token, and the surface a test claims to cover

YAML and JSON seeds are the fixture. A scoped token, org, repo, GitHub App with RS256 JWT, webhook with HMAC-SHA256: that only holds if the seed looks like the case that breaks outside.

If the suite uses an admin token and production uses an App with contents: read, green lies. With oauth_apps configured, client_id validation is strict. Without that list, the emulator accepts any client, which is convenient on day one and dangerous if the suite forgets the loose mode.

Call reset() between cases. State leaking from one test to another reproduces the old staging flake, only faster and on localhost. In CI, createEmulator per service, with a fixed port and close() at the end of the suite, makes isolation explicit. A versioned seed in the repository stops each person from running a different imaginary GitHub.

What an agent sandbox actually proves

Letting the model create a repo, open a PR, merge, fire a webhook, and inspect the result on localhost trains the harness: permission, retry, error parsing, what the agent does when the issue vanishes in a cascade. No network, no invoice, no junk in the real org.

The sandbox shows whether the agent obeys the protocol you modeled: auth, state transition, signed webhook. Real GitHub can still return a different field the following week. Capture live responses from time to time and compare. Where they diverge, the seed is thin or the code is stuck on a detail Emulate does not cover.

Read the limits the README itself lists. Several services declare what they skip: Socket Mode, Connect, exact rate limits, paid corners. Agent tests on those edges pass in a vacuum.

Green CI and the list of what you refuse to emulate

Nineteen Vercel write endpoints and broad GitHub coverage, including search, branch protection, and workflows, show the project aims at the flow the app actually runs, not a user GET. Stars in a few days show the pain is common.

Treat Emulate as a development environment and isolated CI. Staging remains the right question when the production token, the installed app, and the public webhook need to talk. Localhost answers another: does the code react to the transitions the team modeled?

If the external integration suite disappears and nobody looks at the service again, the next API change arrives as an incident. The space between a fragile mock and expensive staging exists. It needs an owner for the seed and an explicit list of what you deliberately do not emulate.

If your company is weighing something similar, reach me on LinkedIn or WhatsApp.

References

#AI Agents, #Architecture, #Radar, #Software Quality