Testing against production safely

• Playwright by product module
• read-only enforced, not trusted
• OTP read over IMAP
• k6 load phases

The problem

The dashboard had many modules: stores, deploys, logs, analytics, an AI agent, settings, admin tools. Testing them by hand before a release was slow and patchy. Testing them only on a test environment missed the bugs that only exist in production: real data, real config, real routing.

Running a browser suite against production sounds reckless. A test that clicks "Delete" on the wrong page is an outage. And the login used an emailed one-time code, so the suite could not even start without a person.

I built an end-to-end stack that was safe to point at production and could run itself on a schedule.

Plan first, rig before coverage

I wrote the plan as its own change before any test code: a suite organised by product module, matching the dashboard's routes, driven by one command that picks the target, the environment and the module.

e2e --target=production --module=deploys

The first real change was only the rig: the config, the login state capture, the safety fixtures and one smoke test per module. Proving the rig worked end to end came before writing coverage. Then coverage landed module by module, each with page objects and specs. The AI agent module got its own change, because it had the largest blast radius. Admin-only screens were limited to test environments by construction.

The page objects ended up documenting the app. Where a test had to be skipped, the skip carried its reason: "this page is a backups page, not a data browser", or "creating a workspace takes minutes". A skip that explains itself is documentation. A silent one is a lie.

Read-only, proven not trusted

The key piece was a fixture, not a test. Every test that is not tagged @mutating gets an automatic fixture that watches every request the page sends to the dashboard. Any method other than GET, HEAD or OPTIONS is recorded, and after the test it fails with the list of offending URLs. A short allowlist covers the few POSTs that only read.

page.on('request', (req) => {
  if (!SAFE.has(req.method()) && !allowed(req.url())) violations.push(req.url())
})

I did not want production safety to depend on every author remembering a tag. The framework checks what a test did, not what it claims. Tests that do mutate are careful too: a test that flips a setting puts it back in a finally block, even when the check after the flip fails.

I also added an opt-in capture mode: trace, video and labelled screenshots at named steps, so a reviewer could scroll through what a test did without opening a trace viewer. With capture off, every existing spec ran unchanged.

Logging in without a person

The emailed code was the last manual step. I wrote a small IMAP helper. It polls the inbox for unseen mail from a pinned sender with a matching subject, pulls out the code with the same pattern the login form uses (so the two cannot drift apart), marks the message as read and returns the code.

A few details made it reliable:

• Fail fast. The IMAP settings are checked before the browser starts, so a bad config fails in a second, not after a timeout.
• No stale codes. Only mail that arrived after the run started is accepted, so a late email from an earlier run is ignored.
• Narrow matching. The sender is required and the subject must match exactly, so a random email cannot feed the suite a code.

With that, the suite ran on a schedule in CI.

Load testing a real store

Functional tests prove a journey works. They do not say whether it still works under a rush. For a store expecting heavy traffic, I built a k6 suite against a full clone of the store. It ran a weighted mix of realistic journeys, mostly browsing with smaller shares checking out and abandoning carts, in phases: a baseline with a few virtual users, then load, then stress.

To exercise checkout on the clone, I needed a payment method the store's rules normally hid. I added it with a temporary override, clearly labelled as temporary, with a note of the other change made for the run outside the code. A Playwright suite for the same journeys ran next to it: load tests measure how fast, the functional suite checks the shopper can still finish.

What I'd do differently

• Give the test environment the same email path as production early. The OTP helper failed once because the real subject line did not match, and once because mail arrived slowly. Both were small fixes, found late.
• Fix stale specs when routes change. When pages moved, the tests failed before the product did. Treating tests as part of a route change would have caught that.

Takeaway

Testing against production is safe when safety is enforced by the framework, not by convention. Watch every request, make the login automatic, and load test a clone before the traffic arrives.