whatisgithub

What is playwright-mutation-gate?

vladyslavdmitriiev/playwright-mutation-gate — explained in plain English

Analysis updated 2026-05-18

4TypeScriptAudience · developerComplexity · 3/5Setup · easy

In one sentence

A CI tool that flips each test's key assertion to prove it can actually fail, catching tests that pass without checking anything real.

Mindmap

mindmap
  root((Mutation Gate))
    What it does
      Inverts marked assertions
      Reruns tests in isolation
      Fails build on hollow tests
    Tech stack
      TypeScript
      Playwright
      Node.js
    Use cases
      Verify test quality in CI
      Catch hollow assertions
      Track deliberate skips
    Audience
      QA engineers
      Developers
      DevOps teams

Code map

Detail Auto

An interactive map of this repo's files and how they connect — its source is parsed live in your browser. Click Visualize to build it.

filefunction / class

What do people build with it?

USE CASE 1

Run this in CI to verify Playwright end to end tests actually fail when they should.

USE CASE 2

Catch newly added tests that lack a real, working assertion.

USE CASE 3

Track intentionally skipped tests with an owner and expiration date instead of letting them linger.

USE CASE 4

Check whether an assertion truly depends on the network data it claims to verify.

What is it built with?

TypeScriptPlaywrightNode.jsGitHub Actions

How does it compare?

vladyslavdmitriiev/playwright-mutation-gate0labs-in/vision-linkalfons-fhl/cursor-plan2api
Stars444
LanguageTypeScriptTypeScriptTypeScript
Setup difficultyeasymoderatemoderate
Complexity3/53/53/5
Audiencedeveloperdeveloperdeveloper

Figures from each repo's GitHub metadata at analysis time.

How do you get it running?

Difficulty · easy Time to first run · 30min

Requires Node 20 or higher and an existing Playwright test setup already configured in the project.

Details are shown in a LICENSE file linked from the README, but the license type itself is not stated in the text provided.

So what is it?

This is a testing tool for teams that write end to end tests with Playwright, the browser automation framework. Its purpose is to catch a specific problem: tests that pass but do not actually check anything meaningful, sometimes called hollow assertions. The idea is simple. You mark the main assertion in each test with a comment, then run this tool as part of your CI pipeline. It takes that marked assertion, flips it to expect the opposite result, and reruns the test in a temporary copy. If the test still passes after the assertion was reversed, that means the original assertion was never really testing anything, and the tool fails the build to flag it. If the test fails once reversed, that is a good sign, since it shows the assertion was actually connected to real behavior. The tool reports each test as killed, meaning the assertion is doing real work, survived, meaning it is not, or cannot mutate, meaning the assertion could not be safely reversed. By default, any test without a marker also fails the check, though this can be turned off while a team is still adding markers to an older test suite. There is also a system for deliberately skipped tests, where a comment can record an owner and an expiration date so the exception does not linger forever unnoticed. A newer, optional mode goes further by rewriting the actual data a test's assertion depends on as it travels over the network, rather than just flipping the assertion itself. This checks whether an assertion is truly tied to the data it claims to verify, rather than a cached or coincidentally correct value. It ships as an npm package, requires Node version 20 or higher plus an existing Playwright setup, and includes a sample GitHub Actions workflow for running it in CI. Cost scales with the number of marked assertions, so the README suggests running it only on changed test files per pull request, or as a full nightly sweep instead of on every commit.

Copy-paste prompts

Prompt 1
Show me how to add a primary-assert marker to my existing Playwright test suite.
Prompt 2
Set up a GitHub Actions workflow that runs playwright-mutation-gate only on changed spec files.
Prompt 3
Explain the difference between killed, survived, and cannot-mutate results from this tool.
Prompt 4
Help me write a gate-skip marker with an owner and expiration date for a test I need to skip temporarily.
Prompt 5
Walk me through what the --behavior mode does differently from normal assertion inversion.

Frequently asked questions

What is playwright-mutation-gate?

A CI tool that flips each test's key assertion to prove it can actually fail, catching tests that pass without checking anything real.

What language is playwright-mutation-gate written in?

Mainly TypeScript. The stack also includes TypeScript, Playwright, Node.js.

What license does playwright-mutation-gate use?

Details are shown in a LICENSE file linked from the README, but the license type itself is not stated in the text provided.

How hard is playwright-mutation-gate to set up?

Setup difficulty is rated easy, with roughly 30min to a first successful run.

Who is playwright-mutation-gate for?

Mainly developer.

Open on GitHub → Ask about another repo

This repo across BitVibe Labs

Verify against the repo before relying on details.