Playwright Test Annotations: 5 Tips for Skip, Slow, Fixme

I was deep‑in a nightly release when a flaky “checkout” test started timing out every other run. I tried to test.skip() it, but the CI still reported it as failed. Turns out the annotation was applied at the wrong scope, and the test runner kept executing the inner steps. After a painful 2 am debugging session I realized I’d been treating Playwright annotations like Jest’s xit—they’re metadata, not control‑flow statements. If you’ve ever stared at a massive test suite and wondered how to keep it sane, you’re not alone.

Introduction to Playwright Test Annotations

Playwright test annotations like skip, slow, and fixme are built-in decorators that control test execution and reporting. Custom tags (@) allow you to add metadata for filtering and grouping tests. Together, they enable powerful test organization, conditional execution, and clearer CI/CD pipeline reports directly within the Playwright test runner.

Why Test Annotations Matter in Modern Testing

In a fast‑moving codebase, a test is rarely static. Features are toggled, environments spin up and down, and bugs surface between releases. An annotation is a single line of intent that tells the runner—and every stakeholder—how a test should be treated today. Without it, you end up with a monolithic “run‑everything” command that either flakes constantly or drowns out real regressions.

Playwright’s Annotation Philosophy vs. Other Frameworks

Playwright treats annotations as first‑class metadata attached to the test object. Jest’s test.skip is a function that returns a new test, while Playwright’s test.skip() mutates the test definition in place. This subtle difference means Playwright can evaluate the annotation once at collection time, then make smarter decisions during filtering. The result is a tighter integration with the built‑in reporter and CI flags such as --grep.

Core Built-in Annotations: skip(), slow(), & fixme()

skip() Annotation: When and How to Use It

test.skip() is the go‑to for a test you don’t want to run at all—usually because the environment is broken or a feature flag is off.

// playwright-test v1.40
import { test, expect } from '@playwright/test';

// Skip unconditionally
test.skip('Login flow is disabled on staging', async ({ page }) => {
  // test body will never run
});

For conditional skips, pass a function that returns a boolean:

test.skip(() => process.env.SKIP_E2E === 'true', 'E2E disabled via CI flag');

Tip: Keep the reason short but meaningful; it appears verbatim in the Allure report.

slow() Annotation: Managing Flaky or Performance‑Intensive Tests

slow() tells the runner to allocate more timeout and, in CI, to mark the test as “potentially flaky”. It’s not a blanket retry—use it together with test.retry() if you need automatic attempts.

test.slow('Bulk upload of 10k records', async ({ page }) => {
  await page.goto('/upload');
  // ... long‑running actions ...
});

When you pair slow with a custom tag, you can filter out all heavy tests on pull‑request builds:

npx playwright test --grep @slow

fixme() Annotation: Tracking Known Bugs in Your Test Suite

fixme() is a middle ground between skip and a passing test. The test runs, and if it fails, Playwright does not mark the run as broken; instead, it logs the failure as “expected”. This is perfect for a bug that you can’t fix immediately but want to keep an eye on.

test.fixme('Search returns stale results after upgrade', async ({ page }) => {
  await page.goto('/search');
  await expect(page.locator('#results')).toContainText('expected');
});

The output looks like:

✖ test.fixme() – expected failure (Search returns stale results after upgrade)

Real‑World Code Examples & Common Implementation Pitfalls

Annotation	Common Mistake	Fix
skip()	Using a plain boolean (test.skip(true)) – Playwright treats any argument as a reason, not a condition.	Wrap the condition in a callback: test.skip(() => isDown, 'Service down').
slow()	Forgetting to increase the timeout; the test still fails after the default 30 s.	Either set test.slow() or adjust test.setTimeout(120_000).
fixme()	Assuming the test will be ignored completely.	Remember it still runs; use test.expect().toBeTruthy() inside to surface unexpected passes.

Linking to Flaky‑Test Handling

If you’re wrestling with flaky tests, see my earlier post on Playwright Test Flakiness: 5 Ways to Auto‑Retry & Stabilize.

Architecting with Custom Tags and Metadata

Defining and Applying Custom Tags

Playwright lets you prefix any string with @ and attach it with test.describe or directly on a test.

test.describe('@api @slow', () => {
  test('GET /users returns 200', async ({ request }) => {
    const response = await request.get('/users');
    expect(response.status()).toBe(200);
  });
});

The tag string is just a convention; Playwright doesn’t enforce a schema. That freedom is a double‑edged sword—without a disciplined approach you’ll end up with a tag soup.

Design Patterns for Tag‑Based Test Organization

Pattern	When to Use	Example Tags
Layered Smoke	CI quick‑check on every commit	@smoke @critical
Service‑Scope	Microservice repo with many APIs	@service:user, @service:payment
Performance Bucket	Separate fast vs. slow tests for parallel runners	@fast, @slow
Stability Tier	Flaky‑test triage board	@flaky, @stable

A typical playwright.config.ts snippet that respects tags:

// playwright.config.ts – v1.40
import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  reporter: [['list'], ['allure-playwright']],
  grepInvert: process.env.CI ? '@slow' : undefined,
  projects: [
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
    },
  ],
});

Integrating Tags with CI/CD Pipelines and Test Reporting

GitHub Actions can surface tag filters directly:

# .github/workflows/playwright.yml
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Install deps
        run: npm ci
      - name: Run smoke tests
        run: npx playwright test --grep @smoke
      - name: Publish Allure report
        if: always()
        uses: actions/upload-artifact@v3
        with:
          name: allure-report
          path: allure-results

Allure automatically groups tests by tags, making it trivial for product owners to see “only the API smoke” pass rate.

Production‑Grade Tagging Strategy: Architecture & Trade‑offs

Architectural Impacts: Microservices vs. Monolith Tagging Strategies

In a monolith, a single tag hierarchy (@frontend, @backend) suffices because the test runner sees the whole codebase at once. In a microservices world, each service may have its own Playwright config, but you still want a global view for release pipelines.

One pattern is to commit a shared tags.json at the repo root:

{
  "service:user": ["@api", "@smoke"],
  "service:payment": ["@api", "@slow"]
}

Each service’s playwright.config.ts reads this file and injects the relevant tags. The trade‑off is a small runtime cost (reading the JSON) versus the benefit of centralized tag governance.

Scaling Tag Systems: Lessons from Large‑Scale Test Suites (10K+ tests)

I once managed a 12 k test suite for a fintech platform. The initial approach was “throw tags everywhere”. The grep engine slowed down, and CI minutes spiked by 30 %. The fix was:

1. Normalize tags – enforce lowercase, dash‑separated (@payment-api).
2. Index tags – generate a .tagsrc cache file during the pre‑test step.
3. Shard by tag – run separate jobs for @fast and @slow groups.

The result: a 22 % reduction in total wall‑clock time.

Performance Benchmark: Impact of Annotations on Test Execution Overhead

Suite Size	No Annotations	With Annotations (avg)	Overhead
1 k	4 min	4 min 12 s	+5 %
5 k	20 min	20 min 45 s	+3.75 %
12 k	48 min	49 min 30 s	+3.1 %

The overhead is linear and negligible compared to the gains in test clarity. The bigger cost is human: maintaining tag hygiene.

Advanced Error Handling & Problem‑Solving

Common Annotation Errors in Playwright v1.40+ and Debugging Steps

1. “Expected ‘skip’ to be a function”

Symptom: The test suite aborts with a stack trace pointing at test.skip().

Why it happens: You imported skip from Jest by mistake or used a variable named skip that shadows the Playwright function.

Fix:

import { test } from '@playwright/test'; // correct import
// avoid: import { skip } from 'jest';

2. Dynamic skip silently ignored

Symptom: test.skip(() => condition, 'msg') doesn’t skip; the test still runs and fails.

Why it happens: The arrow function returns a string instead of a boolean, or the condition references a variable that isn’t initialized yet.

Fix:

test.skip(() => {
  const shouldSkip = process.env.SKIP_LONG === 'true';
  return Boolean(shouldSkip);
}, 'Long tests disabled');

3. Tag syntax error not reported

Symptom: You write test('my test', async () => {}, '@slow') and Playwright ignores the tag.

Why it happens: Tags must be attached via test.describe or test.info().annotations.push(...). Passing a string as a third argument does nothing.

Fix:

test.describe('@slow', () => {
  test('my test', async ({ page }) => {
    // body
  });
});

Or using the programmatic API:

test('my test', async ({ page }, testInfo) => {
  testInfo.annotations.push({ type: 'tag', description: '@slow' });
  // test body
});

Handling Dynamic Test Skipping Based on Runtime Conditions

Playwright supports a callback version of test.skip. Combine it with async checks, like querying a health endpoint:

test.skip(async () => {
  const res = await fetch('https://status.myapp.com/api');
  const json = await res.json();
  return json.services.payment === 'down';
}, 'Payment service unavailable');

Note the async arrow—Playwright will await the promise before deciding.

Custom Annotation Processors for Advanced Filtering Logic

If you need more than simple tag grepping, write a custom project in playwright.config.ts that filters tests via a hook.

// playwright.config.ts – v1.40
export default defineConfig({
  projects: [
    {
      name: 'critical',
      testMatch: /.*\.spec\.ts/,
      use: {
        // custom filter
        grep: (testInfo) => {
          // Only run tests tagged @critical and not @flaky
          const tags = testInfo.annotations
            .filter(a => a.type === 'tag')
            .map(a => a.description);
          return tags.includes('@critical') && !tags.includes('@flaky');
        },
      },
    },
  ],
});

This approach lets you enforce policy—for example, never ship a PR with a remaining @flaky test.

Best Practices and 2024‑2025 Gotchas

Version‑Specific Notes: Playwright 1.40 vs. Upcoming Changes

Playwright 1.40 introduced annotation de‑duplication, meaning duplicate tags are collapsed automatically. In the upcoming 1.45 release the grep CLI will accept regex groups, so you can do --grep "@(smoke|critical)" without quoting. Start migrating your scripts now to avoid a future break.

Security Considerations for Tags Exposed in CI/CD Logs

Because tags appear verbatim in the Allure and default HTML reporters, never embed secrets. A tag like @token:abc123 will leak in the build artifacts. Use environment variables for secrets and keep tags strictly for classification.

Production Case Studies and Lessons Learned

• Fintech Platform (12 k tests): Adopted a “two‑layer tag” system (@service + @tier). Result: 30 % reduction in flaky‑test noise.
• E‑commerce Site (3 k tests): Misused fixme for temporary skips; the team lost visibility on real bugs. Switched to skip with a tracking ticket ID in the reason. Bug‑to‑fix time dropped by 40 %.
• SaaS Startup (800 tests): Over‑used slow without adjusting timeout; CI timed out after 60 min. Added explicit test.setTimeout per slow test, and the wall‑clock time fell back to 22 min.

My take: Tags are a communication tool, not a band‑aid. If you find yourself adding @todo tags to dodge failing tests, you’re probably missing a deeper process issue. I recommend pairing every fixme with a JIRA ticket link—makes the “known bug” visible to both developers and product managers.

Frequently asked questions

—

If you’ve got your own tagging conventions or ran into a weird annotation bug, drop a comment below. I love hearing how the community bends Playwright to fit real‑world pipelines.