Flag as Flaky

Manually mark a test as flaky from the test detail page

Manually mark a test as flaky when you know it’s unreliable but automated monitors haven’t detected it yet — or when you want to keep a test classified as flaky while monitors consider it healthy.

When to Use It

• A test is intermittently failing but hasn’t been flagged by threshold or pass-on-retry monitors.
• You want to immediately quarantine a test while investigating.
• You’ve identified a flaky test through code review or local observation.

How It Works

Flagging a Test

1. Navigate to the test detail page for the test you want to flag.
2. Click the Flag as Flaky button in the header row, next to the status badge.
3. In the popover that appears, optionally add a reason (up to 256 characters) explaining why you’re flagging it.
4. Click Flag to confirm.

Once flagged:

• The test is immediately marked as flaky, even if no automated monitor has detected it.
• An amber banner appears below the header showing who flagged it, when, and the reason (if provided).
• The flag is additive — if automated monitors later detect the test as flaky too, both signals coexist.

A manual flag does not override a broken status. Broken always takes precedence over flaky, so if an active monitor classifies the test as broken, the test shows as broken until that monitor resolves or is muted. While a test is broken, the Flag as Flaky button is disabled — if you believe the broken classification is incorrect, mute the monitor or update its configuration instead. You can still remove an existing flag while the test is broken.

Removing the Flag

1. On the test detail page, find the amber “Manually flagged as flaky” banner.
2. Click the Remove flag button on the right side of the banner.
3. Confirm by clicking Remove flag in the popover.

After removing:

• The test’s status reverts to whatever the automated monitors determine.
• If monitors are still detecting the test as flaky, it remains flaky. The flag removal only clears the manual override.

Relationship to Monitors

The “Flag as Flaky” action is separate from automated monitors (threshold-based, pass-on-retry) and does not appear in the Monitors tab, but it participates in the same status resolution as health classification monitors: the most severe status wins, and broken outranks flaky. If a test’s status looks wrong — for example, it shows as broken when you believe it’s merely flaky — mute or reconfigure the monitor responsible rather than reaching for the flag.

Scenario	Test status
No monitors active, no flag	Healthy
Flaky monitors active, no flag	Flaky (detected)
No monitors active, flag set	Flaky (manually flagged)
Flaky monitors active, flag set	Flaky (both)
Broken monitor active, flag set	Broken (broken takes precedence)
Broken monitor resolves or is muted, flag set	Flaky (manually flagged)
Flag removed, flaky monitors still active	Flaky (detected)
Flag removed, monitors inactive	Healthy

Flag History

All flag and unflag actions are recorded as events. You can view the history by opening the Flag History panel from the test detail page. Each entry shows who performed the action, when, and the reason (if one was provided).