Flaky Tests interview questions — Medium
249 medium-level Flaky Tests questions from our Automation Testing bank, each with the correct answer and an explanation of why it is correct.
What this covers
Flaky Tests appears throughout Automation Testing interviews. At medium level, interviewers are typically checking that you have used it in practice: what you configured, what broke, and how you knew it worked. Work through these, then explain your answer out loud; the second part is what interviews actually test.
20 example questions
1. "Percentage of runs where a test's outcome flips without code change" — in Flaky Tests, this describes which of the following?
- A. Flake rate.correct
- B. Test isolation.
- C. Wait-for condition.
- D. Quarantine.
Why: flake rate: Percentage of runs where a test's outcome flips without code change.
2. "Fully isolated environment with no external shared dependencies" — in Flaky Tests, this describes which of the following?
- A. Quarantine.
- B. Flake rate.
- C. Hermetic environment.correct
- D. Wait-for condition.
Why: hermetic environment: Fully isolated environment with no external shared dependencies.
3. "Failure mode where tests pass only in a specific sequence" — in Flaky Tests, this describes which of the following?
- A. Hermetic environment.
- B. Order dependency.correct
- C. Quarantine.
- D. Test isolation.
Why: order dependency: Failure mode where tests pass only in a specific sequence.
4. "Isolating unstable tests from the blocking pipeline" — in Flaky Tests, this describes which of the following?
- A. Test isolation.
- B. Deterministic test.
- C. Hermetic environment.
- D. Quarantine.correct
Why: quarantine: Isolating unstable tests from the blocking pipeline.
5. "Polling until an application state is true instead of sleeping" — in Flaky Tests, this describes which of the following?
- A. Wait-for condition.correct
- B. Deterministic test.
- C. Order dependency.
- D. Retry-on-failure.
Why: wait-for condition: Polling until an application state is true instead of sleeping.
6. "Re-running failed tests a bounded number of times" — in Flaky Tests, this describes which of the following?
- A. Deterministic test.
- B. Wait-for condition.
- C. Order dependency.
- D. Retry-on-failure.correct
Why: retry-on-failure: Re-running failed tests a bounded number of times.
7. Which statement correctly describes Flaky Tests?
- A. Automatically retrying tests forever is a root-cause fix for flakiness.
- B. Tests sharing global state never influence each other.
- C. Measuring a flake rate per test over many runs identifies the worst offenders objectively.correct
- D. A flaky test fails consistently every single run.
Why: Historical pass/fail data ranks tests for stabilization work.
8. Identify the accurate statement about Flaky Tests:
- A. Shared mutable state between tests causes order-dependent failures.correct
- B. Tests sharing global state never influence each other.
- C. Adding longer sleep() calls is the recommended permanent fix for flakiness.
- D. Time-of-day can never affect the outcome of an automated test.
Why: Tests must be independent to run in any order or in parallel.
9. Which of the following statements about Flaky Tests is TRUE?
- A. Adding longer sleep() calls is the recommended permanent fix for flakiness.
- B. Test data collisions in shared environments cause intermittent failures under parallel runs.correct
- C. A flaky test fails consistently every single run.
- D. Time-of-day can never affect the outcome of an automated test.
Why: Unique per-test data or isolated environments prevent collisions.
10. Which Flaky Tests concept matches this description: "Isolating unstable tests from the blocking pipeline"?
- A. Quarantine.correct
- B. Retry-on-failure.
- C. Test isolation.
- D. Wait-for condition.
Why: quarantine: Isolating unstable tests from the blocking pipeline.
11. In Flaky Tests, which of the following is described as: "Ensuring each test sets up and cleans its own state"?
- A. Test isolation.correct
- B. Deterministic test.
- C. Wait-for condition.
- D. Flake rate.
Why: test isolation: Ensuring each test sets up and cleans its own state.
12. Which Flaky Tests concept matches this description: "Test that produces the same result for the same code every run"?
- A. Deterministic test.correct
- B. Order dependency.
- C. Wait-for condition.
- D. Test isolation.
Why: deterministic test: Test that produces the same result for the same code every run.
13. Which Flaky Tests concept matches this description: "Fully isolated environment with no external shared dependencies"?
- A. Order dependency.
- B. Hermetic environment.correct
- C. Wait-for condition.
- D. Test isolation.
Why: hermetic environment: Fully isolated environment with no external shared dependencies.
14. In Flaky Tests, which of the following is described as: "Percentage of runs where a test's outcome flips without code change"?
- A. Quarantine.
- B. Flake rate.correct
- C. Hermetic environment.
- D. Test isolation.
Why: flake rate: Percentage of runs where a test's outcome flips without code change.
15. In Flaky Tests, which of the following is described as: "Polling until an application state is true instead of sleeping"?
- A. Test isolation.
- B. Wait-for condition.correct
- C. Deterministic test.
- D. Retry-on-failure.
Why: wait-for condition: Polling until an application state is true instead of sleeping.
16. Which Flaky Tests concept matches this description: "Failure mode where tests pass only in a specific sequence"?
- A. Hermetic environment.
- B. Order dependency.correct
- C. Wait-for condition.
- D. Quarantine.
Why: order dependency: Failure mode where tests pass only in a specific sequence.
17. Which Flaky Tests concept matches this description: "Re-running failed tests a bounded number of times"?
- A. Test isolation.
- B. Wait-for condition.
- C. Deterministic test.
- D. Retry-on-failure.correct
Why: retry-on-failure: Re-running failed tests a bounded number of times.
18. In the context of Flaky Tests, which of the following is a correct statement?
- A. Adding longer sleep() calls is the recommended permanent fix for flakiness.
- B. Test data collisions in shared environments cause intermittent failures under parallel runs.correct
- C. Flaky tests should be deleted immediately without investigation in all cases.
- D. A flaky test fails consistently every single run.
Why: Unique per-test data or isolated environments prevent collisions.
19. In Flaky Tests, which of the following is described as: "Failure mode where tests pass only in a specific sequence"?
- A. Retry-on-failure.
- B. Hermetic environment.
- C. Deterministic test.
- D. Order dependency.correct
Why: order dependency: Failure mode where tests pass only in a specific sequence.
20. In Flaky Tests, which of the following is described as: "Re-running failed tests a bounded number of times"?
- A. Wait-for condition.
- B. Test isolation.
- C. Order dependency.
- D. Retry-on-failure.correct
Why: retry-on-failure: Re-running failed tests a bounded number of times.