Improving Selenium Test Stability with Pytest Retries and Waits

Introduction

Flaky tests—those that fail intermittently—are a common headache for test automation teams. They can be especially frustrating in Selenium tests because of the dynamic nature of web applications. Elements might take time to load, page navigation could be slow, or JavaScript-heavy applications might delay interactions. These issues lead to false negatives in tests, where tests fail even though the application works fine.

In this blog, we’ll explore how to use Pytest retries and explicit/implicit waits to improve the stability of your Selenium tests and reduce flaky test failures.

Why Selenium Tests Are Flaky

Revolutionize Your Business With Generative AI

From product design and software development to virtual agents, content creation, and reporting, GenAI is transforming business. Our AI experts help you unlock GenAI’s full potential and drive growth.

Let’s Get Started

Flaky tests typically fail due to the following issues:

• Dynamic Content: Elements that take time to load (like AJAX content) or slow-rendering pages.
• Network Issues: Delays or failures in loading resources or API calls.
• Timing Issues: Trying to interact with elements before they’re fully loaded or ready.

The key to reducing flaky tests lies in two techniques: retries and waits.

Using Pytest Retries for Flaky Tests with pytest-rerunfailures

A simple solution to mitigate flaky tests is to retry failed tests a certain number of times. The pytest-rerunfailures plugin allows you to automatically rerun tests that fail, thus reducing the impact of intermittent failures.

1. Installation: Install the pytest-rerunfailures plugin:

   bash
   pip install pytest-rerunfailures
   

2. Configuration: To enable retries, use the –reruns option when running your tests. For example, to retry a failed test 3 times, run:

   bash
   pytest --reruns 3
   

   You can also set the number of retries in your pytest.ini configuration file:

   ini
   
   [pytest]
   
   reruns = 3
   
   rerunsDelay = 2  #Delay between retries in seconds

    

3. Example of Retries: Let’s say you have a test that clicks a button to submit a form. Sometimes, due to timing issues, the button might not be clickable. By adding retries, the test will automatically retry if the failure is due to a transient issue.

   def test_submit_button(mocker):
       # Simulate flaky behavior
       mocker.patch('selenium.webdriver.common.by.By.ID', return_value='submit')
       # Trigger a click action on the button
       button = driver.find_element_by_id('submit')
       button.click()
       assert button.is_enabled()  # Check button state
   

Using Waits to Ensure Elements Are Ready

In Selenium, waits are crucial to ensure that the elements you want to interact with are available and ready. There are two types of waits: implicit and explicit.

1. Implicit Waits: Implicit waits instruct the WebDriver to wait a certain amount of time for elements to appear before throwing an exception.

   driver.implicitly_wait(10)  # Waits for 10 seconds for elements to load

   While easy to use, implicit waits can sometimes slow down tests and make debugging more difficult because they apply globally to all elements.

2. Explicit Waits: Explicit waits are more powerful and precise. They allow you to wait for specific conditions before proceeding with interactions. WebDriverWait combined with expected conditions is commonly used.Example: Wait for an element to be clickable before clicking on it:

   from selenium.webdriver.support.ui import WebDriverWait
   from selenium.webdriver.support import expected_conditions as EC
   from selenium.webdriver.common.by import By
   
   wait = WebDriverWait(driver, 10)
   element = wait.until(EC.element_to_be_clickable((By.ID, "submit_button")))
   element.click()
   

3. Using Waits for AJAX Content: Often, web pages use AJAX to load content dynamically. Explicit waits are perfect for waiting until AJAX calls finish loading.

   # Wait until the AJAX content is visible
   wait.until(EC.visibility_of_element_located((By.ID, "ajax-content")))
   

Best Practices for Waits and Retries

• Use explicit waits for better control: Explicit waits allow you to wait for specific conditions (like visibility or clickability), improving test reliability.
• Combine retries with waits: Ensure that retries are only triggered after sufficient wait time to account for potential page load or element rendering delays.
• Optimize test timing: Use waits for specific elements rather than using global implicit waits, which can slow down tests.

Conclusion

By using Pytest retries and explicit/implicit waits, you can significantly improve the stability of your Selenium tests. Retries help handle intermittent failures, while waits ensure that elements are ready before interacting with them. Together, these strategies reduce flaky test results, making your test suite more reliable and consistent. Happy Testing!