Do not use soft assertions for multiple-page Selenium tests

The following Selenium test uses soft assertions:

@Test
public void searchReturnsResultsTest() {

   SoftAssertions assertions = new SoftAssertions();

   HomePage homePage = new HomePage(driver);
   homePage.open();

   assertions.assertTrue(homePage.isDisplayed());

   ResultsPage resultsPage = homePage.search(MAKE, MODEL, LOCATION);

   assertions.assertTrue(resultsPage.isDisplayed());
   assertions.assertEquals(resultsPage.resultsPerPageCount(), 100);

   Result result = resultsPage.getResult(1);

   String resultTitle = result.title();

   assertions.assertFalse(resultTitle.isEmpty());

   DetailsPage detailsPage = result.select();

   assertions.assertTrue(detailsPage.isDisplayed());

   assertions.assertEquals(detailsPage.title(), resultTitle);

   assertions.assertAll();

}

The test uses multiple assertions to verify that

• home page is displayed

• results page is displayed

• results per page count is 100

• title is not empty for the 1st result

• details page is displayed

• title of result from details page is equal to the same title from results page

Since all assertions are wrapped with a soft assertion, even if their conditions are not met, they do not fail until the end of the script when all assertions are evaluated.

Why would we do this?

The idea is that, if an assertion fails, the code execution stops so, if there are additional issues, we will not know of them until the failed assertion passes.

With soft assertions, we would like to get all failed assertions for a test, not only the first one.

Unfortunately, this is not a good thing to do in this case.

Using soft assertions for a test like this that goes through different pages and makes assertions on each page is a bad practice.

Let’s see why.

---

Assume that the first assertion, the one that checks if home page is displayed, fails. Why could it fail?

It could fail because

• home page is not displayed at all or

• an error page is displayed or

• the site redirects the user to some other page or

• the home page URL is incorrect

This is a critical error due to which the execution of the remaining of the test is just a waste of time.

If home page is not displayed, there is no way the vehicle search can be executed.

So, if there is something wrong with home page, the test should stop and tell us exactly what happened: home page is not displayed.

---

Ignoring this fact and continuing the test execution is not helpful at all.

Let’s assume that the test executes to the end in some way.

We do get the status of all assertions when all assertions are evaluated.

Out of the 6 assertions, only the first one provides some useful information (home page is not displayed).

All others provide invalid information as their outcomes depend on the first assertion passing.

This is how the assertions depend on each other:

assertions.assertTrue(homePage.isDisplayed());
                     |
                     v
assertions.assertTrue(resultsPage.isDisplayed());
                     |
                     v
assertions.assertEquals(resultsPage.resultsPerPageCount(), 100);
                     |
                     v
assertions.assertFalse(resultTitle.isEmpty());
                     |
                     v
assertions.assertTrue(detailsPage.isDisplayed());
                     |
                     v
assertions.assertEquals(detailsPage.title(), resultTitle);

If the 1st assertion fails, the state of the site is invalid so the 2nd assertion will not matter. Same for any other assertion.

---

So, soft assertions are a bad idea for this type of test.

Using plain assertions is much better:

@Test
public void searchReturnsResultsTest() {

   HomePage homePage = new HomePage(driver);

   homePage.open();
  
   Assert.assertTrue(homePage.isDisplayed());

   ResultsPage resultsPage = homePage.search(MAKE, MODEL, LOCATION);

   Assert.assertTrue(resultsPage.isDisplayed());
   
   Assert.assertEquals(resultsPage.resultsPerPageCount(), 100);

   Result result = resultsPage.getResult(1);
   String resultTitle = result.title();
   
   Assert.assertFalse(resultTitle.isEmpty());

   DetailsPage detailsPage = result.select();

   Assert.assertTrue(detailsPage.isDisplayed());

   Assert.assertEquals(detailsPage.title(), resultTitle);

}

We will see in the next post when soft assertions should be used.