Hard and Soft Assertions in Selenium

This post covers how to handle Assertion Failure in Selenium. Test scripts can be very robust and large. Assertions are the best method to perform any kind of validations in the tests. When an assertion fails the test script stops execution unless handled in some form. There are two types of assertions:

1. Hard Assertions
2. Soft Assertions.

Hard Assertions

A Hard Assertion is type of assertion that throws an exception immediately when an assert statement fails and continues with the next test in the test suite. In order to achieve this, we need to handle the Assertion Error that is thrown with a catch block like a Java exception. After the suite completes execution, the particular test is marked as passed instead of a FAIL.

Let’s assume that there are two tests in a suite, and the first test in the suite has an assertion that fails. In order to continue with the second test in the suit, we will have to handle the assertion failure in the first test.

Here is the code to handle an assertion:

package com.example;
import org.testng.Assert;
import org.testng.annotations.Test;
import org.testng.asserts.SoftAssert;
public class Sample {
@Test
public void test1(){
try{
Assert.assertTrue(2<1);
System.out.println(“Assertion Failed in Test 1”);
Assert.assertFalse(1>0);
System.out.println(“Assertion Failed in Test 1”);
Assert.assertEquals(“Sample”, “Sample”);
System.out.println(“Assertion Passed in Test 1”);
} catch(AssertionError ae){
ae.printStackTrace();
}
}

@Test
public void test2(){
try{
Assert.assertTrue(2>1);
System.out.println(“Assertion passed in Test 2”);
Assert.assertFalse(1<0);
System.out.println(“Assertion passed in Test 2”);
Assert.assertEquals(“Sample”, “Sample”);
System.out.println(“Assertion Passed in Test 2”);
} catch(AssertionError ae){
ae.printStackTrace();
}
}
}

We can add multiple catch blocks for a single try block to handle other exceptions.

Output would be like the below.

java.lang.AssertionError: expected [true] but found [false]
at org.testng.Assert.fail(Assert.java:94)
at org.testng.Assert.failNotEquals(Assert.java:494)
at org.testng.Assert.assertTrue(Assert.java:42)
at org.testng.Assert.assertTrue(Assert.java:52)
at com.example.Sample.test1(Sample.java:12)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:84)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:714)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:901)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1231)
at org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:127)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:111)
at org.testng.TestRunner.privateRun(TestRunner.java:767)
at org.testng.TestRunner.run(TestRunner.java:617)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
at org.testng.SuiteRunner.run(SuiteRunner.java:240)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1224)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1149)
at org.testng.TestNG.run(TestNG.java:1057)
at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:111)
at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:204)
at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:175)

Assertion Failed in Test 2
Assertion Failed in Test 2
Assertion Passed in Test 2

PASSED: test1 (Should have FAILED instead of PASSED)
PASSED: test2

===============================================

Revolutionize Your Business With Generative AI

From product design and software development to virtual agents, content creation, and reporting, GenAI is transforming business. Our AI experts help you unlock GenAI’s full potential and drive growth.

Let’s Get Started

Default test

Tests run: 2, Failures: 0, Skips: 0

===============================================

The disadvantage of Hard Assertion is that the test passed even though there was assertion failure which led to create customized error handlers which could fail the test as needed.

Soft Assertions

To deal with the disadvantage of Hard Assertions, customized error handler provided by TestNG is called Soft Assertion. Soft Assertions are the type of assertions that do not throw an exception when an assertion fails and would continue with the next step after assert statement. This is usually used when our test requires multiple assertions to be executed and the user want all of the assertions/codes to be executed before failing/skipping the tests.

Consider the below example:

public class Sample {

@Test
public void test1(){
SoftAssert sa= new SoftAssert();
sa.assertTrue(2<1);
System.out.println(“Assertion Failed”);
sa.assertFalse(1<2);
System.out.println(“Assertion Failed”);
sa.assertEquals(“Sample”, “Failed”);
System.out.println(“Assertion Failed”);
}
}

Output:
Assertion Failed
Assertion Failed
Assertion Failed

PASSED: test1

Even now the test PASSED instead of FAILED. The problem here is the test would not fail when an exception is not thrown. In order to achieve the desired result we need to call the assertAll() method at the end of the test which will collate all the exceptions thrown and fail the test if necessary.

Consider the below example.

public class Sample {

@Test

public void test1(){
SoftAssert sa= new SoftAssert();
sa.assertTrue(2<1);
System.out.println(“Assertion Failed”);
sa.assertFalse(1<2);
System.out.println(“Assertion Failed”);
sa.assertEquals(“Sample”, “Failed”);
System.out.println(“Assertion Failed”);
sa.assertAll();
}
}

Output:

Assertion Failed
Assertion Failed
Assertion Failed

FAILED: test1

TestNG, providing all the necessary features in a testing framework stands out when compared to others.