Mocking HTTP Clients in AEM Projects

The first thing I learned as an AEM developer was mvn clean install -P autoInstallBundle. The second thing was how to debug a remote Java process. The first allowed me to deploy updates I made to the bundle. The second allowed me to step through and debug the code. This was the normal cadence.

This blog is not about 90%. I’m sure you can find a lot of examples out there. This is about that 10%, in particular with HTTP clients. Right out of the box, AEM comes with the Apache HTTP Client. If you are running AEM under Java 11, its HttpClient is an option as well. Either way, you face the same problem. How do you mock constructors and static classes?

@Component(service = { Servlet.class })
@SlingServletResourceTypes(resourceTypes = NameConstants.NT_PAGE,
                           methods = HttpConstants.METHOD_GET,
                           selectors = "proxy",
                           extensions = "json")
@Designate(ocd = ProxyServlet.Configuration.class)
public final class ProxyServlet extends SlingSafeMethodsServlet {

    private static final long serialVersionUID = -2678188253939985649L;

    @Activate
    private transient Configuration configuration;

    @Override
    protected void doGet(final SlingHttpServletRequest request, final SlingHttpServletResponse response) throws IOException {

        response.setContentType("application/json");
        response.setCharacterEncoding("utf-8");

        final HttpUriRequest httpRequest = new HttpGet(this.configuration.uri());
        try (CloseableHttpClient httpClient = HttpClients.createDefault();
                CloseableHttpResponse httpResponse = httpClient.execute(httpRequest)) {

            httpResponse.getEntity()
                        .writeTo(response.getOutputStream());
        }
    }

    @ObjectClassDefinition
    public @interface Configuration {

        @AttributeDefinition String uri() default "https://randomuser.me/api/?results=1&inc=id,email,name,gender&seed=9579cb0f52986aab&noinfo";
    }
}

Static Mocking

@ExtendWith(MockitoExtension.class)
class ProxyServletTest {

    @Mock
    private ProxyServlet.Configuration configuration;
    @InjectMocks
    private ProxyServlet servlet;

    @ParameterizedTest
    @ValueSource(strings = { "https://someapi.com" })
    void doTest(final String uri) throws Exception {

        // mock osgi config
        doReturn(uri).when(this.configuration)
                     .uri();

        // mock http components
        final CloseableHttpResponse httpResponse = mock(CloseableHttpResponse.class);
        final HttpEntity httpEntity = mock(HttpEntity.class);
        doReturn(httpEntity).when(httpResponse)
                            .getEntity();

        final CloseableHttpClient httpClient = mock(CloseableHttpClient.class);
        final MockedConstruction.MockInitializer<HttpGet> mockInitializer = (mock, context) -> {
            assertEquals(uri, context.arguments().get(0));
            doReturn(httpResponse).when(httpClient)
                                  .execute(mock);
        };

        // mock http get and client construction
        try (MockedConstruction<HttpGet> construction = mockConstruction(HttpGet.class, mockInitializer);
                MockedStatic<HttpClients> mockedStatic = mockStatic(HttpClients.class)) {

            mockedStatic.when(HttpClients::createDefault)
                        .thenReturn(httpClient);

            // mock sling
            final SlingHttpServletRequest request = mock(SlingHttpServletRequest.class);
            final SlingHttpServletResponse response = mock(SlingHttpServletResponse.class);
            final ServletOutputStream outputStream = mock(ServletOutputStream.class);
            doReturn(outputStream).when(response)
                                  .getOutputStream();

            // execute
            this.servlet.doGet(request, response);

            // verify headers were set
            verify(response).setContentType("application/json");
            verify(response).setCharacterEncoding("utf-8");
            // verify uri was retreived from config
            verify(this.configuration).uri();
            // verify only one construction of http get
            assertEquals(1, construction.constructed()
                                        .size());
            // verify default client created
            mockedStatic.verify(HttpClients::createDefault);
            // verify get was executed by client
            verify(httpClient).execute(construction.constructed()
                                                   .get(0));
            // verify the entity was written to the output stream
            verify(httpEntity).writeTo(outputStream);
        }
    }
}

Writing Testable Code

@Component(service = { Servlet.class })
@SlingServletResourceTypes(resourceTypes = NameConstants.NT_PAGE,
                           methods = HttpConstants.METHOD_GET,
                           selectors = "proxy",
                           extensions = "json")
@Designate(ocd = ProxyServlet.Configuration.class)
@Accessors(fluent = true,
           chain = true,
           prefix = "supply")
public final class ProxyServlet extends SlingSafeMethodsServlet {

    private static final long serialVersionUID = -2678188253939985649L;

    @Activate
    private transient Configuration configuration;

    @Setter(AccessLevel.PACKAGE)
    private transient Function<String, HttpUriRequest> supplyHttpRequest = HttpGet::new;
    @Setter(AccessLevel.PACKAGE)
    private transient Supplier<CloseableHttpClient> supplyHttpClient = HttpClients::createDefault;

    @Override
    protected void doGet(final SlingHttpServletRequest request, final SlingHttpServletResponse response) throws IOException {

        response.setContentType("application/json");
        response.setCharacterEncoding("utf-8");

        final HttpUriRequest httpRequest = this.supplyHttpRequest.apply(this.configuration.uri());
        try (CloseableHttpClient httpClient = this.supplyHttpClient.get();
                CloseableHttpResponse httpResponse = httpClient.execute(httpRequest)) {

            httpResponse.getEntity()
                        .writeTo(response.getOutputStream());
        }
    }

    @ObjectClassDefinition
    public @interface Configuration {

        @AttributeDefinition String uri() default "https://randomuser.me/api/?results=1&inc=id,email,name,gender&seed=9579cb0f52986aab&noinfo";
    }
}

Here, I am leveraging Lombok. If you have read my previous post Writing Less Java Code in AEM with Sling Models & Lombok, you know I love it. It did shave off a few lines of boilerplate code. In this particular case, this is as far as I would go with refactoring. The functional interfaces that provide the HttpClient and HttpGet are good enough. Anything else would get too convoluted. Below is the unit test.

@ExtendWith(MockitoExtension.class)
class ProxyServletTest {

    @Mock
    private ProxyServlet.Configuration configuration;
    @InjectMocks
    private ProxyServlet servlet;

    @ParameterizedTest
    @ValueSource(strings = { "https://someapi.com" })
    void doTest(final String uri) throws IOException {

        // mock osgi config
        doReturn(uri).when(this.configuration)
                     .uri();

        // mock http components
        final CloseableHttpResponse httpResponse = mock(CloseableHttpResponse.class);
        final HttpEntity httpEntity = mock(HttpEntity.class);
        doReturn(httpEntity).when(httpResponse)
                            .getEntity();

        final CloseableHttpClient httpClient = mock(CloseableHttpClient.class);
        final HttpUriRequest httpGet = mock(HttpUriRequest.class);
        doReturn(httpResponse).when(httpClient)
                              .execute(httpGet);

        // mock sling
        final SlingHttpServletRequest request = mock(SlingHttpServletRequest.class);
        final SlingHttpServletResponse response = mock(SlingHttpServletResponse.class);
        final ServletOutputStream outputStream = mock(ServletOutputStream.class);
        doReturn(outputStream).when(response)
                              .getOutputStream();

        // execute with suppliers
        this.servlet.httpClient(() -> httpClient)
                    .httpRequest(value -> {
                        assertEquals(uri, value);
                        return httpGet;
                    })
                    .doGet(request, response);

        // verify headers were set
        verify(response).setContentType("application/json");
        verify(response).setCharacterEncoding("utf-8");
        // verify uri was retreived from config
        verify(this.configuration).uri();
        // verify the entity was written to the output stream
        verify(httpEntity).writeTo(outputStream);
    }
}

In-Memory HTTP Server

There are lots actually. To name a few:

• Mock Web Server
• WireMock
• MockServer
• Jadler

I’m not here to compare and contrast the different implementations. That would be up to you and the set of features you think you need. For my example, I am happy with Mock Web Server primarily because it is lightweight and has a simple queuing mechanism. Let us see how fast I can mock this up.

@ExtendWith({ MockitoExtension.class, AemContextExtension.class })
class MockServerProxyServletTest {

    @Mock
    private ProxyServlet.Configuration configuration;
    @InjectMocks
    private ProxyServlet servlet;

    private MockWebServer mockWebServer;

    @BeforeEach
    void beforeEach() throws IOException {

        this.mockWebServer = new MockWebServer();
        this.mockWebServer.start();
    }

    @AfterEach
    void afterEach() throws IOException {

        this.mockWebServer.shutdown();
    }

    @Test
    void doTest(final AemContext context) throws IOException {

        // setup
        final String url = this.mockWebServer.url("/").toString();
        doReturn(url).when(this.configuration).uri();
        this.mockWebServer.enqueue(new MockResponse().setBody("hello, world!"));

        // execute
        this.servlet.doGet(context.request(), context.response());

        // assert
        final String result = context.response().getOutputAsString();
        assertEquals("hello, world!", result);
    }
}

In the end, I wind up using all three testing frameworks. Mockito to mock the OSGi configuration. AEM Mocks to mock the Sling HTTP serverlet request/response. Last, Mock Web Server to mock the remote REST API. It is all so much simpler. You no longer have to worry about managing so many mocks. In fact, your unit test is now client agnostic. Even if you go back to your proxy and switch out the Apache HTTP Client for the Java 11 HTTP Client, it will still work without a rewrite to the unit test.

Conclusion

mvn -B archetype:generate \
    -D archetypeGroupId=com.adobe.aem \
    -D archetypeArtifactId=aem-project-archetype \
    -D archetypeVersion=26 \
    -D appTitle="My Site" \
    -D appId="mysite" \
    -D groupId="com.mysite" \
    -D aemVersion=6.5.0

Make the following changes to pom.xml

<!-- 1. update version of junit-bom -->
<dependency>
    <groupId>org.junit</groupId>
    <artifactId>junit-bom</artifactId>
    <!--<version>5.6.2</version>-->
    <version>5.7.0</version>
    <type>pom</type>
    <scope>import</scope>
</dependency>

<!-- 2. replace mockito-core with mockito-inline -->
<dependency>
    <groupId>org.mockito</groupId>
    <!--<artifactId>mockito-core</artifactId>
    <version>3.3.3</version>-->
    <artifactId>mockito-inline</artifactId>
    <version>3.11.2</version>
    <scope>test</scope>
</dependency>

Make the following changes to core/pom.xml

<!-- 1. add the following two -->
<dependency>
    <groupId>org.projectlombok</groupId>
    <artifactId>lombok</artifactId>
    <version>1.18.20</version>
    <scope>provided</scope>
</dependency>
<dependency>
    <groupId>com.squareup.okhttp3</groupId>
    <artifactId>mockwebserver</artifactId>
    <version>4.9.1</version>
    <scope>test</scope>
</dependency>

<!-- 2. remove junit-addons -->
<!-- <dependency>
    <groupId>junit-addons</groupId>
    <artifactId>junit-addons</artifactId>
    <scope>test</scope>
</dependency>-->

<!-- 3. replace mockito-core with mockito-inline -->
<dependency>
    <groupId>org.mockito</groupId>
    <!--<artifactId>mockito-core</artifactId>-->
    <artifactId>mockito-inline</artifactId>
    <scope>test</scope>
</dependency>