As all AEM Developers know, AEM, and more-so the underlying JCR, has limitations on how many children a single parent can have before we start to see performance issues.  First in the UI, and then more critically in the ability to look-up and manage content nodes themselves.  When designing for a net-new AEM website, this can be planned around, however, many times when dealing with a migration, there may be requirements to both keep existing links in-tact, which may put us in a situation where a large flat list, at least as exposed to the end-user, is unavoidable.

In this post, I will not go down the rabbit hole of whether 500, 1000, or 10000 children would start to have performance issues (although I do certainly have opinions here), and instead will focus on ways to allow for a single parent, to appear to have many more than the maximum number of children.

Enter Sling Mappings

Now, Sling mappings are not new for most, and commonly used for things like stripping the initial prefix (/content/<project>) away from outgoing URLs, but these mappings have additional untapped power which documentation, in my opinion, does not clearly emphasize.  Before we jump in, lets talk about a few key features of sling mappings as a whole.

Sling mappings are configured based on regular expressions, and often inherited by the node names themselves, within /etc/map.  Within each node, there are optional properties we can add to define how mappings should behave.  For example, if we are wanting to introduce a higher complexity regular expression, we cannot rely on the node name alone (characters required for regex mappings are restricted from being used as a node name).  This means that when we need to define a mapping which does NOT align with the name of the node, we can insert the following property (single value only):

Now, the above is a relatively straight-forward example, but we’re here to learn about mappings, not regular expressions.  Do note, I am using a capture group here – multiple capture groups are certainly also supported, but for ease of this post, am keeping it quite simple.

The other item to note on the above is that I have added this node as a root node in the /etc/map/localhost tree.  As such, the mapping begins from the root element.  If we were to structure the /etc/map node differently, the start-point of that regular expression would change as well, for example:

In the above scenario, we’d structure the sling:match accordingly, as the content and wknd nodes would have already been matched against:

Well, that is half of the issue, is being able to capture the incoming request.  Next is to actually map it to a different internal resource.  This is where the “magic” comes in – on the same node, add an additional property, sling:internalRedirect, as follows:

In the above, we’re really just finding content in the same location it was requested.  The $1 in the internalRedirect is referencing the (.*) of the sling:match property, and saying that whatever was matched there, should first look at the location requested to find the resource, technically speaking, we have not really mapped anything at this point.   Now, what if the magazine section above is a long-standing blog, which has all urls published under the same /magazine root.  If organized 1:1 with the JCR structure, this would quickly cause reason for concern!  This is where the often overlooked ARRAY support of the internalRedirect comes to save the day.  From here, we can supply alternative locations to look for the same page.  For example, if we were to archive posts on a yearly basis, as in 2023 posts would live in the JCR structure /content/wknd/us/en/magazine/2023/<post>, while being publicly available still under /content/wknd/us/en/magazine/<post>, we could configure that using the “internalRedirect” array, as follows:

Now, when a request matches, it will first search under the magazine root folder, and if it does not find a resource there, will look to the 2023 folder, and so-on until all items of the Array have been searched against.  It does search from the top, down, so this means that order is important, especially if same-name nodes are expected.  If the entire list is exhausted and there is no resource found, then the normal 404 behavior is observed.

Now, lets assume we’ve got the following content (note the 2023 and 2024 folders added, with children):

With the forward mapping applied, we should be able to access the “san-diego-surf-2023” and “san-diego-surf-2024” under the /magazine folder directly.  Lets test it out!

Awesome.  We’re accessing the content without using the “2023” folder in the URL!

Great!  Well this solves the problem of accessing the content, however, if we look at a page which links to this content, we still are seeing an issue….

Looks like we still have one other piece of the puzzle to solve for.  One other important thing to know about regular expressions (with capture groups) is that when including in a sling mapping, they will only work in one direction.  Since we have configured it to capture on the request side (match) it will only handle the logic of handling the request.  In order to create a rewrite of outgoing URLs, we will need to also handle the reverse mapping.  For this, I use the suffix “-reverse” with the same node name as initial mapping for consistency.

So, next step is to copy the node we just created, and paste it as a sibling (under same parent).  We’ll then reverse the regular expression logic to have capture groups in the “sling:internalRedirect” property, and the references in the sling mapping property, as follows:

This behaves in much the same way as the other mapping node, but because we have swapped the matching logic for the regular expression, it will now be applied in the url rewriting or output sling mapping as well.

Now, lets take a look back at our links to this content, and….

We’re golden!  Content is hosted under multiple JCR Nodes while being exposed under a single parent to the end-user.

I would like to add a disclaimer here: while this method is certainly a suitable work-around, a much more desirable implementation would have the JCR structure more closely matching the expected sitemap.

Hope this quick demonstration has been helpful!

Reference:

• https://sling.apache.org/documentation/the-sling-engine/mappings-for-resource-resolution.html

Large scale data breaches and critical security vulnerabilities have companies thinking about security more than ever.  Many developers are familiar with the OWASP top 10 (https://owasp.org/www-project-top-ten/) and there are already many resources on generic mitigation for these vulnerabilities.  Instead in this series, I cover security issues and mitigations specific to AEM.  Today’s topic is Sling Resolution.

Previous posts in series:

• Part 1 – XSS

Sling Resolution Vulnerabilities

Dispatcher Allow rulesets are one of the largest risk vectors within AEM because of default Sling Resolution behavior.  Spend a significant amount of your security testing time here, because this is the most common way for malicious actors to find other vulnerabilities, compromise data within the system, and achieve remote code execution.

Remote code execution (RCE) is the highest impact concern, as an attacker may gain a broad range of further exploit options.  This can include breaking the system, compromising data, stealing user credentials, and creating phishing pages on legitimate domains.

In creating defensive measures, it’s useful to know how RCE can occur in the first place.  RCE is possible if hacker can get access to the OSGI console, writing to /apps or (in insecure systems) /content, Querybuilder, Groovy console, ACS AEM Tools, or WebDAV.  It is best to completely disallow access to these features on Production publishers for any user.  From there, utilize Adobe’s best practices of Deny all paths and then Allow necessary paths.   Most of these potential vulnerabilities should be mitigated by the latest version of the dispatcher rules present in Adobe’s AEM Project Archetype, but it’s worth confirming in your own dispatcher as well.

The following is not a comprehensive list of vulnerabilities but it will give you an idea of what to start looking for.

• Sometimes developers write deny rules that are too rigid, or allow rules that are not rigid enough. For example, a request to /bin/querybuilder.json/test.css may still return the AEM querybuilder page.  Instead of using:

{ /type “deny” /path “/bin/querybuilder” }

Utilize wildcard (*) matching for child and extension paths. E.g.

{ /type “deny” /path “/bin/querybuilder*” }

• Adding selectors, URL Params, multiple slashes to the URL may change how Sling and the Dispatcher interpret the request. Many vulnerabilities rely on exposing the JCR content structure through .json rendering requests.
• If a malicious user gains access to the content structure, they can often mine usernames, credentials stored in plain text, PII, exposed admin pages, or clues about other vulnerabilities within the system. All it may take, is the attacker finding 1 username where they can brute-force the password, to have a much bigger data breach.
• By default, numbered (-1,1,2,3…), “children”, and “infinity” selectors in a request are interpreted by Sling to display the current content path and Sling resources under it as a JSON response.  This functionality should not be available over the dispatcher.
• If you have enabled Sling Model Exporters for your Sling Model objects, ensure no sensitive data or properties are exposed by accident in the response.

Putting into Practice

As AEM increases in market share, so too does the incentive for hackers to find exploits, so it’s very important to test and stay up to date on the latest recommended allow rules.  Test any rules that deviate from the standard ruleset very thoroughly.

Need some practical examples?  Below I’ve provided some basic URLs to test on your dispatcher.  Over the public internet these should return a 404 or redirect appropriately.  If not, that represents a vulnerability within your application.

• /content.infinity.json
• /.1.json
• /content.5.json
• /content.ext.json
• /content.children.json/test.css
• /etc.-1.json
• /content.html/test.png/test.1.json
• /bin/querybuilder.json/test.css
• /bin/querybuilder.json?a.html
• /bin/querybuilder.json.css
• /bin/querybuilder.json.servlet.css
• //bin//querybuilder.json
• /system/console/bundles?.css

For more information on how Perficient can help you achieve your AEM Security goals and implement your dream digital experiences, we’d love to hear from you.

Contact Perficient to start your journey.

AEM has a vast infrastructure, and it requires time to hold expertise on it. Every expert was once a beginner, so you are never too late to start and have fun on AEM. In this blog, we will focus on the developers’ perspective of AEM and how one can start as an AEM developer. We will go through some of the topics which will help you station your pillars on AEM and get the crux of it.

To begin with AEM, you should have a basic understanding of:

• Java – Servlets, JSP, Collections, OPPs concepts.
• JS
• HTML/CSS

Note: This tutorial will focus on the latest AEM – 6.5.

Firstly, What is AEM?

AEM is a Content Management System (CMS) that provides solutions to build websites, mobile apps, and forms. What makes it different from other CMSs is its ability to build content and deliver them in a single platform using various marvelous features such as Dam, Forms, Communities, Cloud, etc.

Note: In this tutorial, we will explore the necessary steps to complete a given functionality. For complete and more details on the related topic, visit the links at the end of this blog.

Go through the following diagram to get a glimpse of the topics we are going to discuss in the blog:

So, Let’s begin the groove.

Install AEM on your Local System:

For trying your hands-on, you will need a local AEM instance setup. For that, you will require:

• AEM quickstart jar and a license file
• Java 8 or 11 installed in our system

Note: In this tutorial, we are focusing on the 6.5 version of AEM. Recently, it has been the most widely used version with all the available features.

Installing AEM is the first step in, to begin with, AEM. Since we are focusing on the developers’ perspective, we will set up an author instance in our system. Now, to start with, there are two modes in which you can set up your instance:

Author: Instance where you author or design your site. Here you can write your pages, assets, experience fragments, templates, configurations, etc.

Publish: Instance where all the content created in the author instance is deployed after publishing. This is the actual instance where the end user will interact with your site.

Now, let’s set up an author instance, as we will create all our content here, and the developer’s work revolves around this instance.

Follow the steps to set up your instance:

• Copy and paste the AEM quickstart jar and the license file into a dedicated folder.
• Rename the jar file to aem-author-p4502.
• Double-click to run the jar file. It will take around 8-10 mins to run.

After it is done thoroughly, you will notice AEM running on your browser at the 4502 port number, as it is specified in the file name as well(aem-author-p4502). Login with admin/admin, and you are good to go and explore AEM.

Please refer to the link below for more details on setting up your local instance:

https://experienceleague.adobe.com/docs/experience-manager-learn/foundation/development/set-up-a-local-aem-development-environment.html?lang=en

Walkthrough the AEM Environment:

There are many areas in AEM supporting a wide range of functionalities. Go ahead and try your hands on various features.

Step into the Author’s shoes

Who is an Author?

The Author is the one who generates content for our pages by authoring and configuring the components. All the blocks of code(components) are combined and integrated by the Author to develop a simple web page. In short, the Author is the mediator between the user and our code.

To develop a component, it is important to understand how the Author will configure the elements on the page and how they will behave and look at the other end.

Please go through the following tutorials to learn authoring pages and components:

Follow this video for an overview of AEM

Go through this document to understand more about authoring in a brief

Editable templates

Once you have understood the authoring pages and components on an AEM page, it’s time to dig deeper and understand pages and their structure.

Any website has a template upon which it is built. Similarly, AEM provides a means by which we can create our templates and edit them dynamically. A template can be created or edited using the template console, as shown in the screenshot below:

Follow the link below to Learn about authoring pages:

https://experienceleague.adobe.com/docs/experience-manager-learn/sites/page-authoring/template-editor-feature-video-use.html?lang=en

After you have created a template, you can create any number of pages from that template, and it will serve as a structure for your page.

This was just an introduction to understanding AEM’s environment and its basic concepts. Being an AEM developer requires a lot more steps. Let’s get into the first few steps that will help you run swiftly.

Time to Develop

Once you are set up in the environment and begin to understand how AEM works as a content management system, it’s time to see how an AEM developer creates custom components and extend the existing component.

Before we move forward, let’s understand a few key concepts used by an AEM developer day in and day out. Following are some of them:

Sightly: This is a scripting language used for rendering the content on the page. Previously, JSP was used for generating the contents. Still, newer versions of AEM sightly became the recommended one for obvious reasons such as code separation, protection against cross-site scripting, etc. Sightly is similar to HTML, with a few new tags helping integrate with AEM concepts.

crx/de Lite: This is a developer console that helps the developer quickly make the changes and verify them on the page. It contains the actual tree structure of the instance organized sophisticatedly. Every node has its purpose of serving. For example, all the components for a project are stored under the app’s node, and all pages are stored under the content node.

http://localhost:4502/crx/de/index.jsp

We will learn about more such concepts as we move ahead and explore AEM further.

At this point, you understand how pages are created and authored in AEM and how they serve the content to the end user. Now, let’s understand how these components are designed and how are they made available to the authors.

First, let’s begin by understanding what a component in AEM is?

A component is a working piece of code encapsulated to serve a specific web page area. We can use this component anywhere on our pages and customize its behaviors as required using dialogs.

Creating the First AEM component

For the first component, we will start by reusing the existing component from the we-retail site (Sample site, which is shipped with AEM 6.5).

• Go to crx/de
• Navigate through the path apps -> weretail -> components -> content. The components folder contains all the components created for we-retail site. We can further sub-classify them into various categories such as content, structure, etc.
• Open the button component. You will see the following nodes as shown in the image below:

For now, let’s focus on button.html and cq:dialog.

button.html: It contains the actual HTML code rendered when the component is added to the page.

cq:dialog: This node forms the node’s dialog, which is served to the Author as an authoring dialog to configure the fields. These values are stored in a node structure under /content node. For example, if you added the button component under men page of we.retail site then these values will be stored under the path /content/we-retail/us/en/men/jcr:content/root/responsivegrid/button_1022012327 as shown in the image below:

Here, I labeled my button as Sample. But, just saving them is not our motive. We have to get these values and render them too. Thankfully, we have Sightly, which makes our job easy. Look at the underlined sentence in button.html in the previous screenshot.

${properties.label}

This is a sightly way of accessing the values from the dialog. So, here properties are implicit objects that AEM provides to access values from a dialog using a dot operator. There are many such implicit objects we can use to get and process the values. They are also known as Global Objects. Please go through Sightly’s documentation to understand more about them.

Here, we are accessing the property/field label. As shown in the screenshot above, the value of the button’s label is stored in the label field. So, we are fetching that value and rendering it on the button using our HTML code. The property’s name is determined by the dialog, as shown below. So whatever name is specified in the dialog, values are stored with that name, and we can access them using the same name.

You will notice that everything is interconnected. It might be pretty confusing to get it in one go, but as you move ahead, this will be just a piece of cake for you.

Try adding this component to any of the we-retail site pages and experimenting with the component.

Create an AEM Project Using the Latest Archetype

We can experiment and try our hands out in crx/de to understand the functioning of AEM. An AEM developer uses crx/de just for testing, quick fixes, and debugging; hence, we cannot develop the entire project on crx/de. To create any working AEM site, we need to create a project and deploy it in higher environments.

Let’s create an actual AEM project.

Following are the prerequisites for setting up any AEM project: For any AEM project to start working, the following things should be installed in your system:

1. Maven
2. Java 8 or higher
3. Git and Npm
4. Any IDE, IntelliJ or Eclipse
5. Adobe Public Maven Repository: This maven2 repository provides access to public artifacts used in conjunction with developing Java-based applications on Adobe frameworks. Just copy the content of this link in your C:\Users\My user.m2\settings.xml directory or /.m2/settings.xml in your mac.

Refer to the following link for more information about this: https://experienceleague.adobe.com/docs/experience-cloud-kcs/kbarticles/KA-17454.html?lang=en

Now, let’s create an AEM project using the maven command.

Go to the local drive location where you want to save your project and open the command prompt from there. Alternatively, you right-click on the desired location on the drive and open Git bash. Now, run the following command:

mvn -B org.apache.maven.plugins:maven-archetype-plugin:3.2.1:generate \
    -D archetypeGroupId=com.adobe.aem \
    -D archetypeArtifactId=aem-project-archetype \
    -D archetypeVersion=35 \
    -D appTitle="WKND Sites Project" \
    -D appId="wknd" \
    -D groupId="com.adobe.aem.guides" \
    -D artifactId="aem-guides-wknd" \
    -D package="com.adobe.aem.guides.wknd" \
    -D version="0.0.1-SNAPSHOT" \
    -D aemVersion="cloud"

As soon as the command finishes its execution, you will see a project being created at the desired location.

This was a straightforward way to kickstart any project. Once you have got a hold of AEM, this is the only step required to create and begin on any project, but since you are a beginner, please go through the following document and understand the project structure and modules of any AEM project.

https://experienceleague.adobe.com/docs/experience-manager-learn/getting-started-wknd-tutorial-develop/project-archetype/project-setup.html?lang=en

Please run the following command to install the project on your local instance:

mvn clean install -PautoInstallPackage -PautoInstallBundle

Please go to the following link if you want to understand more about the command:

https://experienceleague.adobe.com/docs/experience-manager-core-components/using/developing/archetype/overview.html?lang=en

Once the project has been built successfully, open the project in your IDE(IntelliJ or Eclipse)

To enable your IDE to understand AEM modules and concepts, you need to install plugins in your IDE.

Eclipse: Install AEM Developer Tools for Eclipse. Follow the link for a step-by-step procedure – https://experienceleague.adobe.com/docs/experience-manager-64/developing/devtools/aem-eclipse.html?lang=en

IntelliJ: Install AEM IDE Tooling 4 IntelliJ. Just go to File -> Settings -> Plugins in your IDE. Search the plugin and install it.

These plugins provide an easy way to manage and develop any AEM project by strengthening the features IDE provides. It is always good to have these plugins installed in your IDE for faster development and effortless mappings.

Creating a Custom Component

AEM allows you to create custom components and extend the existing core component, which is shipped with AEM. Here, we will see how to create custom components and concepts such as sightly, sling models, and dialogs.

Please follow the following tutorial on how to create a custom component:

https://experienceleague.adobe.com/docs/experience-manager-learn/getting-started-wknd-tutorial-develop/project-archetype/custom-component.html?lang=en

Note: The tutorial refers to another git repository, but you can also create this component in your newly created project.

While going through the tutorial, you must have come across a few new concepts like sightly and sling models. You don’t need to understand them in one go. So, no worries, you will get them as you work with them more. Again, AEM is not a very simple thing to understand, but once you start, there is no looking back.

To understand more about the new concepts, refer to the following links:

Sling Models

Sightly

As you start understanding these concepts, everything will make sense, and you will be able to match the pieces together.

Many More To Go

These were just baby steps to an understanding of AEM and beginning with it. AEM is not, that’s all. We haven’t even explored 10% of it. But the concepts we went through earlier will help you get hold of AEM so that you can explore further on it. It was ABC for AEM. To name a few, the following are a few more concepts that will help you understand and develop a site in AEM:

• Sling Servlets
• Client Libraries
• Extending Core Components
• Resource Mergers
• Workflows

So, Don’t Wait Up. Get Set and Go Ahead on AEM…

Let’s talk about extract, transform, and load, also known as ETL. If you are an AEM professional, this is something you have previously dealt with. It could be something along the lines of products, user bios, or store locations.

The extract and transform parts may differ depending on your source and requirements. The loading part is almost always going to be into AEM. While there may be a few ways to do that, let us talk about what is there for you out-of-the-box.

Sling Post Servlet

As an AEM developer, the Sling Post Servlet is something you should be familiar with. In particular, there is an import operation. This allows us to do the following:

curl -L https://www.boredapi.com/api/activity | \
curl -u admin:admin \
     -F":contentFile=@-" \
     -F":nameHint=activity" \
     -F":operation=import" \
     -F":contentType=json" \
     http://localhost:4502/content/mysite/us/en/jcr:content/root/container

You can run this many times. You will get activity_* nodes under /content/mysite/us/en/jcr:content/root/container. This assumes that the source is already in the format you desire. Meaning you have already done the transform part.

And the import operation can deal with more complex JSON structures, even XML. Here is a possible output that could be provided by a transform:

{
    "jcr:primaryType": "cq:Page",
    "jcr:content": {
        "jcr:primaryType": "cq:PageContent",
        "jcr:title": "My Page",
        "sling:resourceType": "mysite/components/page",
        "cq:template": "/conf/mysite/settings/wcm/templates/page-content",
        "root": {
            "jcr:primaryType": "nt:unstructured",
            "sling:resourceType": "mysite/components/container",
            "layout": "responsiveGrid",
            "container": {
                "jcr:primaryType": "nt:unstructured",
                "sling:resourceType": "mysite/components/container"
            }
        }
    }
}

Save this to a file named mypage.json and run the following curl command.

curl -u admin:admin \
     -F":name=my-page" \
     -F":contentFile=@mypage.json" \
     -F":operation=import" \
     -F":contentType=json" \
     -F":replace=true" \
     -F":replaceProperties=true" \
     http://localhost:4502/content/mysite/us/en

And boom! You have an instant page. This time instead of the :nameHint I used the :name and :replace properties. Running this command again will update the page. The loading part becomes really trivial and you need only worry about extracting and transforming.

Finding the Content Importer Service

While the Sling Post Servlet is well documented, its internal implementation is not. Luckily, it is open source. You won’t have to do any decompiling today! Let’s read the doPost function of the implementation. There are too many goodies we could dive into. Let’s stay focused. We are looking for the import operation. Did you find it?

You should have wound up at the doRun function of the ImportOperation.java. This is where all those request parameters from the curl commands above come into play. Go further down. You will find a call to ContentImporter.importContent(Node, String, String, InputStream, ImportOptions, ContentImportListener). Can you find its implementation?

Finally, you should have wound up on the DefaultContentImporter.java implementation. An OSGi component that implements the ContentImporter interface.

Programmatically Using the Content Importer

Yes! Programatically doing things. Now that we know that the ContentImporter is available as an OSGi component all we need is:

@Reference
private ContentImporter contentImporter;

And assuming you have your content via InputStream we can import the content under any node. As an example, I am using the SimpleServlet generated as part of the AEM Maven Archtype. I’m using Lombok to speed things up a little.

@Component(service = { Servlet.class })
@SlingServletResourceTypes(resourceTypes = "mysite/components/page", methods = HttpConstants.METHOD_GET, extensions = "txt")
@ServiceDescription("Simple Demo Servlet")
@Slf4j
public class SimpleServlet extends SlingSafeMethodsServlet {

    private static final long serialVersionUID = 1L;

    @Reference
    private ContentImporter contentImporter;

    @Override
    protected void doGet(final SlingHttpServletRequest request, final SlingHttpServletResponse response)
            throws
            IOException {

        final MyContentImportListener contentImportListener = new MyContentImportListener();
        final Node node = request.getResource().adaptTo(Node.class);

        if (node != null) {

            final MyImportOptions importOptions = MyImportOptions.builder()
                                                                 .overwrite(true)
                                                                 .propertyOverwrite(true)
                                                                 .build();
            try (InputStream inputStream = IOUtils.toInputStream("{\"foo\":\"bar\"}", StandardCharsets.UTF_8)) {
                this.contentImporter.importContent(node, "my-imported-structure", "application/json", inputStream, importOptions, contentImportListener);
            } catch (final RepositoryException e) {
                log.error(e.getMessage(), e);
            }
        }

        response.setContentType("text/plain");
        response.getWriter().println(contentImportListener);

    }

    @Builder
    @Getter
    private static final class MyImportOptions extends ImportOptions {

        private final boolean checkin;
        private final boolean autoCheckout;
        private final boolean overwrite;
        private final boolean propertyOverwrite;

        @Override
        public boolean isIgnoredImportProvider(final String extension) { return false; }
    }

    @Getter
    @ToString
    private static final class MyContentImportListener implements ContentImportListener {

        private final com.google.common.collect.Multimap<String, String> changes =
                com.google.common.collect.ArrayListMultimap.create();

        @Override
        public void onReorder(final String orderedPath, final String beforeSibbling) {this.changes.put("onReorder", String.format("%s, %s", orderedPath, beforeSibbling)); }

        @Override
        public void onMove(final String srcPath, final String destPath) { this.changes.put("onMove", String.format("%s, %s", srcPath, destPath)); }

        @Override
        public void onModify(final String srcPath) { this.changes.put("onModify", srcPath); }

        @Override
        public void onDelete(final String srcPath) { this.changes.put("onDelete", srcPath); }

        @Override
        public void onCreate(final String srcPath) { this.changes.put("onCreate", srcPath); }

        @Override
        public void onCopy(final String srcPath, final String destPath) { this.changes.put("onCopy", String.format("%s, %s", srcPath, destPath)); }

        @Override
        public void onCheckin(final String srcPath) { this.changes.put("onCheckin", srcPath); }

        @Override
        public void onCheckout(final String srcPath) { this.changes.put("onCheckout", srcPath); }
    }
}

Conclusion

A Simple Example

Let us use this servlet as a real simple example.

@Component(service = { Servlet.class })
@SlingServletResourceTypes(resourceTypes = NameConstants.NT_PAGE,
                           methods = HttpConstants.METHOD_GET,
                           selectors = "proxy",
                           extensions = "json")
@Designate(ocd = ProxyServlet.Configuration.class)
public final class ProxyServlet extends SlingSafeMethodsServlet {

    private static final long serialVersionUID = -2678188253939985649L;

    @Activate
    private transient Configuration configuration;

    @Override
    protected void doGet(final SlingHttpServletRequest request, final SlingHttpServletResponse response) throws IOException {

        response.setContentType("application/json");
        response.setCharacterEncoding("utf-8");

        final HttpUriRequest httpRequest = new HttpGet(this.configuration.uri());
        try (CloseableHttpClient httpClient = HttpClients.createDefault();
                CloseableHttpResponse httpResponse = httpClient.execute(httpRequest)) {

            httpResponse.getEntity()
                        .writeTo(response.getOutputStream());
        }
    }

    @ObjectClassDefinition
    public @interface Configuration {

        @AttributeDefinition String uri() default "https://randomuser.me/api/?results=1&inc=id,email,name,gender&seed=9579cb0f52986aab&noinfo";
    }
}

@ExtendWith(MockitoExtension.class)
class ProxyServletTest {

    @Mock
    private ProxyServlet.Configuration configuration;
    @InjectMocks
    private ProxyServlet servlet;

    @ParameterizedTest
    @ValueSource(strings = { "https://someapi.com" })
    void doTest(final String uri) throws Exception {

        // mock osgi config
        doReturn(uri).when(this.configuration)
                     .uri();

        // mock http components
        final CloseableHttpResponse httpResponse = mock(CloseableHttpResponse.class);
        final HttpEntity httpEntity = mock(HttpEntity.class);
        doReturn(httpEntity).when(httpResponse)
                            .getEntity();

        final CloseableHttpClient httpClient = mock(CloseableHttpClient.class);
        final MockedConstruction.MockInitializer<HttpGet> mockInitializer = (mock, context) -> {
            assertEquals(uri, context.arguments().get(0));
            doReturn(httpResponse).when(httpClient)
                                  .execute(mock);
        };

        // mock http get and client construction
        try (MockedConstruction<HttpGet> construction = mockConstruction(HttpGet.class, mockInitializer);
                MockedStatic<HttpClients> mockedStatic = mockStatic(HttpClients.class)) {

            mockedStatic.when(HttpClients::createDefault)
                        .thenReturn(httpClient);

            // mock sling
            final SlingHttpServletRequest request = mock(SlingHttpServletRequest.class);
            final SlingHttpServletResponse response = mock(SlingHttpServletResponse.class);
            final ServletOutputStream outputStream = mock(ServletOutputStream.class);
            doReturn(outputStream).when(response)
                                  .getOutputStream();

            // execute
            this.servlet.doGet(request, response);

            // verify headers were set
            verify(response).setContentType("application/json");
            verify(response).setCharacterEncoding("utf-8");
            // verify uri was retreived from config
            verify(this.configuration).uri();
            // verify only one construction of http get
            assertEquals(1, construction.constructed()
                                        .size());
            // verify default client created
            mockedStatic.verify(HttpClients::createDefault);
            // verify get was executed by client
            verify(httpClient).execute(construction.constructed()
                                                   .get(0));
            // verify the entity was written to the output stream
            verify(httpEntity).writeTo(outputStream);
        }
    }
}

@Component(service = { Servlet.class })
@SlingServletResourceTypes(resourceTypes = NameConstants.NT_PAGE,
                           methods = HttpConstants.METHOD_GET,
                           selectors = "proxy",
                           extensions = "json")
@Designate(ocd = ProxyServlet.Configuration.class)
@Accessors(fluent = true,
           chain = true,
           prefix = "supply")
public final class ProxyServlet extends SlingSafeMethodsServlet {

    private static final long serialVersionUID = -2678188253939985649L;

    @Activate
    private transient Configuration configuration;

    @Setter(AccessLevel.PACKAGE)
    private transient Function<String, HttpUriRequest> supplyHttpRequest = HttpGet::new;
    @Setter(AccessLevel.PACKAGE)
    private transient Supplier<CloseableHttpClient> supplyHttpClient = HttpClients::createDefault;

    @Override
    protected void doGet(final SlingHttpServletRequest request, final SlingHttpServletResponse response) throws IOException {

        response.setContentType("application/json");
        response.setCharacterEncoding("utf-8");

        final HttpUriRequest httpRequest = this.supplyHttpRequest.apply(this.configuration.uri());
        try (CloseableHttpClient httpClient = this.supplyHttpClient.get();
                CloseableHttpResponse httpResponse = httpClient.execute(httpRequest)) {

            httpResponse.getEntity()
                        .writeTo(response.getOutputStream());
        }
    }

    @ObjectClassDefinition
    public @interface Configuration {

        @AttributeDefinition String uri() default "https://randomuser.me/api/?results=1&inc=id,email,name,gender&seed=9579cb0f52986aab&noinfo";
    }
}

@ExtendWith(MockitoExtension.class)
class ProxyServletTest {

    @Mock
    private ProxyServlet.Configuration configuration;
    @InjectMocks
    private ProxyServlet servlet;

    @ParameterizedTest
    @ValueSource(strings = { "https://someapi.com" })
    void doTest(final String uri) throws IOException {

        // mock osgi config
        doReturn(uri).when(this.configuration)
                     .uri();

        // mock http components
        final CloseableHttpResponse httpResponse = mock(CloseableHttpResponse.class);
        final HttpEntity httpEntity = mock(HttpEntity.class);
        doReturn(httpEntity).when(httpResponse)
                            .getEntity();

        final CloseableHttpClient httpClient = mock(CloseableHttpClient.class);
        final HttpUriRequest httpGet = mock(HttpUriRequest.class);
        doReturn(httpResponse).when(httpClient)
                              .execute(httpGet);

        // mock sling
        final SlingHttpServletRequest request = mock(SlingHttpServletRequest.class);
        final SlingHttpServletResponse response = mock(SlingHttpServletResponse.class);
        final ServletOutputStream outputStream = mock(ServletOutputStream.class);
        doReturn(outputStream).when(response)
                              .getOutputStream();

        // execute with suppliers
        this.servlet.httpClient(() -> httpClient)
                    .httpRequest(value -> {
                        assertEquals(uri, value);
                        return httpGet;
                    })
                    .doGet(request, response);

        // verify headers were set
        verify(response).setContentType("application/json");
        verify(response).setCharacterEncoding("utf-8");
        // verify uri was retreived from config
        verify(this.configuration).uri();
        // verify the entity was written to the output stream
        verify(httpEntity).writeTo(outputStream);
    }
}

@ExtendWith({ MockitoExtension.class, AemContextExtension.class })
class MockServerProxyServletTest {

    @Mock
    private ProxyServlet.Configuration configuration;
    @InjectMocks
    private ProxyServlet servlet;

    private MockWebServer mockWebServer;

    @BeforeEach
    void beforeEach() throws IOException {

        this.mockWebServer = new MockWebServer();
        this.mockWebServer.start();
    }

    @AfterEach
    void afterEach() throws IOException {

        this.mockWebServer.shutdown();
    }

    @Test
    void doTest(final AemContext context) throws IOException {

        // setup
        final String url = this.mockWebServer.url("/").toString();
        doReturn(url).when(this.configuration).uri();
        this.mockWebServer.enqueue(new MockResponse().setBody("hello, world!"));

        // execute
        this.servlet.doGet(context.request(), context.response());

        // assert
        final String result = context.response().getOutputAsString();
        assertEquals("hello, world!", result);
    }
}

Note On Dependencies

In order to make the examples above work, I used an AEM Maven Archetype Project

mvn -B archetype:generate \
    -D archetypeGroupId=com.adobe.aem \
    -D archetypeArtifactId=aem-project-archetype \
    -D archetypeVersion=26 \
    -D appTitle="My Site" \
    -D appId="mysite" \
    -D groupId="com.mysite" \
    -D aemVersion=6.5.0

<!-- 1. update version of junit-bom -->
<dependency>
    <groupId>org.junit</groupId>
    <artifactId>junit-bom</artifactId>
    <!--<version>5.6.2</version>-->
    <version>5.7.0</version>
    <type>pom</type>
    <scope>import</scope>
</dependency>

<!-- 2. replace mockito-core with mockito-inline -->
<dependency>
    <groupId>org.mockito</groupId>
    <!--<artifactId>mockito-core</artifactId>
    <version>3.3.3</version>-->
    <artifactId>mockito-inline</artifactId>
    <version>3.11.2</version>
    <scope>test</scope>
</dependency>

<!-- 1. add the following two -->
<dependency>
    <groupId>org.projectlombok</groupId>
    <artifactId>lombok</artifactId>
    <version>1.18.20</version>
    <scope>provided</scope>
</dependency>
<dependency>
    <groupId>com.squareup.okhttp3</groupId>
    <artifactId>mockwebserver</artifactId>
    <version>4.9.1</version>
    <scope>test</scope>
</dependency>

<!-- 2. remove junit-addons -->
<!-- <dependency>
    <groupId>junit-addons</groupId>
    <artifactId>junit-addons</artifactId>
    <scope>test</scope>
</dependency>-->

<!-- 3. replace mockito-core with mockito-inline -->
<dependency>
    <groupId>org.mockito</groupId>
    <!--<artifactId>mockito-core</artifactId>-->
    <artifactId>mockito-inline</artifactId>
    <scope>test</scope>
</dependency>

In the first post of the Exploring the Sling Feature Model series, I discussed the process of converting the Sling CMS app from the Sling Provisioning Model to the Sling Feature Model. So how does this apply to your custom applications?

To illustrate, let’s convert my personal site, danklco.com, which is currently managed via Sling CMS, to the Feature Model.

It’s worth noting that I could keep running my site the way it is, by using a pre-built Sling CMS Runnable Jar, but that my goal is to run my site in Kubernetes for simplicity of upgrades, deployment, and management.

Step 1: Refactor Project Structure

Currently, my personal website code is a single OSGi Bundle which I deploy with Github Actions. To support the Sling Feature Model, I’m going to convert the project into a multi-module project and add a new sub-project for my feature.

The new project structure will look like:

/mysite
/bundle
/feature
/images

Step 2: Generate Features

The custom feature is pretty simple, defining my custom code bundles and configurations. A number of parameters are supplied so they can be changed and I’m not putting secrets in code:

{
    "bundles": [
        {
            "id": "com.danklco:com.danklco.slingcms.plugins.disqus:1.1-SNAPSHOT",
            "start-order": "20"
        },
        {
            "id": "com.danklco:com.danklco.slingcms.plugins.twitter:1.0",
            "start-order": "20"
        },
        {
            "id": "com.danklco:com.danklco.site.cna.bundle:1.0.0-SNAPSHOT",
            "start-order": "20"
        }
    ],
    "configurations": {
        "org.apache.sling.cms.core.analytics.impl.GeoLocatorImpl": {
            "scheduler.expression": "0 0 0 ? * WED",
            "licenseKey": "${MAXMIND_LICENSE_KEY}"
        },
        "org.apache.sling.cms.reference.impl.SearchServiceImpl": {
            "searchServiceUsername": "dklco-com-search-user"
        },
        "org.apache.sling.commons.crypto.internal.FilePasswordProvider~default": {
            "names": [
                "default"
            ],
            "path": "/opt/slingcms/passwd"
        },
        "org.apache.sling.commons.crypto.jasypt.internal.JasyptRandomIvGeneratorRegistrar~default": {
            "algorithm": "SHA1PRNG"
        },
        "org.apache.sling.commons.crypto.jasypt.internal.JasyptRandomSaltGeneratorRegistrar~default": {
            "algorithm": "SHA1PRNG"
        },
        "org.apache.sling.commons.crypto.jasypt.internal.JasyptStandardPBEStringCryptoService~default": {
            "algorithm": "PBEWITHHMACSHA512ANDAES_256",
            "saltGenerator.target": "",
            "securityProviderName": "",
            "ivGenerator.target": "",
            "securityProvider.target": "",
            "keyObtentionIterations": 1000,
            "names": [
                "default"
            ],
            "stringOutputType": "base64"
        },
        "org.apache.sling.commons.messaging.mail.internal.SimpleMailService~default": {
            "connectionListeners.target": "",
            "transportListeners.target": "",
            "username": "${SMTP_USERNAME}",
            "mail.smtps.from": "${SMTP_USERNAME}",
            "messageIdProvider.target": "",
            "mail.smtps.host": "${SMTP_HOST}",
            "names": [
                "default"
            ],
            "password": "${SMTP_ENC_PASSWORD}",
            "mail.smtps.port": 465,
            "cryptoService.target": "",
            "threadpool.name": "default"
        },
        "org.apache.sling.commons.messaging.mail.internal.SimpleMessageIdProvider~default": {
            "host": "danklco.com",
            "names": [
                "default"
            ]
        }
    }
}

To create a usable model, I’ll need to combine the Sling CMS model and my custom model, which can be accomplished with the Sling Feature model. To support the Composite Node Store, I’ll want to generate two separate aggregates, one for seeding and one for running the instance.

Since the Sling Feature Model JSON will resolve dependencies at runtime from Apache Maven, we’ll also want to generate Feature Archives or FAR files which bundles the models with their dependencies.

<plugin>
    <groupid>org.apache.sling</groupid>
    <artifactid>slingfeature-maven-plugin</artifactid>
    <version>1.3.0</version>
    <extensions>true</extensions>
    <configuration>
        <framework>
            <groupid>org.apache.felix</groupid>
            <artifactid>org.apache.felix.framework</artifactid>
            <version>6.0.3</version>
        </framework>
        <aggregates>
            <aggregate>
                <classifier>danklco-com-seed</classifier>
                <filesinclude>**/*.json</filesinclude>
                <includeartifact>
                    <groupid>org.apache.sling</groupid>
                    <artifactid>org.apache.sling.cms.feature</artifactid>
                    <version>0.16.3-SNAPSHOT</version>
                    <classifier>slingcms-composite-seed</classifier>
                    <type>slingosgifeature</type>
                </includeartifact>
                <includeartifact>
                    <groupid>org.apache.sling</groupid>
                    <artifactid>org.apache.sling.cms.feature</artifactid>
                    <version>0.16.3-SNAPSHOT</version>
                    <classifier>standalone</classifier>
                    <type>slingosgifeature</type>
                </includeartifact>
                <title>DanKlco.com</title>
            </aggregate>
            <aggregate>
                <classifier>danklco-com-runtime</classifier>
                <filesinclude>**/*.json</filesinclude>
                <includeartifact>
                    <groupid>org.apache.sling</groupid>
                    <artifactid>org.apache.sling.cms.feature</artifactid>
                    <version>0.16.3-SNAPSHOT</version>
                    <classifier>slingcms-composite-runtime</classifier>
                    <type>slingosgifeature</type>
                </includeartifact>
                <includeartifact>
                    <groupid>org.apache.sling</groupid>
                    <artifactid>org.apache.sling.cms.feature</artifactid>
                    <version>0.16.3-SNAPSHOT</version>
                    <classifier>standalone</classifier>
                    <type>slingosgifeature</type>
                </includeartifact>
                <title>DanKlco.com</title>
            </aggregate>
        </aggregates>
        <scans>
            <scan>
                <includeclassifier>danklco-com-seed</includeclassifier>
            </scan>
            <scan>
                <includeclassifier>danklco-com-runtime</includeclassifier>
            </scan>
        </scans>
        <archives>
            <archive>
                <classifier>danklco-com-seed-far</classifier>
                <includeclassifier>danklco-com-seed</includeclassifier>
            </archive>
            <archive>
                <classifier>danklco-com-runtime-far</classifier>
                <includeclassifier>danklco-com-runtime</includeclassifier>
            </archive>
        </archives>
    </configuration>
    <executions>
        <execution>
            <id>aggregate-features</id>
            <phase>prepare-package</phase>
            <goals>
                <goal>aggregate-features</goal>
                <goal>analyse-features</goal>
                <goal>attach-features</goal>
                <goal>attach-featurearchives</goal>
            </goals>
            <configuration>
                <replacepropertyvariables>MAXMIND_LICENSE_KEY,SMTP_HOST,SMTP_USERNAME,SMTP_ENC_PASSWORD</replacepropertyvariables>
            </configuration>
        </execution>
    </executions>
</plugin>

Step 3: Build Docker Images

Since the goal is to run this in Kubernetes, we’ll create Docker images for running Sling CMS and Apache web server. Since I’m running a lean server, I’ll want to run this as a standalone instance using the Composite Repository so the datastore persists between instances.

To populate variables into the images and coordinate the full build, we’ll use Apache Maven to process the Docker files and input files as Maven artifacts and kick off the docker build. Unlike the Sling CMS build, we’re not leveraging Apache Maven to download the artifacts within the Docker build, we’ll pre-fetch them during the maven build and supply them to the Docker build.

Side Note – Variables

One challenge to note when attempting to reproduce an actual instance, there are a quite a few variables required for the application to actually work. For my local testing I have a bash script to provide all of the required properties to Maven, but since they include secrets like passwords I’ve not put it in source control.

See it in Action!

Seeing something work is work a thousand words, so check out this GIF of the build process in action:

and check out the code on GitHub: https://github.com/klcodanr/danklco.com-site/tree/cloud-native-sling

What’s Next?

All of this is leading up to having a fully running Cloud Native Apache Sling CMS instance in Kubernetes, but before that my next post is going to talk about using Sling Content Distribution and Sling Discovery to support publishing content between Author and Renderer Apache Sling CMS instances. Check back soon!

In a comment on my previous post Apache Sling JVM Performance, Gil Tene made an insightful comment about the potential possibility of performance impact from speed from the underlying environment or other tests.

To accommodate for this possibility, I re-ran the tests inside a loop, randomizing order of the JVM execution for each iteration 25 times. As Gil predicted this did bring the OpenJDK implementations closer together with GraalVM and OpenJ9 as outliers. 

Startup Performance

Interestingly, with the multiple iterations, OpenJ9 actually became the fastest starting JVM implementation, though practically tied with OpenJDK Hotspot and Azul Zulu. GraalVM was almost 6 seconds slower to start on average.

Package Installation Performance

Package performance was quite interesting as every JVM besides OpenJ9 averaged out nearly identically. 

Transaction Performance

The transaction performance varies significantly from the initial results with GraalVM taking the lead in the rate and quantity of transactions and OpenJ9 handles almost 5 transactions fewer per second than GraalVM.

This is honestly quite different than I expected. My hypothesis was that the OpenJDK-based implementations would net out pretty similarly, but in actuality, there was a statistically significant difference between each implementation.

Memory Usage

The full run of 25 iterations showed roughly the same results in terms of memory usage. OpenJ9 used significantly less memory and GraalVM significantly more with OpenJ9 using 60% of the average memory of the OpenJDK implementations.

Outliers

One of the interesting things to observe is that there were some extreme outliers, for example, package installation which generally took ~30 seconds, occasionally taking over 2.5 minutes. It seems like this is related to the underlying hardware as there’s not a pattern in the iteration order, iteration number of JVM implementation. To avoid skewing the data, I excluded these outliers from the other charts.

Revised Summary and Findings

With multiple runs, the differences between the OpenJDK codebase implementations (e.g. OracleJDK, Amazon Coretto, Azul Zulu and OpenJDK Hotspot), reduces significantly. The performance and startup differences small enough, that licensing would be the primary criteria I’d recommend when considering the JVM implementation to use.

If raw performance is the primary concern, GraalVM demonstrated a consistently higher transaction rate over the iterations at the cost of a slower startup and higher memory usage.

For lower-end usages or container-based usages, OpenJ9 continues to be an excellent choice with it’s low memory usage and especially after it demonstrated the promised faster startup on average over the multiple iterations. 

With the recent proliferation of Java Virtual Machine (JVM) implementations, it’s difficult to know which implementation is the best for your use case. While proprietary vendors generally prefer Oracle Java, there are several open source options with different approaches and capabilities. 

Given how the implementations vary in some underlying technical specifics, the “correct” JVM implementation will vary based on the use case. Apache Sling, in particular has some specific needs given the complexity of the OSGi / JCR backend and the Sling Resource Resolution framework. 

Test Strategy

To help get some real data on which JVM implementation works best for running Apache Sling, I created a project to:

1. Install a number of JVM implementations and monitoring tools
2. For each JVM:
   1. Setup an instance of Apache Sling CMS, using no additional parameters
   2. Install a content package to the Sling CMS instance
   3. Run a performance test using siege
3. Consolidate the data into a CSV

If you are curious, you can checkout the Sling JVM Comparison project on Github.

The project installs and compares the following JVM implementations on version 11:

• Eclipse OpenJ9 – 11.0.8 – openj9-0.21.0, JRE 11
• Amazon Coretto – 11.0.8.10.1
• OpenJDK Hotspot – 11.0.8+10
• Oracle JDK – 11.0.8+10-LTS
• Azul Zulu – 11.0.8+10-LTS – 11.41+23-CA
• GraalVM – CE 20.1.0 – 11.0.7+10-jvmci-20.1-b02

To create a meaningful comparison, I setup and ran the test an Amazon EC2 m5.large instance running Ubuntu 18.04 LTS “Bionic Beaver” and captured the results.

Startup / Install Performance

An important performance comparison is the amount of time it takes to get an instance running. To measure this, I captured the time in milliseconds to start the Apache Sling CMS instance and the amount of time required to upload and install the same content package. There is a potential variance in the capture of the startup time as the test process polls the Sling instance to see when it responds successfully to a request to determine startup time.

OpenJDK Hotspot and Amazon Coretto are essentially tied as the leaders of the pack with Oracle JDK and GraalVM following shortly behind. Azul Zulu and Eclipse OpenJ9 take 78% and 87% longer to start as OpenJDK Hotspot. Interestingly, most of the JVM implementations take approximately the same time to install the content package, however, Eclipse OpenJ9 takes 35% longer to install the content package.

Performance Under Load

To check performance under load, I tested the instances using siege using a list of URLs over the course of an hour with blocks of 15 minutes on and 15 minutes off. 

First, we can take a look at the throughput per second:

And next, we can look at the raw transaction count:

Both show the same story, OpenJDK Hotspot, Amazon Coretto and Oracle JDK taking the top spots for performance with GraalVM, Azul Zulu and Eclipse OpenJ9 trailing behind.

Memory Usage

Finally, given how memory intensive Java applications can be, it’s important to consider memory usage and here the differences are quite stark:

Eclipse OpenJ9 is significantly less memory intensive, using only 55% of the average memory of the 4 middle-tier JVM implementations. GraalVM also sits outside the average, using 15% more memory than the same middle-tier JVM implementations.

Summary and Findings

From a raw performance perspective, OpenJDK Hotspot is the clear winner with Amazon Coretto close behind. If you are all in on Amazon or want a long-term supported JVM option, Amazon Coretto would be worth considering.

For those running Apache Sling on memory-limited hosting options, Eclipse OpenJ9 is the best option. While there is a tradeoff for performance, when you only have a Gigabyte or two of memory, reducing the load by 45% will make a tremendous difference.

Credit

Thanks to Paul Bjorkstrand for coming up with idea for this post. 

In my previous post Exploring the Sling Feature Model, I described the process of migrating from a Sling Provisioning project setup to a Sling Feature Model project. 

Now that we have the Sling Provisioning Model project converted, we can move on to the fun stuff and create a Composite NodeStore. We’ll use Docker to build the Composite Node Store in the container image.

Creating the Composite NodeStore Seed

The Composite NodeStore works by combining one or more static “secondary” node stores with a mutable primary NodeStore. In the case of AEM as a Cloud Service, the /apps and /libs directories are mounted as a secondary SegmentStore, while the remainder of the repository is mounted as a MongoDB-backed Document Store. 

For our simplified example, we will create a secondary static SegmentStore for /apps and /libs and combine that with a primary SegmentStore for the remainder of the repository. Since the secondary SegmentStore will be read-only, we must “seed” the repository to pre-create the static paths /apps and /libs. 

To do this, we have a feature specifically to seed the repository with the /apps and /libs temporarily mutable. We can then use the aggregate-features goal of the Sling Feature Maven Plugin to combine this with the primary Feature Model to create a feature slingcms-composite-seed. When we start a Sling instance using this feature, it will create the nodes under these paths based on the feature contents. 

As shown below, while seeding the repository is written to the libs SegmentStore. It’s also worth mentioning that with the Feature Model Launcher, by default, the OSGi Framework runs in a completely different directory from the repository and pulls the bundle JARs from the local Maven repository.

Our updated Dockerfile runs the following steps to build the container image:

• Downloads the Feature Model Launcher JAR and Feature Model JSON files
• Starts the Sling instance using the slingcms-composite-seed model in the background
• Polls the Felix Health Checks until the tag “systemalive” returns 200
• Once the 200 status is returned the Sling instance is stopped and the build cleans up the launcher and symlinks the SegmentStore directory into the expected path

Naming Gaffes

Hopefully, you are more careful than me, but one thing to keep in mind is that the Sling Feature Launcher will happily start as long as it has a valid model. For example, you can easily spend a significant amount of time trying to understand why nothing responds with this model:

org.apache.sling.cms.feature-0.16.3-SNAPSHOT-composite-seed.slingosgifeature

Instead of the one I meant:

org.apache.sling.cms.feature-0.16.3-SNAPSHOT-slingcms-composite-seed.slingosgifeature

Since the non-aggregate model is a valid model, the Sling Feature Launcher will happily start, but it simply creates an OSGi container with only a couple of configuration which naturally does… nothing. 

Starting and Running

Once the repository is been fully started and seeded, we’ll run a different Feature Model to run the instance. Similar to the Composite Seed Feature Model, the slingcms-composite-runtime Composite Model will use the composite repository, however it runs the libs mount in readonly mode.

To use the runtime Feature Model, the CMD directive in the Dockerfile calls the Sling Feature Model Launcher with the slingcms-composite-runtime Feature Model. In addition, we’ll mount a volume in the docker-compose.yml to separate the mutable volume out from the container disk, that way the repository persists between restarts and container deletion.

While in runtime mode, the Composite repository looks like the diagram below, leveraging a Docker volume for the global SegmentStore and the local seeded repository for the libs SegmentStore:

End to End

Here’s a quick video showing the process of creating a Container-ized version of Sling CMS with a Composite NodeStore from end to end.

Details: Build Arguments & Dependencies

The current example implementation uses Apache Maven to pull down the Feature Models with a custom settings.xml and Build Arguments in the Dockerfile. By changing the settings.xml and the Build Arguments, you could override the Feature Model being produced to use a custom Feature Model, for example an aggregate of Sling CMS and your custom Sling CMS app.

We’ll cover the process of producing a custom aggregate in the next blog post in the Exploring the Sling Feature Model series. If you’d like to learn more about the Sling Feature Model, you should check out my previous post on Converting Provisioning Models to Feature Models.

In my previous post about AEM as a Cloud Service, How AEM Scales, I discussed the new Sling Feature Model which is a key part of how Adobe scales their new AEM as a Cloud Service solution using Kubernetes. 

The goal of the Feature Model is to create a better way to provision Sling instances by creating a more descriptive, flexible grammar for building targeted instances instead of building a fat JAR file and adding bundles after the fact.

This week / weekend, I took some time to finally upgrade my pet project, Sling CMS to use the Sling Feature Model and wanted to take a moment to document the process and my learnings.

Once you understand the process and model, the process of converting a Provisioning Model-based project to Feature Model is straight-forward. To get started, I followed along with the Feature Model How-To Guide and then dug into the code for the Sling Kickstart and (Feature Model compatible branch of) Sling Starter. 

Converting a Provisioning Model Project to Feature Model

The first step was using the sling-feature-converter-maven-plugin to convert the legacy Provisioning Model configuration to the Feature Model. To do this, I created a temporary sub-module to execute the conversion process:

https://github.com/apache/sling-org-apache-sling-app-cms/blob/SLING-8913-multiple-instance-types/converter/pom.xml

I’ve modified this POM based on the Sling Kickstart POM to look up the Provisioning files in the builder sub-module of Sling CMS, configured it with my desired artifact information and added a step to create an aggregated Feature Model so I could validate the conversion.

Interestingly, I found that mvn clean after failed builds seemed to often result in Maven dependency resolution issues (I’m assuming because it was evaluating the Feature files) so rm -rf target/ was my go to cleaning action.

To run the conversion, I can execute mvn clean install -Dbnd.baseline.skip=true. Once the process is complete, the feature files can be found under target/fm and you can validate by downloading the Feature Model Launcher and running:

java -jar org.apache.sling.feature.launcher-1.1.4.jar -f target/slingfeature-tmp/feature-slingcms.json

One feature file will be created per Provisioning Model file and the contents of the Provisioning Model file will be transformed into the format used by the Feature Model.

During this step, I also found that the Sling Feature Model is more strict than the Provisioning Model and called out a few mistakes in the Provisioning Model, including duplicate configurations (for example I had duplicated the configuration org.apache.sling.serviceusermapping.impl.ServiceUserMapperImpl.amended-sling.rewriter).

Once I successfully converted the Provisioning files to Feature Models, I copied them from target/fm into a new features submodule and updated the Artifact ID accordingly. 

Aggregating the Feature Model

The purpose of the new feature submodule is to produce the final artifacts based on the Feature Model files. The first and easiest artifact is to create an aggregate feature to execute with the Sling Feature Model Launcher. 

To do this, I added the following goals from the slingfeature-maven-plugin plugin:

• aggregate-features – aggregate the feature files into a single feature file
• analyse-features – analyse the feature files to ensure they didn’t have any issues
• attach-features – attach the feature models as Maven artifacts
• attach-featurearchives – attach the feature archives as Maven artifacts

The outcome of all this is a consolidated Feature Model and Feature Archive for Sling CMS based on the input Feature Model files we migrated from the Provisioning Model. Before we can complete the process however, there are a few issues which need to be resolved. 

Feature Models vs. Feature Archives

Quick segue about Feature Archives. By default the Sling Feature Model will download dependencies as artifacts from the Maven repository. To avoid having to download files at runtime, package private artifacts or to enable Sling Features to run offline Sling Feature Model supports Feature ARchives or FAR files. The FAR file packages the Feature descriptor and binary dependencies into a ZIP archive.

Feature Archives can be used in the same manner as Feature Models, reference the path to the Feature Archive instead of the Feature Model in the -f or -sf flags for the Sling Feature Launcher.

Resolving Feature Errors

The analyse-features goal of the slingfeature-maven-plugin flagged a number of issues with the legacy Sling CMS Provisioning Model setup around the start order of the bundles. Each error will be logged with a message like the following:

[ERROR] Artifact org.apache.sling:org.apache.sling.servlets.resolver:2.7.2 requires [org.apache.sling.servlets.resolver/2.7.2] osgi.ee; filter:="(&(osgi.ee=JavaSE)(version=1.8))" in start level 20 but no artifact is providing a matching capability in this start level.

Blindly updating start order to resolve the build issues caused Sling to not be able to render content. I would recommend making changes one bundle (or set of bundles) at a time and validating with every change. In the Sling CMS setup, the primary problem I had was with the ordering of the Apache Sling integration with Apache Oak and the bundles org.apache.sling.jcr.jackrabbit.usermanager and org.apache.sling.jcr.jackrabbit.accessmanager.

Version Variables

One of the conveniences in the Provisioning model is Variables, which allow you to set shared variables for things like version numbers when you have multiple bundles from the same project (Oak, Jackson, Composum, etc) you want to keep in sync. The Feature Model has a similar concept called Placeholders.

The great thing about Placeholders is they can be injected from the POM. For example, I created the following properties in my POM’s properties elements:

<composum.version>1.12.0</composum.version>
<jackrabbit.version>2.20.0</jackrabbit.version>
<jackson.version>2.11.1</jackson.version>
<oak.version>1.26.0</oak.version>
<slf4j.version>1.7.25</slf4j.version>

Each instance of these variables will be replaced within all of the Feature Model files.

Creating a Runnable JAR

While the goal of the Feature Model is to create a more flexible model and get away from running a fat JAR, but for local development and getting started quickly having a runnable JAR is just easier. 

To create a runnable JAR as a part of the same build, I configured Maven to combine the Sling Feature Model Launcher and my Sling CMS Feature Archive at build time into a standalone JAR using the Maven Assembly Plugin.

The Main class / method is quite brief, it:

• Gets the version number passed via a properties file
• Resolves the URL for the Feature Archive from the classpath
• Appends (if not present) the -f flag with URL of the Feature Archive to the arguments
• Calls the Sling Feature Model Launcher Main method

With that, we now have a standalone JAR which provides a double click run of Sling CMS as well as the rich Feature Model based aggregation in the form of Feature Models and Feature Archives all in one build.

In the next blog post in the Exploring the Feature Model series, I will be setting up an Oak Composite Node Store. 

Did you miss my webinar with the Detroit Adobe Experience Meetup on Sling RepoInit? Sling RepoInit is a newer technology which enables projects to set up permissions, configurations and content via OSGi configurations in a purpose-built grammar.

Check out the presentation deck and recording to see how RepoInit could be useful on your project.

Watch the Recording

Curious about using Sling RepoInit? Want to learn more in depth about how Sling RepoInit can enable your AEM DevOps team to manage the initial repository state in code?

I’ll be leading a virtual discussion on Sling RepoInit with the Detroit AEM Meetup on Thursday July 9th from 6:00 – 6:50 PM EST.

This talk will:

• Introduce the benefits Sling RepoInit as a provisioning method
• Compare Sling RepoInit to other methods of initializing a repository
• Show how to manage permissions, configurations, and initial content using RepoInit
• Demonstrate using Sling RepoInit for both AEM as a Cloud Service and AEM 6
• Discuss a method for making it easier to manage Sling RepoInit configurations and future options 

More info about Repoinit: https://sling.apache.org/documentation/bundles/repository-initialization.html 

This talk will be useful for AEM Technical Experts, Architects, and Developers, especially those interested in AEM as a Cloud Service.

Register Here