Innovation and Product Development

Apache Sling JVM Performance Followup

Finish Banner

In a comment on my previous post Apache Sling JVM PerformanceGil Tene made an insightful comment about the potential possibility of performance impact from speed from the underlying environment or other tests.

To accommodate for this possibility, I re-ran the tests inside a loop, randomizing order of the JVM execution for each iteration 25 times. As Gil predicted this did bring the OpenJDK implementations closer together with GraalVM and OpenJ9 as outliers. 

Startup Performance

Interestingly, with the multiple iterations, OpenJ9 actually became the fastest starting JVM implementation, though practically tied with OpenJDK Hotspot and Azul Zulu. GraalVM was almost 6 seconds slower to start on average.

Package Installation Performance

Package performance was quite interesting as every JVM besides OpenJ9 averaged out nearly identically. 

Transaction Performance

Platforms & Technology - A Business Leaders Guide to Key Trends in Cloud
A Business Leaders Guide to Key Trends in Cloud

Cloud’s dynamic nature can make it hard to keep up with the wide-ranging capabilities that make it a key enabler to improve business processes and support a larger digital transformation.

Get the Guide

The transaction performance varies significantly from the initial results with GraalVM taking the lead in the rate and quantity of transactions and OpenJ9 handles almost 5 transactions fewer per second than GraalVM.

This is honestly quite different than I expected. My hypothesis was that the OpenJDK-based implementations would net out pretty similarly, but in actuality, there was a statistically significant difference between each implementation.

Memory Usage

The full run of 25 iterations showed roughly the same results in terms of memory usage. OpenJ9 used significantly less memory and GraalVM significantly more with OpenJ9 using 60% of the average memory of the OpenJDK implementations.

Outliers

One of the interesting things to observe is that there were some extreme outliers, for example, package installation which generally took ~30 seconds, occasionally taking over 2.5 minutes. It seems like this is related to the underlying hardware as there’s not a pattern in the iteration order, iteration number of JVM implementation. To avoid skewing the data, I excluded these outliers from the other charts.

Revised Summary and Findings

With multiple runs, the differences between the OpenJDK codebase implementations (e.g. OracleJDK, Amazon Coretto, Azul Zulu and OpenJDK Hotspot), reduces significantly. The performance and startup differences small enough, that licensing would be the primary criteria I’d recommend when considering the JVM implementation to use.

If raw performance is the primary concern, GraalVM demonstrated a consistently higher transaction rate over the iterations at the cost of a slower startup and higher memory usage.

For lower-end usages or container-based usages, OpenJ9 continues to be an excellent choice with it’s low memory usage and especially after it demonstrated the promised faster startup on average over the multiple iterations. 

About the Author

Dan is a certified Adobe Digital Marketing Technologist, Architect, and Advisor, having led multiple successful digital marketing programs on the Adobe Experience Cloud. He's passionate about solving complex problems and building innovative digital marketing solutions. Dan is a PMC Member of the Apache Sling project, frequent Adobe Beta participant and committer to ACS AEM Commons, allowing a unique insight into the cutting edge of the Adobe Experience Cloud platform.

More from this Author

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.