Benefits of Automation
If you’re reading this, you’re probably already aware of the importance of front-end page performance to the end-user experience. You might even be working on improving the performance of your pages and using Lighthouse to track your progress.
While it is quite easy to run tests via the Lighthouse tab under Chrome’s developer tools, there are a few important points to keep in mind:
- Installed browser extensions can affect the score, so we need to remember to always run Lighthouse in an Incognito tab,
- Anything rendered on the page, such as cookies, user selections or notifications, can alter the final page score. So, it is important to ensure identical page state across different runs to make accurate comparisons.
- Finally, even after controlling for page state, you may find the score fluctuates significantly every time you run the test. In which case it can be useful to average the score across several runs, rather than rely on a single sample.
Automation can help address all the above concerns by ensuring repeatability and allowing you to quickly run multiple tests. We can also use scripts to calculate the average score from multiple reports.
Finally, you can use automation to create a dashboard that tracks the score of specific pages over time, to ensure your website is moving in the right direction. This will also allow you to catch any changes that inadvertently affect page performance.
Automation Options
I experimented with three different approaches, for automating Lighthouse testing. Outlined below is a high-level overview of each, along with some of the main pros and cons.
Sitespeed.io
Sitespeed.io is an open source tool for both testing and monitoring your website. You can install it either as a Docker container or as an npm module. It is very powerful, with the ability to test page performance in both Chrome and Firefox. The generated reports not only detail page metrics, but also include a video recording of the Browser session.
Going over all the features of Sitespeed.io could fill an entire series of blog posts but, for our purposes, I will just mention the Lighthouse plugin which adds Lighthouse to the suite of tests that are run.
Pros
- Powerful tool for everything web performance related
- Easy to setup performance dashboard via provided Docker compose file
- Excellent documentation
- Custom page interactions / browser automation possible via Selenium
Cons
- May be overkill for a developer looking for quick Lighthouse test results
- An open bug with the Lighthouse plugin at the time of my testing prevented me from using it to generate Lighthouse scores
Using a Docker Container
For this option, we use an existing Docker container, which can be fetched via: docker pull femtopixel/google-lighthouse
Tests are then run via the command line. The following example will test Google’s homepage and write the test results to a json file:
docker run --rm --name lighthouse -it -v "/path/to/your/report:/home/chrome/reports" femtopixel/google-lighthouse http://www.google.com –output=json --chrome-flags="--ignore-certificate-errors"
Additional documentation is available on the GitHub page.
Pros
- Nothing to install (other than Docker)
- Fastest to set up
- Can script with PowerShell
Cons
- No support for custom page interactions / browser automation
- Docker networking errors reported by some users
Using Node.js
As per Google’s own documentation, the “Node CLI provides the most flexibility in how Lighthouse runs can be configured and reported”. Lighthouse can be installed as a global npm module via: npm install -g lighthouse
Tests can then be run via the command line. The syntax is very similar to that for the Docker container above. The following command will run an equivalent test:
lighthouse https://www.google.com --output=json --output-path=./test.json --chrome-flags="--ignore-certificate-errors"
The chief advantage of this approach, over the Docker Container, is the ability to use Puppeteer for page interactions like authentication.
Pros
- Approach recommended by Google
- Custom page interactions / browser automation possible via Puppeteer
- Can script with JavaScript
Cons
- Is only compatible with the latest versions of Node.js. This may need you to switch between versions for running Lighthouse and site development
In Conclusion
While it doesn’t matter how you test your site, it is important to incorporate Lighthouse testing both into your development process, as well as to track your site’s performance trends over time.
Hopefully you can adopt one of the above approaches to meet your needs. We should also be able to combine the best of both the Docker and Node.js approaches, by creating a container that includes Lighthouse and Puppeteer. Please let me know if I have overlooked an existing container that does this.
