If you’re running an e-commerce website, you probably have your customers searching for your products on your site. In most instances, you’ve likely closed those search pages to search engine indexation, since they may be duplicative of some category pages. However, you may not realize that search pages often fill a gap in your category page structure with a wealth of query-relevant content. If done strategically, opening the right search pages to indexation can significantly benefit your organic traffic.

In this article, you’ll learn how to run the necessary analysis for opening query-aligned search pages to improve organic traffic performance. Incorrectly opening search pages to indexation can spell disaster, so read and follow the best practices laid out when attempting this type of advanced SEO optimization.

Identify Your Gaps

To begin, identify your content gaps. Where do you have product inventory but no category page that displays it? For example, you may carry plenty of widgets XYZ but no category page with that inventory available.

Keyword research can help you understand what people are looking for. Take that information and compare the query demand with your available category pages that are a good match. Once you know your inventory, you can identify missing pages, which products aren’t covered by existing category pages, and where your customer demand lies.

Tools like SEMRush help identify these gaps. To make analysis tools easier to use, keep your category pages in a single folder. Enter your category pages and the category pages of your competitors. The gap analysis tools show you which keywords your competitors are ranking for that you’re not, so you can determine if you have enough product inventory to fulfill that need.

Open Search Pages to Indexation

Once you’ve identified your keyword gaps, determine the best way to fill them:

• Pull a list of keywords that your category pages rank for.
• Open a specific set of search pages that fill any keyword gaps not currently filled by category pages
• Let Google index those search pages.

Remember, opening search pages to indexation when an existing category page already meets the same query demand can damage your SEO. Presenting a category page and search page that target the same query and contain the same content dilutes signals for Google and can harm your overall ranking potential.

Excel is a great tool for the type of comparative analysis you’ll need to perform. Using Excel lets you see which of your category pages aren’t ranking in the top 25 or so positions for the specific keywords identified in the gap analysis. Once you understand where you can support unmet demand, you’ll know which search pages to open for indexation.

How to Open Search Pages for Indexation?

A nonindex tag or disallow rule in robots.txt may cut search pages off from indexation. However, a noindex tag requires Google to crawl your pages to discover the tag and may be a waste of crawl budget.

In either case, opening search pages to crawling requires you to either remove the disallow rule from your robots.txt file or remove the noindex tag. It’s also a good idea to add a self-referential canonical tag to the pages. The canonical tag will send an added signal to Google that the pages are intended for indexation.

Getting New Search Pages Indexed

By now, you’ve opened a very targeted set of search pages to indexation that fill obvious demand where a category page isn’t created. After a few days, you find Google hasn’t indexed any of them yet. Why?

Google is cautious about indexing new pages if it thinks they were accidentally opened for indexation. In this case, the changes were very intentional. So, how do you convince Google to start indexing your new pages?

You have several options:

• Request indexation manually in Google Search Console (GSC)
  • This isn’t a great option if you’ve opened hundreds or thousands of pages
  • This tool is temporarily unavailable as of Oct. 14, 2020 (Source)
• Create an XML sitemap and submit it in GSC, or add them to your existing XML sitemap
  • This is an SEO best practice for any new indexable pages
  • However, this alone may not be enough to convince Google to index your pages
• Create an HTML sitemap to get the pages into the crawl path
  • This is a reasonable option, as well
  • However, Google has indicated that they do not pass page rank through HTML sitemap links, so this may be considered a weak signal
• Link your new pages to already indexed and ranking pages
  • This is your best bet but requires extra effort
  • Look for pages that are relevant to the new search pages, and add a link from that page to the new one
  • On a larger scale, look for category pages with an existing template where you can add a block of related links
    • Tagging or another automated system for associating links is required, but it can work well once it’s done

Finally, be patient. Google can take weeks or months to decide that your new search pages are worth indexing. However, the effort is worth it once they’re indexed and have the potential to drive organic search traffic.

Once your newly targeted search pages are indexed, it’s time to track performance. It’s best to track keyword ranking and traffic. Add the keywords you identified in your gap analysis into your favorite tracking tool to monitor over time. Page clicks and impressions in Google Search Console are also good to watch.

ADA compliance and web accessibility are more serious than you likely know. Consider this scenario: You or one of your clients suddenly receives a letter stating that the website you administer is not ADA compliant and you’re facing litigation. Facing litigation? Now what!
The best course of action is to proactively review your website for ADA compliance and ensure that it is accessible to people with disabilities before you get into trouble. The level of compliance necessary is outlined in the Web Content Accessibility Guidelines (WCAG) 2.0 (available here). These guides are quite detailed, but it will help you fully comply with the law and insulate your company from litigation because it’s comprehensive.
A good place to start for website ADA compliance and accessibility is to use the following:

• Check the current state of your website accessibility with tools like WAVE wave.webaim.org and the Google Lighthouse tool (available in the Chrome browser)
• Ensure that all images have descriptive alt text
• Provide closed captioning on any videos your site may have
• Provide text transcripts of any video or audio only files
• Give users the ability to pause, stop or hide any automated content like email signups
• Use simpler design, be sure the website isn’t overly complex and provide options for adjustments to size/color of text and content
• Be sure your website supports keyboard navigation (think navigation between elements with arrows and tab keys)
• Provide support features so a person with a disability can contact the webmaster and receive a response
• Be sure any forms on your website have instructions for their use and that each form element is labeled with clear and understandable text
  • Also, use the id and label HTML elements on form items

Once the above checklist has been followed, it is advisable to have a legal professional review your website in light of the WCAG 2.0 guidelines.

In its 20 years as a company, Google has revolutionized the way we find information. The search engine giant is in the midst of rolling out even more changes – it’s moving from answers to journeys, shifting away from queries, and, now, the shift to visual searching.

Strings to Things to Concepts

One easy way to understand Google’s search technology evolution is through three main ideas: strings, things, and concepts. As we move into the concepts phase of internet search, it’s helpful for us to review the steps that came first.

1. Strings

When Google began, it was all about keywords. Those were the “strings”—the words (and sets of words) that helped Google provide users with the most relevant, high-quality information. We can’t overstate how revolutionary keyword technology was, but keyword-based search placed most of the responsibility on the user to find the right information. If you didn’t enter the right keywords, you wouldn’t see the search results you wanted to see.

2. Things

After a while, Google’s algorithms got smarter. With the launch of the Knowledge Graph in 2012, Google began to understand what people meant when they used fuzzy search criteria, and began to steer them toward the stronger searchable terms and relevant information. Put simply, it was a progression from basic keywords to semantically related keywords and ideas.
The Knowledge Graph enabled Google to aggregate millions of search queries to understand what users were actually interested in when they used certain search terms. This 2012 blog post laid out Google’s hopes for the future:
“We’ve always believed that the perfect search engine should understand exactly what you mean and give you back exactly what you want. And we can now sometimes help answer your next question before you’ve asked it, because the facts we show are informed by what other people have searched for.”

3. Concepts

In 2018, Google announced it would be focusing not just on words, but also on images and other visual content. With this shift, Google hopes to move from answering users’ questions to being their personal assistant. Instead of just responding to your searches, Google will pick up where you leave off, taking users on an information journey. One of the biggest changes since 2012 is that more than half of all Google searches are coming from mobile devices. The visual shift we’re seeing specifically targets those mobile users. In 2018 we also saw Google’s understanding of content and query intent reach a whole new level.

Good Content vs. Great Content

We know now that Google is moving in a more visual direction, focusing on the mobile experience and integrating images, videos, and other visual content. But what does this mean for SEO? The good news is that the fundamentals remain the same:

• High-quality content
• Relevancy
• Authoritative perspective
• Answering users’ questions useful

Google’s algorithms will only continue to sharpen their accuracy in finding the best, most relevant visual content. This is still about finding content that addresses user needs the best. This visual shift means that SEO experts will need to help content creators create and maintain relevancy. It will also be critical that content creators put out fresh content on a regular basis, as the algorithms will prefer sites that are frequently updated with highly query relevant text and visual information.
Google’s understanding of content appears to be exponential in nature, not linear. In other words, their algorithmic abilities tend to leap rather than crawl, and the next few years will see dramatic improvements in those abilities. This advanced understanding means good quality content won’t cut it anymore. Rather, sites that want to perform well in search rankings will need truly outstanding content written by experts. In some industries, this expert-level content is already necessary.

Next Steps for SEO

As Google paves the way for a drastically different search experience, here are a few concrete steps SEOs can take to stay relevant in search.

1. Understand the basics

This means having a thorough understanding of how to create high quality and relevant titles, H1 tags, and body content. For visual content, context is key. Stock photos likely will not cut it anymore; you’ll need images that are highly related to your specific content and unique on the web.

2. Consider the user’s journey

Create Content that includes visuals that are optimized for search. Include captions for your visual content that show how those images are a core component of your content. This will help your images/photos perform better in image searches and help users find the information they want quickly and easily.

3. Build visually

For higher visibility and accessibility, optimize your product images for Google Lens. Don’t rely on a user’s ability to type in specific search terms to find your product online. Google Lens shows users relevant images automatically, especially ones with direct links back to product pages. Google is also building its own AMP stories—AI-constructed visual experiences that immerse the user in text, video, and photos. With highly optimized visuals and text, Google may pull your authoritative content into one of these stories.
Differentiating between good and truly world class content used to be a person’s job. Now it’s the purview of intelligent and powerful algorithms. As we move into the future of search, SEO experts need to stay rooted in the basics of high-quality content, all while remembering that “content” is much more than just words on a page.

Can the speed at which your pages load really make a difference for your site?
In this episode of our popular Here’s Why digital marketing video series, our page speed expert John Dietrich makes the case for why you should try to improve page speed, and offers some tips on how to do it.

Don’t miss a single episode of Here’s Why. Click the subscribe button below to be notified via email each time a new video is published.

Subscribe to Here’s Why

Resources

• Page Speed Guide
• John Dietrich on Twitter
• See all of our Here’s Why Videos | Subscribe to our YouTube Channel

Transcript

Eric: We’re very excited to have a special guest with us for this episode of Here’s Why. Let me introduce you all to John Dietrich, one of our expert marketing consultants who works directly with some of our clients to give them the SEO results they seek. Welcome, John.
John: Thanks Eric, I’m delighted to be here.
Eric: We asked you to join us today because he wrote an excellent guide for page speed on the Perficient Digital blog. We’ll tell our viewers how to get it at the end of this video. But first, this is a Here’s Why video. So, since that’s the case, tell us why we should pay attention to how fast page speed is on a site.
John: I’d be happy to.
First of all, it has a direct effect on user experience, especially on mobile, and that can directly affect your bottom line. Numerous studies have shown that even very small changes in page load speed can significantly impact conversations on mobile. As you can see in this table for enterprise-level commerce sites, it can mean a difference of millions or even billions of dollars.

Eric: Wow.
John: Google has also said having a slow page load speed could affect a page’s ranking in search. So, there are ample reasons to do what you can speed up your site.
Eric: Great. How can site owners know how their pages are performing?
John: That’s a really good question. Google provides two different tools for that, and there’s a link to them in the guide that we’ve written.
[Tweet “Learn about the tools Google provides to check your site’s page load speed…and why you should!”]
One of those tools provides a competitive analysis showing how you stack up against other sites. For example, here is how the top five online retailers compare with each other.

Another tool estimates how much revenue you could lose each second of decrease in your page load speed. So, using that calculator, we see that eBay could increase revenue by nearly $20 million with just a two-tenths of a second increase in their page load speed.

Eric: Wow, that’s cool. But you’ll find links to even more tools that can help you see what to do to improve your page speed in John’s guide. He also provides some instructions on how to use some of those tools.
For now, John, give our viewers some quick tips on best practices for improving the load time of their pages.
John: Sure, Eric.
I would start with the following:

1. Optimize your images, CSS and JavaScript
2. Leveraging browser caching and use server compression; various tools are available for all of these steps.
3. Reduce your server response time
4. Prioritize your visible content with lazy load
5. Consider using a content delivery network

Eric: Cool. Not sure what some of those are or how to do them? No worries. John gives you what you need to get started in his guide.

Don’t miss a single episode of Here’s Why. Click the subscribe button below to be notified via email each time a new video is published.

Subscribe to Here’s Why

See all of our Here’s Why Videos | Subscribe to our YouTube Channel

AI is here to take your job! This oft-repeated mantra is used to grab a few headlines, but for now the reality is quite different. The true power of AI is in how we learn to work with these tools to better perform our jobs; thus, AI is here to augment our current abilities and not replace us.

Working Together with Machines

Machine Learning-based AI is an immensely powerful tool that is driving the next revolution in digital marketing. In a report that researched 1,500 companies, the Harvard Business Review found that, “firms achieve the most significant performance improvements when humans and machines work together.” (Collaborative Intelligence: Humans and AI Are Joining Forces).

Improving Through Machine Learning with Existing Data

Data is increasing at an exponential rate. Consider this staggering statistic, reported by Forbes: “Over the last two years alone, 90 percent of the data in the world was generated.” The obvious questions then become, “Where is all this data coming from?” and, “What can we do with it all?”
[Tweet “90% of the data in the world was generated in the last two years. Learn how AI can help us deal with it!”]
In the world of SEO, we generate a lot of data through crawls of large websites. Tools like Botify, DeepCrawl, and Perficient Digital’s own proprietary SEOCrawl^tm tool can generate datasets with tens of millions of URLs, including basic SEO data like titles and H1s. Don’t even think about trying to open a file like that in Excel!
Even database tools like MySQL are quickly brought to their knees with this size dataset. Big data platforms like Casandra or Hadoop are required just to begin analysis.
What if we want more? What if we could take tens of millions of webpage records and associate them with rank and traffic data at a massive scale? How could we possibly analyze this type of data in a meaningful way? One answer is to use Machine Learning-based Artificial Intelligence tools.

Is This a Magic Bullet or Does it Take Work?

Yes, with ML and AI we’ve created a magic bullet! Okay that’s tongue in cheek, so let me clarify a bit.
There is work involved and it does take some developer talent to make the magic happen, but the basic structure is fairly simple. We can apply what’s called “unsupervised machine learning.” With this type of machine learning, we’re able to give an algorithm our data sets without any human-generated labels or “training data,” and that algorithm is able to give us back meaningful information.
For example, Google uses a form of unsupervised machine learning to provide results in Google News. Google sorts through an enormous number of potential news articles to classify, categorize and finally rank them. This includes identifying the hottest story topics, and then finding all the publishers that have created content on those topics.
All of this is done without human input. This is made possible, in part, through unsupervised machine learning. You’ll notice on the bottom of the Google News results page: “The selection and placement of stories on this page were determined automatically by a computer program. The time or date displayed reflects when an article was added to or updated in Google News.”
[Tweet “Unsupervised machine learning makes it possible to generate almost-instant applications from massive real-time data. Learn more at”]
Learn more!

• If you’d like to dive deeper into document classification using ML, check out this blog post from Google Cloud.
• For more on using machine learning and AI, read this interview series with Eric Enge and Chris Penn or watch the videos!

Case Studies Using AI in the Real World

AI is currently being used to improve the way a wide range of industries do business. For example, in this case study, Modern Marketing Concepts (MMC) used an AI tool called RapidMiner to quickly improve productivity. The drag and drop modeling interface allowed the team to use it productively within days. In this example, MMC used the AI tool to create predictive models, which helped them roll out better sales and marketing campaigns for their clients.

Other Uses for ML/AI in the Real World

AI agents such as Microsoft’s Cortana can do useful things for businesses, right out of the box. For example, Cortana can transcribe meetings and automatically distribute notes. Amazon’s AWS cloud service offers meeting transcription tools, and there’s an Amazon Alexa Skill called EchoScribe to help with meeting transcriptions as well. As of this writing, it doesn’t appear that the Google Assistant is capable of handling the more complex requirements of meeting transcription. However, with Google Duplex on the horizon, anything is possible.
There are other commercially available AI tools as well. Companies like domo.com have options—here’s a list of over 50 AI-powered marketing tools.

Custom AI agents can also be trained to do things like help with customer service.
For example:

SEB, a major Swedish bank, now uses a virtual assistant called Aida to interact with millions of customers. Able to handle natural-language conversations, Aida has access to vast stores of data and can answer many frequently asked questions, such as how to open an account or make cross-border payments. She can also ask callers follow-up questions to solve their problems, and she’s able to analyze a caller’s tone of voice (frustrated versus appreciative, for instance) and use that information to provide better service later. Whenever the system can’t resolve an issue—which happens in about 30% of cases—it turns the caller over to a human customer-service representative and then monitors that interaction to learn how to resolve similar problems in the future. With Aida handling basic requests, human reps can concentrate on addressing more-complex issues, especially those from unhappy callers who might require extra hand-holding.(Source)

Brainstorming & Making Use of AI at Your Company

What processes does your company struggle with? It’s helpful to know your pain points before trying to solve them. Are you having trouble with hiring? AI can help identify better-qualified candidates to fill positions (Google Hire is now doing this). Are you not even sure what ways AI may be able to help? Start with a Google search (by the way, that tool is backed by machine learning).
Gather your stakeholders and have them help you envision ways to collaborate with AI. Be sure to help dissuade any fears that AI is here to take anyone’s job. Fear can be a big roadblock to creativity and success. This group should become champions of new AI ideas within your company, working with people around the company to integrate new directions.

AI is a Black Box—What About Accountability?

To a certain extent, AI tools are something of a black box, even to those who develop and use them. Increasingly, people want to hold companies accountable for the data they’re gathering and want to know how that data is being used. GDPR is a good example of the type of legislation that attempts to provide data accountability.
In addition to jobs for people who work with these new ML/AI tools, there are jobs for people who teach people how they work, so the outputs can be explained both internally and externally. People must be available to sustain and maintain AI systems, so those systems function properly and are used effectively.
The Harvard Business Review breaks it down like this: “The AI technology does what it does best, sifting through and processing copious amounts of data to recommend certain offerings or actions, and humans do what they do best, exercising their intuition and judgment to make a recommendation or select the best fit from a set of choices.”

Conclusion—Using ML/AI in Business

AI is here to say, and it’s changing the way companies do business every day. From analyzing massive data sets to interacting with customers, AI has many use cases. The only question is, “How will you use these new tools to stay ahead of the wave?”

When a page loads quickly, your website visitors get a better experience. Research supports this:

• According to DoubleClick by Google: “53% of mobile site visits are abandoned if pages take longer than 3 seconds to load.”
• Consider this Think with Google article, where a deep neural network found with 90% accuracy that, “as page load time goes from one second to 10 seconds, the probability of a mobile site visitor bouncing increases 123%.”
• Finally, from the Google AdWords blog, “In retail, we see that for every one-second delay in page load time, conversions can fall by up to 20%.”

Check out these page speed stats from three major brands:

(Sources linked here for Amazon; Walmart; Yahoo)
Page speed matters for SEO, too. If your site is very slow, your rankings could suffer. More importantly, conversions fall significantly when pages load slowly.

Do You Want Your Mobile Visitors to Be Happy?

Of course you do! In a recent Perficient Digital study titled Mobile vs Desktop Usage in 2018: Mobile Takes the Lead, we reported that mobile “has reached 63% of all traffic in the US.” If you’re not serving a mobile audience well, you’re missing out on significant web traffic.
Remember, mobile users are often on slower connections, so page speed matters even more. If you’re not serving mobile pages quickly, your visitors will leave your site and may even tell their friends about their poor experience.

Don’t Forget About Mobile-First Indexing

Google continues to push the importance of mobile and is currently rolling out its mobile-first index. This means Google will now use the mobile version of your website to crawl and base its rankings on. As more mobile traffic continues to arrive on your site, the importance of page speed will increase.
Keep in mind that for mobile-first indexing, your mobile site needs to have the same content accessible from desktop as it does on mobile. Google needs to be able to crawl and find the same links on your mobile site as they do when crawling your desktop version, so your mobile website hierarchy (i.e. the accessibility of pages in your website) is also very important.
Be sure your links remain crawlable and just as easily discoverable on mobile as they are on the desktop. Your main navigation should be just as usable and have the same links on mobile as it does on a desktop.

Google Provides Tools to Help with Page Speed

Recently, Google launched two new tools focused on mobile and page speed. These tools are available here: https://www.thinkwithgoogle.com/feature/mobile/.
One of these tools consists of competitive analysis, where you can see how your page speed stacks up against the competition. For example, look at five of the top online retailers, stacked up against each other by page speed:

Now, think about who is winning in this space. Most people would answer Amazon, and it’s clear that they are the winner with page speed as well.

The other new tool (the second on this Think with Google page) lets you see how much revenue you’re losing with each second of page speed decrease. On a positive note, this tool can also show you how much revenue you could gain by improving your page speed.
Let’s think about that 0.2 second page load difference between eBay and Amazon. It might not sound like much, but consider this: improving page speed by 0.2 seconds could increase eBay’s annual revenue by nearly $20,000,000:

(The above numbers are estimates based on data from average monthly organic traffic shown in SEM Rush, average order value for online shopping in the U.S. during Q3 2017 according to Statista, and a conversion rate based on U.S. conversion rates for Q1 2017 according to SmartInsights.)
Several tools are offered by Google and third parties to help webmasters improve their page speed, including:

• Google’s mobile speed testing tool
• Google’s PageSpeed Insights tool
• The Google Lighthouse tool
• Pingdom Website Speed Test
• GTmetrix tool
• WebPageTest.org

These tools can all provide valuable insights, but the Google Lighthouse tool and the WebPageTest.org tools stand out as particularly useful. Both provide the deep technical details needed when digging into what can be a complex topic. We’ll cover the Lighthouse tool and the webpagetest.org tool in some detail below but be sure to check out the other tools on this list as well.
[Tweet “Tools that help you asses your page speed are part of this complete Page Speed Guide.” quote=”Tools that help you asses your page speed are part of this complete Page Speed Guide.”]

Using the Google Lighthouse Tool

The Lighthouse tool goes deeper than page speed, and offers insights into accessibility, progressive web apps, best practices and more. The easiest way to get started with Lighthouse is to use the Chrome Developer Tools built into the latest versions of the Chrome web browser.
Once you’ve selected Developer Tools in Chrome, click on the last tab for “Audits,” where you’ll be able to click a button to perform an audit. Select one or more of the following options:

• Performance
• Progressive Web App (PWA)
• Best practices
• Accessibility
• SEO

Note, the tool will run on whatever website you have open in the active Chrome tab, so browse to the website you want to audit first and then select the developer tools to use Lighthouse.
The Performance report focuses on:

• First meaningful paint
  This “measures when the primary content of a page is visible.”
• First Interactive
  Currently in beta, this “marks the time at which the page is fully interactive.”
• Consistently Interactive
  Also in beta, it “marks the time at which the page is fully interactive.”
• Perceptual Speed Index
  “Shows how quickly the content of a page is visibly populated.”
• Estimated Input Latency
  “An estimate of how long your app (or webpage) takes to respond to user input, in milliseconds.”

Additionally, the Lighthouse report provides a list of opportunities specific to the webpage analyzed, and may also include:

• Lazy-loading of offscreen or otherwise hidden images
  Provides a list of specific images with their path.
• Reduce render-blocking stylesheets
  Provides the paths to specific CSS files.
• Reduce render blocking scripts
  Provides details to each specific script.
• Keep server response times low
  – Provides the time it takes in milliseconds for the server to send the first response.
  – Also referred to as Time to First Byte (TTFB)
• Save images in next-gen formats
  – Identifies several newer image formats which provide better compression than PNG or JPEG formats.
  – Recommended formats include JPEC 2000, JPEG XR, and WebP.
  – A list of images with potential kilobyte (KB) savings is provided.
• Unused CSS rules
  Identifies and lists unused CSS rules which can be removed from stylesheets to help save precious file size.
• Properly size images
  Saves file size by saving images in the sizes which they are used, rather than relying on CSS rules and the browser to resize.
• Minify CSS
  This identifies any CSS files which aren’t minified and estimates the potential KB savings.

Next, Lighthouse provides a diagnostics section with “more information about the performance of your application (webpage).” This provides additional opportunities to improve your page load time. Finally, Lighthouse provides a section detailing the audit items that your web page passed.

Using Webpagetest.org

This tool allows for some interesting tests, right off the bat. After entering your URL into the search box, you then select a test location, which also includes several specific mobile devices from both Apple and Android.
Here’s what the initial test entry form looks like:

The advanced test settings can get very detailed and include the ability to set the test’s connection speed. This allows you to find out how your page behaves on connections like:

• Cable
• 3g
• 4g
• LTE

You can also capture a video of your page loading, which is very helpful in determining which portions of your page become visible at various times during the rendering process.
Some of the advanced settings available include:

After running a test, you’ll be presented with a summary score with letter grades A through F, like this:

Each of these grades can be clicked on to get details for your specific test case. You’ll also be provided with a lot of metrics, such as:

• Load time
• First byte
• Start render
• Speed index
• First interactive (beta)
• Document complete
  – Time
  – Requests
  – Bytes in
• Fully loaded
  – Time
  – Requests
  – Bytes in
  – Cost

Test results also include a waterfall view, which breaks down the time it takes for each element on the page to load and visually shows you when each element begins and ends loading. This can help you identify bottlenecks in page load times.
Here’s what the waterfall view looks like:

Another helpful view is the content breakdown, which shows you what percentage of requests were taken up by various MIME (file identification) types, as well as a breakdown of total bytes consumed by each type. There is a similar visual view for a breakdown of domains, so you can see which domains are taking up the most requests and bytes when your page loads.
The content breakdown by file type view looks like this:

Overall, webpagetest.org provides a great deal of detail but is helpful for first time users as well.

What Can I Do to Improve Page Speed?

By now, it’s clear that page speed matters, and it matters a lot. The next obvious question is, how can you improve page speed?
While an in-depth look at optimizing page speed is specific to each website, there are a few top-level items to consider when optimizing.
Here are a few items to help you begin:

• Optimize images, CSS and Javascript
• Leverage browser caching & use server compression
• Reduce server response time (TTFB)
• Prioritize visible content with lazy load
• Use a Content Delivery Network (CDN)

[Tweet “Five Tips to Improve Page Speed (part of this complete guide to page speed for SEO)” ]
Optimizing images, CSS, and Javascript are all about compression. Basically, you’re looking to reduce the overall size of the resources on your web pages so they take less time to download. Leveraging browser caching means that once someone has visited your website, their second visit will be a lot faster because many of the on-page elements will be available to their device locally and don’t need to be downloaded again. Server compression utilizes your server’s resources to make elements smaller before they’re served to your visitors.
Reducing server response time, often called Time to First Byte (TTFB), is another way of leveraging your server to reduce the overall perceived page load time. A visitor makes a request for resources on your webpage, and the time it takes your server to respond is TTFB. The faster that server responds, the faster the requested page loads.
Prioritizing visible content with lazy load is a way of only loading the visible portion of a webpage and deferring the remaining parts of the page until after the above-the-fold part is fully rendered. One of the most impactful ways to do this is to defer the loading of below-the-fold images.
Finally, Content Delivery Networks help improve page load times by storing your resources on fast servers around the globe, so no matter where your users are located, there will be a fast server nearby ready to serve your page quickly.

Summary

The basic takeaway is that page speed directly impacts your users, which in turn directly impacts your revenue streams. Ample case studies and examples show just how large these impacts can be. Thankfully, you’re not alone and many tools are available to help webmasters get a handle on and improve page speed.

Last updated: April 12, 2019 – Pagination: Where would we be without it? If you’re running an eCommerce site of virtually any size, you almost certainly use pagination to organize your products. From eCommerce giants like Amazon and eBay to smaller niche brands, pagination is an essential component for both user experience and for search engines.
Aside from eCommerce, pagination is used in a variety of ways to help users navigate large amounts of data on a website and to help search engines better understand those sites. Examples where pagination is used include:

• Resource sites
  • Various industry or niche information
  • Technical documentation, press releases, industry articles, how-to guides, etc.
• Directory sites
  • The good kind, with useful information
  • Business directories, professional directories, etc.
• Blogs
  • Good blogs have regular and fresh content
  • Lots of content requires paginated pages to help organize the information

If you’re not quite sure what pagination is, it’s basically a connected series of pages with similar products or information, grouped together for ease of navigation.
For example, there may be 1,000 different men’s jeans in a category. Rather than displaying a long list of 1,000 products on a single page, pagination breaks them up into a series of numbered pages containing maybe 25 or 50 products each.

Pagination and Search Engines

In March 2019 Google’s John Muller announced that Google no longer took rel=prev/next tags into account. In fact, it appears that Google hasn’t taken these tags into account for, “a number of years” (source). Prior to this revelation, to ensure that search engines could properly crawl through to discover your product display pages (PDPs) and other high-quality content, you needed correct canonicalization and rel=prev/next tags set up to coordinate with each other. On March 19 here’s what John Muller said about rel=prev/next tags:

This begs the question; do we really need to worry about pagination at all? The answer is yes, but now it’s simpler since there’s no need to implement the rel=prev/next tags. When it comes to pagination and search engines, it’s important to get canonicalization and a few other technical details right. Small to mid-size companies may have thousands of products or pages, and enterprise giants often have millions.
Now, the most important element to ensuring search engines like Google can crawl your paginated pages, is to ensure that your paginated links render in HTML, rather than using a JavaScript implementation. Search engines are getting smarter about crawling pages rendered with JavaScript, but they have room for improvement. Having plain HTML links in your rendered page is a best practice for good SEO.
Another important element to consider with paginated pages and in particular with faceted navigation, is URL proliferation and crawl bloat. When a website offers filters for a product set for example, each filter has the potential to create a new unique URL and when filters can be combined a virtually limitless URL space can exist.
An excellent way to combat this URL proliferation is by using an AJAX solution. An AJAX enabled navigation allows for product filters to exist, but new URLs are not created for each filter selection. Rather, selected filters simply bring the filtered products back to the user view on the same URL. It’s faster for users and better for search engine crawlers.
Here’s a handy list of best practices for pagination and SEO, which I’ll cover in more detail below:

• Canonicalization
• Rel=Prev/Next tags (Now deprecated for Google, but possibly important for search engines like Bing.)
• HTML links and search engine crawl path
• JavaScript

Root Page Canonical or Self-Canonical

Here’s how your canonical tags should look:

• Root paginated page (the first page in your paginated series)
  • <link rel=”canonical” href=”https://www.domain.com/category“/>

• Second page in the series (first after the root)
  • <link rel=”canonical” href=” https://www.domain.com/category?page=2″/>

Continue the above pattern of canonical links on each page in your paginated series. Whether you have just two pages or 200, this simple pattern will help search engines like Google better understand your paginated pages.
Adding a canonical tag to your paginated pages is basically telling Google, “Hey, all these similar-looking pages are actually unique and valuable.”
Keep in mind that your paginated pages need to be unique and valuable, with unique products and/or content on each page. You shouldn’t have pages and pages of virtually identical content with minor variations and expect Google to index (let alone rank) those pages.
Rel=Prev/Next Tags No Longer Needed for Google
However, Bing may still make use of these tags. In addition, the use of these tags is still considered a standard by the W3C (source).
The next step in pagination and canonicalization is the use of rel=prev/next tags. Google initiated the use of these tags back in 2011, “to indicate the relationship between component URLs in a paginated series.” The purpose of these tags is to help search engines better understand your paginated series of pages. The proper way to implement rel=prev/next tags is like this:

• One tag on the root page, pointing to the next page in series
  • <link rel=”next” href=” https://www.domain.com/category?page=2″/>
• Two tags on page 2
  • <link rel=”prev” href=” https://www.domain.com/category”/>
  • <link rel=”next” href=” https://www.domain.com/category?page=3″/>
• Two tags on page 3
  • <link rel=”prev” href=” https://www.domain.com/category?page=2″/>
  • <link rel=”next” href=” https://www.domain.com/category?page=4″/>
• Repeat the process of implementing a rel=”prev” pointing to the prior page in the sequence and rel=”next” pointing to the next page in the sequence until the second-to-last page.
• One tag on the last page in your paginated series
  • <link rel=”prev” href=” https://www.domain.com/category?page=N”/>, where you replace the “N” with the page number for the second-to-last page in the sequence.

Be sure that on page 1, your rel=next tag points to page 2, and that on page 2, you have both a rel=next tag for page 3 and another rel=prev tag pointing back to page 1. This pattern of having both rel=next and rel=prev tags continues until the final page in your series, which only has a single rel=prev tag on it, pointing to your next-to-last page.
Another factor to consider is the robustness of your mobile site. To be sure your m-dot (m.mydomain.com) mobile implementation is fully crawlable, your rel=prev/next tags should be available on the m-dot version of your site as well.

Combine Pagination with Canonical and Rel= Tags

For the greatest opportunity to get your pages crawled and indexed in search engines like Bing, you should combine the use of pagination with canonical and the rel=prev/next tags. It’s critical that these tags work together, and that they aren’t in conflict.
As mentioned earlier, don’t put a canonical link back to your root page on all your paginated pages, because this would essentially tell the search engines that there’s only one page it should concern itself with. Even if you’re adding the correct rel=next/prev tags, but you’re getting the canonical link wrong, search engines will likely have trouble, because the tags would send conflicting signals.
Here’s what a correct implementation should look like:

Each page has a self-referencing canonical tag, and each page also has the appropriate rel=prev/next tags setup.
If you’re not concerned with traffic from search engines other than Google there’s probably no need to implement them. If you already have them setup correctly, you could leave them. However, if you’re unsure of the implementation then removing the rel=prev/next tags is probably a good idea.

What About Infinite Scroll?

In the last few years, infinite scroll techniques have allowed users to scroll through lists of information without needing to click through to the next page. This technique can work well from a user perspective, but it does require added consideration to ensure search engines like Google can access all the available pages.
To help developers and webmasters better understand an SEO-friendly infinite scroll implementation, Google’s Webmaster Central Blog published Infinite scroll search-friendly recommendations in 2014. John Mueller also put together this demo of how infinite scroll should be set up, so that Google can crawl and index your content.

JavaScript

JavaScript provides useful technology for the web, but it does present challenges for search engines. The basic consideration is a clean crawl path in prerendered HTML, which search engine crawlers can easily follow.
To a certain extent, Google does parse JavaScript. So, if JavaScript is used to inject paginated links as HTML into your page, that’s okay. However, what Google will not do is take any action that requires user input. This means that if your pagination setup requires any user interaction for the links to be generated, Google will not “see” any pages beyond page 1 of your series.
Another consideration with JavaScript is the time it takes for Google to index content which requires JS to render. Consider these comments from John Muller, “When Google crawls and indexes content it does two passes. The first pass looks at HTML only. Then, sometime later, it’ll do a second pass looking at the whole site. Mueller says there is “no fixed timeframe” between the first and second pass. In some cases it can happen quickly, in other cases it can take a matter of days or weeks.” (source)
Taking weeks to get your content indexed is likely not a risk you want to take, so be sure your HTML is presented fully rendered and that you’re not relying on JavaScript to render important content. If your site is heavily reliant on JavaScript, consider options like prerender.io which allows you to render JS on the server and still present plain HTML to users and search engines.

Make Sure It Works

Once you have taken all the technical aspects of SEO-friendly pagination into consideration, review your setup. Check Google Search Console to make sure Google is indexing the pages in your paginated lists. Manually click through your paginated links, checking that they work and render properly in HTML, with the appropriate canonical tags setup.
There are quite a few details you’ll need to get right, but once you do, your webpages will stand a much better chance of being fully indexed and ranked properly to drive traffic for years to come

Whether you’re running a global operation or expanding to offer an additional language on your website, proper use of the hreflang tag is a must. From an SEO perspective, getting this right can mean the difference between ranking well with your target audience and not ranking well.

Hreflang tells search engines where various language-specific versions of a webpage exist. Not only does this help your language/regionally targeted content rank, but it also helps reduce the likelihood of having issues with duplicate content. As a bonus, hreflang tags can help Google more quickly find and index content on your website.

[Tweet “hreflang helps language/regionally-targeted pages to rank but has other benefits too. Learn more”]

What Is the Hreflang Tag?

Hreflang is a tag defined by the Internet Engineering Task Force (IETF) as an HTML meta element that specifies language and regional targeting for documents. These tags are used by some search engines (like Google and Yandex, but not Bing), to better understand the regional and language targeting for a webpage and rank those webpages more appropriately.

The basic form of an hreflang tag is as follows:

<link rel=”alternate” hreflang=”en-GB” href=”http://example.com/page.html” />
The important critical piece to this is the “en-GB” code. That’s what lets Google know the language and region being targeted. In this example, the language is being specified as English (designated by the “en” portion of the “en-GB” text), and the geographic region is being specified as the U.K. (designated by the “GB”). ISO standards are used to specify the language found here in ISO 639-1 and the region found here in ISO 3166-1 using the Alpha-2 code.

How Do I Use Hreflang Tags?

There are three places you can put hreflang tags:

1. In on-page markup
2. In the HTTP header of your documents
3. In an XML sitemap

The best practice is to choose only one of these and to not mix implementations.
We’ll start with on-page markup. Let’s say your domain is registered in the UK and your business is based there, but you additionally operate in the United States and Mexico. To better serve your different audiences, you’ve translated your website content to meet U.S. and Mexican language standards. You now have three versions of each page on your website:

• UK
• United States
• Mexican

Unless you use hreflang tags, Google may see each version as intended for a UK audience, and may even see one or more versions as duplicate content.

To properly set up hreflang tags to target each page to its intended language and region, you’d place tags like these on every page across all three versions of your page:

<link rel=”alternate” hreflang=”en-GB” href=”http://example.com/page.html” />
<link rel=”alternate” hreflang=”en-US” href=”http://example.com/en-us/page.html” />
<link rel=”alternate” hreflang=”es-MX” href=”http://example.com/es-mx/page.html” />

Additionally, Google recommends specifying a default version of a page for users who don’t have a specified country or region. The default tag looks like this:

<link rel=”alternate” hreflang=”x-default” href=”http://example.com/page.html” />
It’s also important to know that you can specify a language by itself, but not a country code alone. This is important, so I’ll emphasize it again:

• You can specify language targeting alone
• You can specify language plus country
• You can not specify a country code alone

Another place you can implement hreflang is in the HTTP header of your documents. Here’s an example of what that can look like:

HTTP/1.1 200 OK
Content-Type: text/html
Link: <http://example.com/en-us/page.html; rel=”alternate”;
hreflang=”en-US”
Content-Length: 5710
(… rest of your HTTP response headers…)

Implementing Hreflang for a Site with Many Translations

The third way you can implement hreflang is in an XML sitemap. If you have a website with many translations, the on-page hreflang tag or HTTP header implementations outlined above can become cumbersome. For this type of situation, Google recommends using an XML sitemap implementation. The steps are simple. Here’s an XML example:

<url>
<loc> http://example.com/page.html</loc>
<xhtml:link rel=”alternate” hreflang=”en-GB” href=”http://example.com/page.html” />
<xhtml:link rel=”alternate” hreflang=”en-US” href=”http://example.com/en-us/page.html” />
<xhtml:link rel=”alternate” hreflang=” es-MX” href=”http://example.com/es-mx/page.html” />
<xhtml:link rel=”alternate” hreflang=”x-default” href=”http://example.com/page.html” />
</url>

Simply follow the above convention to set up an XML sitemap with hreflang implemented.
There’s no need to add the on-page tags or implement in the HTTP header in this case, but do be sure to add this information for all the XML links for translated versions of a page so hreflang works properly. In fact, adding these tags in multiple locations isn’t a good idea, because if one accidentally conflicts with another, this will likely confuse search engines and could lead to negative consequences.
[Tweet “Putting hreflang in an XML sitemap is best option for sites targeting many languages. More at” quote=”Putting hreflang in an XML sitemap is best option for sites targeting many languages.”]

Subtleties and Common Mistakes in Hreflang Use

Are you targeting content to Spanish speakers in the U.S., in Mexico, in Spain or somewhere else? Is your content partially translated, machine translated, only translated and not targeted, some combination of these or some other variable option?

Consider how your content is targeted and the quality of that translation. Be sure that:

• Your content is fully translated.
• You’re not using machine translations. (You can start with these, but have a native speaker edit the results if you do.)
• Your translated content is also targeted.
  • This means the content is not only translated into the new language, but it is targeted for the audience speaking that language.
  • Use appropriate regional idiosyncrasies and use appropriate spellings (like color vs. colour in English for U.S. vs. U.K. or Canada).
  • Find a few ways to make the content better targeted for the audience, as well as translating the content into their language.
• You pay attention to details.
  • Use the local currency symbols and system of measurement.

Another common situation for international sites is targeting different countries with the same language. For example, you may need to have pages in German, but target:

• Germany
• Austria
• Switzerland
• Belgium

This is a perfect use-case for hreflang tags, but it may become a bit cumbersome to implement on-page. We’ll cover the XML sitemap alternative a bit later on in this article. Also, keep in mind that each country, although they may speak the same language, will have its own uses of that language.
As mentioned above, pay attention to the details and keep your audience in mind. Don’t just use the exact same German translation for all German-speaking countries. You don’t need to fully re-write the content, but take the time to make a few strategic updates.

You can use hreflang tags for content on the same domain, on different domains, in sub-folders or sub-domains, etc. However, Google recommends consistency. Choose one convention rather than mixing them.

For example, if your alternate content is in subfolders, don’t mix in alternate content on subdomains as well. It’s worth noting that Google does not recommend using URL parameters (like ?=en-US) for hreflang: “Parameters can be overloaded, more difficult for search engines to understand.” (source).
Also, take care to avoid these common mistakes when implementing hreflang tags:

• Using invalid country or language codes (like U.K. instead of G.B.)
• Mixing the order of codes (always specify language first and then country)
• Specifying a country code only (as noted, you may specify a language only, or a language and a country, but never a country only)
• Missing bi-directional links (if page A has hreflang links to page B, then page B must link back)
  • These may be called “link clusters” and it’s important that they be clearly linked together for hreflang to work properly

How Should I Use Hreflang with Canonical Tags?

The answer is simple. Basically, use self-referencing canonical tags just as you normally would. For example, on each of your various page translations, add a self-referencing canonical like this (assuming you don’t need to enter another page as the canonical—we’ll cover what to do in that case in a moment):

• <link rel=”alternate” hreflang=”en-GB” href=”http://example.com/page.html” />
  • <link rel=”canonical” href=”http://example.com/page.html” />
• <link rel=”alternate” hreflang=”en-US” href=”http://example.com/en-us/page.html” />
  • <link rel=”canonical” href=”http://example.com/en-us/page.html” />

If you have a page that has a canonical version elsewhere, you should not implement hreflang tags on this page, but instead only place the tags on the canonical version of the page.
To learn more about implementing hreflang correctly, watch this video by Eric Enge (or read the transcript here):

Is Hreflang Worth the Effort?

While there are quite a few different ways to implement hreflang, and many technical considerations to keep in mind with each, it’s very much worth tackling. Consider that if you already have content in multiple languages, or in the same language for different countries, then you may risk a duplicate content issue.

Also, consider that by adding hreflang tags, Google is much more likely to rank your content for the specified audience in the specified country. There’s no substitute for proper hreflang implementation when it comes to ranking content.

Author John Dietrich is a Marketing Consultant at Perficient Digital.