Yesterday a client asked me what Key Performance Indicators (KPIs) they should be tracking on their public site search. At first I was afraid I would not be able to answer this question – I don’t know their business or website nearly as well as they do. I know the wide variety of statistics that are possible to capture, but what should I recommend to this particular client?
I thought for a moment and realized they needed to know their goals for search before diving headfirst into any analysis. There is no point capturing and analyzing search statistics unless you need them. They should have a clear reason or objective behind each KPI they track. Second, they needed to think in terms of deltas or comparisons instead of raw numbers or simple orders of magnitude. Statistics like quantity of searches or frequency of keywords are unlikely to reveal anything by themselves, but observing something increasing or decreasing over time can tell a very interesting story. Likewise, comparing any certain statistic between users that ran a search vs. users that did not run a search can be very revealing.
So what are some good search goals or objectives? Well, think back to why you put a search box on your website in the first place. It might have been to decrease how long it takes users to find a desired page. In that case, you might be interested in the number of pages people visit before exiting the site. Running a search and then visiting a small number of pages, or having a short click stream, could indicate a quick, successful visit. Running several searches and visiting many pages per search could indicate a user having trouble finding the desired information. Try tracking rapid, successive page-views. Compare the results for users that ran a search against users that didn’t run a search. If users that ran a search spent less time rapidly hunting through pages, it could be considered a win.
Another goal might be to increase sales conversions or increase sign-ups or even just get more items into a shopping cart. Search might have been added to the site with the hope of making it easier for people to find the product they want, or easier to find information about signing up for a service. Try tracking these activities during a certain time period for users that ran a search vs. users that didn’t run a search. If, for example, users that ran a search added 20% more items to their shopping cart, you can go brag to your boss. You can use similar techniques to validate changes or tweaks to your search. Tracking the average number of items added to the shopping cart or the total number of sales before and after a certain change to the search engine can let you know if the change was positive, negative, or neutral.
One final goal might be site stickiness or user engagement. You might want to track repeat users of search (but not necessarily in rapid success – because that could mean futile searches). Detecting users that repeatedly come back to the site and start their visit with a search might a good sign that search is working as an effective navigational aid or is helping users locate information they want. For example, I can’t remember that last time I started shopping on Amazon.com by browsing. I always, 100% of the time, start with a search. Same for eBay or Etsy, or even tech documentation sites like code.google.com. If the search doesn’t work, I probably would not visit these sites as often. Good search is one of the primary reasons I use them and come back again and again.
In conclusion, think about search statistics that tell a story about your site or about your users. Track them over time to monitor trends, or compare results between audiences that ran a search vs. didn’t run a search. Those comparisons can reveal the effectiveness of your search solution.
Zen and the Art of Search Statistics
by Chad Johnson on October 10th, 2014 | ~ minute read
