Key Interview Points
One of the things I learned when I was out at SMX Advanced in June was that the Quality Score that you see in the search engine PPC services (both adCenter and AdWords) was that the Quality Score you see displayed in your account is not the same as what is used by the engines in ranking your ad. For that reason, I asked Ping Jen of the adCenter team to join me for an interview. Below we talk through exactly how Quality Score works in adCenter. Here are the key points from the interview:
- Original content, content relevance to the ad, location, and layout are all factors in landing page relevance.
- Advertisers whose pages are deemed to be harmful will get banned from the adCenter marketplace.
- (Ping Jen) “Our philosophy is that we want our advertisers to have high ROI, and one of the ways we do that is by requiring them to have higher quality landing page user experiences and relevance. To help advertisers, we provide feedback through the Landing Page User Experience subscore and Landing Page Relevance subscore.”
- (Ping Jen) “If you have some outliers within an ad group or campaign, determine why. Should I use this keyword? Does it belong to this ad group or campaign because usually KWs that share an ad group are tied to the same landing page? In some cases, it may be time to move those keywords to another ad group because they don’t fit into this landing page.”
- (Ping Jen) “Before clicking your ads, search users will look at the content of your ads. Immediately, they can see if they are relevant to what they are looking for. We follow the same logic to validate your ad copy relevance.”
- (Ping Jen) “badly spelled ad copy immediately reduces the confidence a user will have with the ad and they will shy away from it.”
- (Ping Jen) “Placement is still determined by relevance, the landing page experience, the historical CTR and the advertiser’s price.”
- (Ping Jen) “We have always been upfront that the adCenter Quality Score is not directly tied into a rank score.”
- (Ping Jen) “How do you know your KW performance against others bidding on the same term? We tell you with the keyword relevance sub score.”
- Rank score, which is the term adCenter uses for the actual method used to determine ranking, is calculated on a marketplace by marketplace basis. This is done because the needs of each marketplace are different.
- (Ping Jen) “Ads must comply with the adCenter Relevance and Quality guidelines. Then the ad’s competitiveness in KW relevance, landing page relevance, and their bids will decide their ranks.”
Landing Page Quality and Relevance with adCenter
Eric Enge: Can you give us an overview of how Quality Score operates in adCenter?
Ping Jen: Our Quality Score is a strong signal of campaign quality and performance. The reason we introduced the adCenter Quality Score was to help our advertisers enhance campaign performance and raise the visibility of improvement opportunities.
We consider campaign quality an important factor and we want to showcase the best experiences in the marketplace and continue to grow the traffic volume and increase market share.
Eric Enge: How do you measure that landing page user experience? What factors are involved?
Ping Jen: If you Bing “adCenter relevance and quality guidelines”, you will find that adCenter has published very specific requirements for landing page and user experience. We measure the Landing Page User Experience and then validate whether advertisers have followed the guidelines and show the results through the Landing Page User Experience subscore.
Eric Enge: In the landing page guidelines you recommend analyzing the content and the user’s interaction with the content to make sure there is value.
Ping Jen: Yes, besides original content, content relevance location and layout is also very important.
Eric Enge: Your guidelines warn against too much advertising on the page, SEO manipulation, and doorway pages. You can break these guidelines into two categories.
One category is pages that are overtly harmful; for example, pop-ups, parked sites, and automatic software downloads. The second category includes guidelines that point towards an evaluation of the quality of the user experience when they click on an ad and arrive at the page.
Ping Jen: For the first category, we will prevent them from our marketplace if they create a harmful experience. For the second category, if everything else is equal we want to promote the content that provides the best user experience.
Eric Enge: At the SMX Advanced panel you and I attended recently, either Frederick Vallaeys of Google or Craig Danuloff indicated that landing page evaluation was not much of a factor in the Google Quality Score algorithm. I am not asking you to comment on Google’s approaches, I simply want to bring up the fact that adCenter’s approach appears to be different than the Google approach.
Ping Jen: Our philosophy is that we want our advertisers to have high ROI, and one of the ways we do that is by requiring them to have higher quality landing page user experiences and relevance. To help advertisers, we provide feedback through the Landing Page User Experience subscore and Landing Page Relevance subscore.
Eric Enge: It’s the key differentiating point in terms of driving a higher ROI.
Ping Jen: Correct, through Quality Score, adCenter aims to help advertisers improve their landing pages relevance and show ads accordingly. Initially, the traffic volume may not be at its full potential but the traffic that comes to our advertisers is good traffic. In time advertisers will be able to achieve both quality and quantity goals by leveraging Quality Score.
Eric Enge: Can you tell us more about landing page relevance?
Ping Jen: Yes, adCenter has devoted a lot of resources to analyze landing page relevance and shares the findings through landing page relevance subscore. We want to help advertisers align their landing pages content with user intent. When a search query such as “golf” comes in, adCenter analyzes what golf means, which may be different context for different people.
For example, golf could be an event, sporting goods, a clothing line, or a golfer like Tiger Woods. The searcher might be looking for a golf vacation or information on the Volkswagen car. We can analyze these possible user intents from the search query side and validate the landing page to see whether you have enough information on your landing site to engage user intent for all possibilities.
Eric Enge: If an advertiser has an ad for golf and the keywords aren’t clear, you can look at their page and understand whether they are right for a golf clothing line or golf course query and perhaps target when their ads show. If it’s too vague, you can lower their Quality Score because a generic word like golf would be used by people for many purposes and, if they only match one of those, they will only fit a percentage of those users.
Ping Jen: We will list all the possible intents behind the term golf and then assess your landing page to validate whether it has enough related information to engage any of these possible intents. That’s the basics behind our landing page relevance check. adCenter has invested a lot of resources enhancing this feature so I strongly encourage people to take a deeper look at their Landing Page Relevance subscore. They can use this subscore as a strong signal to say “hey, do I have good information on my landing page to convert a search user for this term?”
Eric Enge: If someone has a poor landing page relevance score they have work to do, right? However, there are many reasons why their score could be poor. Are there tools you can suggest that would help them break this down and figure out what aspect is causing their landing page subscore to be low?
Ping Jen: First, look at the whole picture. At the campaign and ad group levels, do you have a good landing page relevance or not? If you can say this ad group or campaign is pretty solid, that’s the first step. If the answer is no, you need to reconsider your KW selections and how they are tied to landing pages.
Second step, look closely at the details. If you have some outliers within an ad group or campaign, determine why. Should I use this keyword? Does it belong to this ad group or campaign because usually KWs that share an ad group are tied to the same landing page? In some cases, it may be time to move those keywords to another ad group because they don’t fit into this landing page.
Role of Ad Copy in adCenter Quality Score
Eric Enge: What about the role of an ad copy on your overall score?
Ping Jen: We check the relevance of your ad copy as well.
Eric Enge: What are some of the aspects you look at inside the ad?
Ping Jen: There are simple ways to identify whether your ad copy can engage with user intent. Before clicking your ads, search users will look at the content of your ads. Immediately, they can see if they are relevant to what they are looking for. We follow the same logic to validate your ad copy relevance.
Eric Enge: I see one of the things you focus on includes correct grammar.
Ping Jen: Absolutely, badly spelled ad copy immediately reduces the confidence a user will have with the ad copy and they will shy away from it.
Eric Enge: Many people type in search phrases which are misspelled. One popular technique is to show the user’s keywords back to them exactly as they typed it. Is that something you recommend?
Ping Jen: No, we identify possible misspellings and what could be the correct term first. We then ask the search users if this is what they are looking for, and then show ads based upon search user’s choice.
Eric Enge: I’d like to talk more about the golf scenario. Let’s say you have someone who promotes golf vacations. A user types in the generic keyword “golf” and the golf vacation company wants to bid on it. A popular technique in your ad tells them they entered golf but your ad is about golf vacations so you screen out the people who were looking to buy golf clubs. Do you encourage this approach?
Ping Jen: Absolutely. Ad copy is advertisers’ first line of defense to low-value traffic. Having the search user help filter out unwanted traffic is a good technique that we recommend.
Eric Enge: A more dramatic example is when someone enters the search phrase “jaguar.” As you know, this is an animal, a football team, an operating system and a guitar. No one is going to cover all these situations so how do you handle this scenario?
Ping Jen: First we identify what the possible user intents are behind it. We assess advertiser’s landing pages to see whether those pages match up with any of possible intents.
Eric Enge: So would you show the most popular meaning when people type in jaguar, which might be the animal?
Ping Jen: No, we don’t try to alter the ad placement on this basis. Placement is still determined by relevance, the landing page experience, the historical CTR and the advertiser’s price.
Quality Score and Ad Position
Eric Enge: In terms of how the Quality Score is used, is it used to help determine the position of an ad or only whoever the ad shows or not? At SMX Advanced Craig Danuloff told us that Google Quality Score is not the Quality Score they use to generate rank score.
We have always been upfront that adCenter Quality Score is not directly tied into a rank score.
Ping Jen: That’s correct. That was news to a lot of people. However, there are good reasons why it’s almost impossible to have the displayed Quality Score determine the ranking of the ad. We have always been upfront that adCenter Quality Score is not directly tied into a rank score.
Eric Enge: So are you saying the Quality Score you display is a hint.
Ping Jen: It’s an indicator of how competitive your keyword is in our marketplace.
Eric Enge: If you do things to improve the Quality Score you see inside of adCenter, then you will probably be moving in the right direction?
Ping Jen: It’s not probably; you are definitely going in the right direction to make your KW/ad copy more competitive and enhance your landing page relevance and user experience.
Eric Enge: But it’s not a one-to-one with what is actually used in the algorithm.
Ping Jen: That’s correct, but it is a very strong indicator.
Click-Through Rates and the Keyword Sub Score
Eric Enge: Do the click-through rates play a large role in the algorithm?
Ping Jen: When we decide adCenter Quality Score, we also consider the advertiser’s expectation. They look at their CTR as a strong indicator of how their ad performed so we respect that notion and make sure adCenter Quality Score is moving consistently with their CTR performance.
Eric Enge: So the rank score is derived from a combination of factors including keyword relevance, ad copy, landing page, click through rate and, finally, the bid price.
Ping Jen: Correct. Hopefully, you can help me and adCenter to drive this message home to the heart and mind of every advertiser. The message is very straightforward. Landing page user experience and landing page relevance is the cornerstone of the search alliance marketplace. We want to make sure you meet our requirements and grow together with our marketplace.
After you enter our marketplace, you have to compete with others looking for the same traffic. How do you know your KW performance against others bidding on the same term? We tell you with the keyword relevance subscore.
You have three different results, poor, no problem, and good. With the search term “jaguar”, the CTR average is 10% so if you are about 10.2 or 9.8 you are average. If you are below the marketplace average for this term you are poor. If you are better than the market average you are good.
It is a useful tool to gauge your performance. Occasionally, people come to us and say I have a CTR of 5% so why are my ads not showing, or why isn’t it in the number 1 position? The answer is the CTR average for your keyword is 10%. Some people come to us and say I only have 0.8% so why do I have a high-quality score? I tell them it is because they are doing better than the 0.2% average and have “No Problem” on landing page relevance and landing page user experience.
Eric Enge: Right, because you calculate the average performance based on each marketplace?
Ping Jen: Each specific term. There is a lot of nuance behind our Quality Score when compared with Google. This is a major difference.
How positioning works in adCenter
Eric Enge: If we were to write down on a piece of paper which ads show first, which ads show second, and how that is calculated, what would that look like?
Ping Jen: Ads must comply with the adCenter Relevance and Quality guidelines. Then the ad’s competitiveness in KW relevance, landing page relevance and their bids will decide their ranks.
Eric Enge: Is there anything else you would like to add?
The adCenter Quality Score is a channel we use to communicate to the advertisers they can monitor for improvement opportunities.
Ping Jen: I want to make sure people understand that the adCenter Quality Score is a channel we use to proactively communicate to the advertiser. For example, your ads are losing strength compared to your competitors or due to a new policy you suddenly see your user experience drop down to poor.
I encourage all the adCenter users to closely monitor their quality score and provide feedback to us on how we can fine-tune the signals.
Eric Enge: Thanks Ping this was very helpful and we appreciate you taking the time to chat with us today.
About Ping
Ping Jen is a Product Manager on the Microsoft Advertiser and Publisher Solutions Team. He has a passion for driving improvements into adCenter which helps advertisers optimize their campaigns and increase their competitiveness in the marketplace. Prior to joining Microsoft in 2009, Ping was a Business Administrator at the University of Cincinnati Department of Neurosurgery. Ping is a Microsoft Certified Solution Developer (MCSD) and holds an MBA degree from the University of Notre Dame.
Other Interviews
Duane Forrester, September 7, 2011
Danny Sullivan, August 8, 2011
Bruce Clay, August 1, 2011
Google’s Tiffany Oberoi, July 27, 2011
Mona Elesseily, July 18, 2011
Vanessa Fox, July 12, 2011
Jim Sterne, July 7, 2011
Stephan Spencer, June 20, 2011
SEO by the Sea’s Bill Slawski, June 7, 2011
Elastic Path’s Linda Bustos, June 1, 2011
SEOmoz’ Rand Fishkin, May 23, 2011
Bing’s Stefan Weitz, May 16, 2011
Bing’s Mikko Ollila, June 27, 2010
Yahoo’s Shashi Seth, June 20, 2010
Google’s Carter Maslan, May 6, 2010
Google’s Frederick Vallaeys, April 27, 2010
Matt Cutts, March 14, 2010
