When I saw Chris Penn speak at our 2018 Next10x event, I knew I wanted to get together with him for a video session to discuss machine learning and AI. We did that recently, and the videos below are the result. Each video is accompanied by a transcript if you prefer to read.
We covered seven major topics in our discussions:
- What Are Machine Learning and AI?
- How Will Machine Learning and AI Change Our World?
- Google and AI: RankBrain
- Google and AI: Beyond RankBrain
- The Rise of Smart Devices
- Predictive Analysis and Content
- Real Applications of Machine Learning and AI Today
As you can see, we covered a wide range of territory. Throughout these conversations, we’ll give you a solid introductory primer into machine learning and AI, as well as an understanding of how companies and individuals are applying them in the world of search. I hope you enjoy these conversations as much as I did.
Conversation #1: What are Machine Learning and AI?
Eric: Chris, my first question for you is, “Can you just tell the viewers a bit about machine learning and AI, first of all, and how they’re different?” and then, “Where are they today?”
Chris: Gotcha. AI is a big broad umbrella term. It basically means getting computers to do things humans do with our intelligence naturally. So you can see me, and if you’re watching this, you can see what’s going on. You’re using vision. If you’re hearing the words I’m saying and it doesn’t sound just like noise, then you’re using natural language processing.
We learn these things instinctively and through our own training as we grow up. But, we’re trying to teach computers to do those things. Now under that umbrella, the foundation of today’s AI and machine learning is all statistics—it’s all math, right? So, if you didn’t like math, sorry.
The good news is software’s helping with a lot of that. Statistics and probability are really the heart of artificial intelligence. With those individual statistical techniques, we build them into what are called algorithms—repeatable processes. Everybody uses algorithms all day long. When you get dressed in the morning, you probably have a sequence of things that you do every day that is predictable. And so that’s the algorithm.
You get into the interesting territories when you give computers these algorithms and you give them data and you say, “Hey, you decide what algorithms to use to make the data reach a conclusion of some kind.” And that’s what we call machine learning where the computer now, instead of we write the software and the computer processes the data, now we provide the data and the computer writes its own software and then comes out with an outcome.
Now, if you were to take machine learning as a layer of a pancake,—a set of algorithms,— and you were to start stacking them on top of each other like Lego blocks, where the data moves from one block to the next, that’s what’s called deep learning. It’s called deep because it can be hundreds of layers deep. That gets computers much closer to either human level or beyond human level capabilities.
[Tweet “Deep learning is like a stack of machine learning Lego bricks, each brick passing information to the others. More on AI from @cspenn” quote=”Deep learning is like a stack of machine learning Lego bricks, each brick passing information to the others.”]
So, when we’re talking about AI, particularly in terms of marketing and search and things like that, we are talking about computers being able to think like humans, create outcomes that humans want, and optimize for those outcomes. A lot of what we’re going to be talking about today deals with how that impacts things like search.
Eric: Absolutely. One of my favorite examples is when Google’s DeepMind subsidiary cracked the code on the game of Go and taught it to beat the world champion. That was an intense machine-learning exercise right there.
Chris: Yes, absolutely.
Conversation #2: How Will Machine Learning and AI Change Our World?
Eric: For the next question, can you talk a little bit about how this is going to impact our world, in terms of the types of jobs and the kinds of things that will change in our environment, overall?
Chris: In the future, there will be two kinds of jobs. You will manage the machines or the machines will manage you. And that’s pretty much the future for everything. If there’s a process or a task that you do that is repetitive, a machine will do it. At some point, a machine will do it because it’s really not worth a human copying and pasting the same thing over and over again.
[Tweet “In the future there will be just two kinds of jobs: you will manage machines or machines will manage you. More form @cspenn on AI and machine learning:” quote=”In the future there will be just two kinds of jobs: you will manage machines or machines will manage you.”]
Think about, for example, in the world of search marketing. What are some of the things that we would do in search marketing? We do stuff like keyword scoring, keyword analysis, and text analysis. All that is stuff machines can do. You don’t need a human to do that anymore.
Another one of the things I think is relevant is if you ever Googled for an Instagram template or an SEO checklist, things like that, right? If you use a template to do your work today, a machine’s going to do it without you tomorrow because you just don’t need to be doing those things anymore. So that’s a big part of the future.
And the most important thing, I think, is that from a marketing and communications perspective, marketing becomes truly one-to-one. We can’t scale. You and I can’t individually talk to a million people every day. You and I are having this conversation here. We’re having a one-to-one conversation, but we can’t do this at scale. We don’t scale. There’s just not enough hours in the day.
But an AI can actually do that and have a meaningful interaction with somebody on a one-to-one basis. Whether you’re searching for something or talking to a voice assistant, you can have these interactions one-to-one and the machines can remember who you are.
For one of my favorite examples, go to Google and look up Watson Conversational Ads. It’s an IBM product. Disclosure: I’m an IBM champion, so they send me clothes to wear.
You can talk to the ad. It’ll ask like, “What’s your favorite ingredient?” And I’ll type in “Sriracha” and they’ll come up with a recipe, on the fly, that’s just for you based on the time, the weather, your search history, and things like that. It’s your recipe, but it also warns you these are not kitchen-tested—use common sense. But that’s one-to-one marketing and that’s how this is going to impact everything going forward.
Conversation #3: Google and AI: RankBrain
Chris: I want to get back to something that we were talking about in terms of search. At the Next10x conference, we were talking a bit about RankBrain and how Google is using AI. What have you seen the big search engines doing with AI and machine learning as it impacts marketers?
Eric: I’m glad you started with RankBrain because there’s a little bit of a myth out there and Google picked a really unfortunate name for it. But the original RankBrain algorithm is what I call a “sparse data algorithm,” and it was really about providing better answers for the kinds of search queries that users enter for which they don’t have data.
So the way that worked is it would actually look at historical search queries, especially on long-tail queries. It might be five, six, seven words long, or even longer queries. Nobody had ever done these queries before, but they could do what they call a similarity vector analysis where they look at the vector for the query entered by the user based on the words.
They might have a similar query where the vector, when they draw it, is really similar. So mathematically, they’re able to determine that these queries are extremely similar. This is building on what you said a moment ago, just doing the statistical analysis.
Looking at those two very similar queries, Google could then actually see how people responded to the other query. Do they not click on the first result? Do they ignore the eCommerce results?
Do they click on the informational result? And based on that, they can tune how they give you the results for the query you actually entered. This is where RankBrain started years ago.
The interesting thing is, this got confounded a little bit more because Google made the statement that it was the number-three ranking factor in the Google algorithm. By the way, the first two, they said, were content and links.
Chris: Huge surprise.
Eric: Which is good. The world hasn’t been turned completely upside down yet. The reality is we have to remember, 70% of all search is in the long-tail. So if RankBrain operates primarily in the long-tail, it can actually have a very large impact but not change ranking for higher volume queries at all, which is basically what they tend to say about it.
[Tweet “70% of search takes place in the long tail, and that’s where RankBrain comes in.” quote=”70% of search takes place in the long tail, and that’s where RankBrain comes in.”]
Chris: But here’s the thing. The way we search is radically changing. So that inflates RankBrain’s importance. Today, when I talk to Google Assistant, I don’t say “best SEO firm.” I don’t speak in these short-clipped phrases. I’ll say, “Hey, Google, what’s the best SEO firm in Framingham, Massachusetts?” Right? It’s a very long-tail query. So does that mean that RankBrain is processing a lot more of the voice interface and the voice searches?
Eric: I think that’s likely the case. As you know, voice queries tend to be much more natural language and much longer, and as a result…Yes, it’s going to trigger RankBrain even more.
Conversation #4: Google and AI: Beyond RankBrain
Eric: I happen to think RankBrain is evolving.
Chris: Into what?
Eric: What we’re seeing now is this idea of comparative analysis and being able to look at query histories using machine learning and AI, and that is particularly interesting. It allows them to try out the idea of experimenting. Let me replay it briefly. RankBrain was looking at past historical query results and learning from them to tweak your results.
Chris: Based on a vector word analysis.
Eric: Now, let’s make a simple modification to that concept and actually run an algorithm where we test certain kinds of listings, see how they perform, and compare them to tests of other kinds of listings. I’m again looking at historical results, but rather than going in the databank and hoping that I have a related phrase that I’ve done something with, I’m going out of my way to dynamically test scenarios.
Chris: Well, yes, we know they do that. They do that with Markov chains in the Attribution 360 product. It’s built right in and they do hundreds of millions of comparisons of all your data based on your past data.
Eric: I think they’re being much more deliberative about that now in what they’re doing with search results.
Above is a screenshot of the search results for the phrase “digital cameras” from February of 2018. What you’re seeing is that there are two reviews results and two eCommerce results. By reviews results, I mean pages giving reviews of lots of different digital cameras.
Fast forward to May of 2018, and it’s changed dramatically. Now we have three eCommerce results, no review page results, and Wikipedia. I’ve seen this for many, many different kinds of SERPs (Search Engine Result Pages) in a way that I’ve never seen in Google before. It’s happening more dynamically. So it’s my conjecture—I have no confirmation, to be fair—but it really looks to me like
they’re deliberately testing scenarios to better determine user intent.
[Tweet “It looks like Google is testing user intent assumptions and adjusting search results in response to the tests. Learn more about how AI is changing search:” quote=”It looks like Google is testing user intent assumptions and adjusting search results in response to the tests. “]
Chris: How do you get around the issue of personalization in the results? When you’re advising clients, do you provide something like, “Here’s the generic, not logged-in result,” and then here’s 12 or 15 personas of standard business users or standard homeowners to show how the results will vary from person to person?
Eric: It’s actually hard to do specific SEO work around personalization. But really, it ultimately all gets back to user intent, and how well your content matches up with user intent. This is something that I think a lot of businesses are dramatically under-invested in because when someone comes to your web page, they’re looking for something. And it might not be just the top-level product on your web page, but all the ancillary needs that they have related to that.
Chris: Do I need a digital camera and…
Eric: Well, I was going to say an SD card, right? I almost said film. I was dating myself terribly there.
Chris: No, you could be retro.
Eric: Well, I could be retro. That would have been an embarrassment. Oh, wait a minute. I did say it. Yes, you have other needs, and you have other things that you’re looking for. So you have to design your content to meet that broader range of needs.
And this, I actually think, is the thing that helps the personalization part of the algorithm work in your favor, because if you’re creating the content that they engage with initially, because you do a good job of putting out there that you’re addressing a broad range of needs, then you’re putting yourself in the situation where the personalization algorithms work in your favor.
Conversation #5: The Rise of Smart Devices
Chris: Now, let’s talk a bit about some of the smart devices like Google Home and Alexa and others in that world. How should we be optimizing for these devices, for these much longer tail searches?
Intent is a big part of it because obviously, if I don’t have to think about what I’m typing, I would say, “Hey, what’s the best SEO firm in Framingham, Massachusetts that accepts B2B clients?” That’s a very long search term, and there’s a lot more rich intent in there than “best SEO firm.” The intent is unclear.
So how do we optimize to take advantage of all these different types of intents that people are going to physically speak into their smart home devices, their watches, even people talking to their refrigerators now?
Eric: Absolutely—my car, right?
Chris: Exactly
Eric: And my watch; I’ve got them all, all those devices. I think one of the big things people have to realize is when you’re dealing with Alexa or the Google Assistant running on Google Home or something running on your smartphone, and you use a voice query and you get a voice response, you get one answer. You know this, right? This is the big thing.
Most of the time, the great majority at that time, when it’s a Google Assistant answer, they’re drawing that from what they serve as featured snippets in the regular search results. So the big thing to do is learn how to generate featured snippets.
But let’s back up and look at this from a Google perspective and how they’re thinking about it. It used to be that when they served regular search results, if the first answer in the search results wasn’t perfect but the user got what they wanted in position two or three, that was actually still a good result for the user. They don’t have that opportunity in the voice environment. They only get one answer. I happen to think that they’re investing in an enormous amount of machine learning…
Chris: Duplex!
Eric: …technology. What’s your take on that?
Chris: I think you’re absolutely right. And I think one of the things that marketers, in particular, are neglecting is the data they already have. So we’ve been doing a couple of projects, mining people’s CRM data, like the stuff that people call in or email in: “Hey, I’ve got a problem with this product or service.” If you mine that data and you pull out the way people are talking to you about your stuff on your website, that is rich search content to fulfill intent, right?
[Tweet “Mining data from your CRM and email using machine learning can yield new search-friendly content opportunities. Learn more from @cspenn at” quote=”Mining data from your CRM and email using machine learning can yield new search-friendly content opportunities.”]
Because you know when somebody searches for SD card class 10, what they’re really asking about is, “How do I have a card that doesn’t cause frame rate issues when I’m recording a video or setting up a security camera?” or things like that. And so, if you mine your CRM data, and you’ve got a whole pile of emails that say, “Hey, I’ve got jittery video,” now you can go back and reinfuse your content that’s public or search volume indexed with that intent to say, “My video’s stuttering.” “Okay. You need a Class 10 card,” and things like that.
I don’t see companies doing that. People are sitting on these years or decades of CRM data and they just let it sit out there and just cost money as storage cost and let it be a security risk, as opposed to saying, “Let’s use this to inform search and marketing and communications.”
Conversation #6: Predictive Analytics and Content
Chris: The other thing I don’t see people doing, or hardly any of, is predictive analytics. This is a problem that marketing automation software has made worse. People assume that everybody who is qualified to buy is ready to buy all the time, right? You’re the CEO of a company, right? So clearly, you’re qualified. You’re the decision-maker. So we’re just going to assume that you’re ready to buy.
Well, no. I mean, if you are a CEO, you’ll have ebbs and flows and things throughout the year, particularly if you’re publicly traded. You have a quarterly calendar you have to go by. And so by using predictive data, especially based on search data, which is reliable─ and well, people ask Google things they would never ask another human being out loud─ you would get a much better sense of when somebody’s doing something good.
I think part of the intent and part of the search results that we’re talking about is that people don’t take into account time. When is somebody searching for an SEO firm? When is somebody searching for a marketing firm? When is somebody searching for a new car? I would be completely surprised if Google did not take into account time in its results.
[Tweet “Predictive analytics can go beyond what people want to when they want it. Effective marketing shows up just when people are ready to buy. Learn more from @cspenn” quote=”Predictive analytics can go beyond what people want to when they want it. Effective marketing shows up just when people are ready to buy.”]
Eric: Yes, I agree and I think people are dramatically under-invested in content. I mentioned this earlier.
Here’s a case study with a search visibility chart pulled from SearchMetrics for a company that we happen to believe has about 15 full-time, knowledgeable content generators putting up over 100 articles a month on their site addressing specific questions and aspects of topics that users have in their market space.
When you look at this, it’s crazy, the traffic lift. They launched in May of 2016 and they’ve actually already achieved dramatic search visibility by understanding what you were just talking about and investing in answering the real user questions.
Conversation #7: Real Applications of Machine Learning and AI Today
Eric: Why don’t you talk to us a little bit about how you guys are using machine learning in your business today.
Chris: It’s really three things.
- Predictive analytics: when is something likely to happen or what drives something?
- Text mining: understanding what’s in the data you already have. There’s so much data you’re sitting on. Please do something with it. Don’t just put it in a digital filing cabinet to rot forever.
- Attribution analysis: the same technology Google uses, the same algorithms like Markov chains and Monte Carlo simulations, you can do on your laptop, though you won’t do it at a Google scale. But you can do it enough to do really good attribution analysis and get a very clean picture of what’s working. What’s really surprising is, in particular, search traffic and referral traffic are so under-weighted in most people’s attribution models because they just go with “last touch,” that if you do a full path analysis, I guarantee you’ll find you are under-investing in search.
No matter what company you’re with, you’re under-investing in searches with the way devices are going and with how social has changed to be all pay-to-play. Whatever your search budget is, just double it, because that’s where this stuff is going as the only way you can be found and not be spending large quantities of ad dollars.
Now, at your Next10x Conference, you mentioned that you would actually take Python courses and such. So what are you guys using in the AI realm?
Eric: Well, to be honest, in the beginning, it was just me trying to get my head around it. Being a geek, I have to go down into the detail before I can come back up and get my own sense of the bigger picture. So, I basically was just learning machine learning. I took the course from Andrew Ng, who’s Chief Scientist at Baidu. And then Geoffrey Hinton, who is directly involved in Google and machine learning out of the University of Toronto.
Where we’ve gone from there with it though is we’re really focusing a lot of energy on understanding how Google is using AI and machine learning. That’s really a big area for us because that actually puts us in a better situation to help our clients with it. And we have also done some dabbling in tools to improve content quality.
In particular, we have something that’s focused on processing user-generated content and automating that too, at this point, just reduce the need for human moderation by 80-90%. It’s a little hard to get to 100% with that.
Chris: Oh, yes, that’s true.
Eric: But if you can cut it down dramatically, then that’s actually a very high-value thing to do.
Chris: I’ll say. One other course that you should take a look at is Google’s crash course in machine learning, completely for free. It uses TensorFlow, as well as their hardware and their software. So if you wanted to literally get it from the horse’s mouth, it’s a completely free course. I’d encourage anyone to try it out.
END OF PART 7
If you’ve made your way down to this portion of the post, you have a definite interest in Machine Learning and AI. Watch this space for more content along these lines!
Christopher S. Penn is co-founder of Brain Trust Insights, a data analytics company focused on helping you make more money with your data, a co-founder of PodCamp with Chris Brogan, and co-host of the Marketing Over Coffee marketing podcast with John Wall. Learn more about him at his personal site: www.christopherspenn.com
Eric Enge is the founder of renowned, award-winning digital Stone Temple Consulting, and was its CEO until it was acquired by Perficient Digital, where Eric now serves as General Manager. He is the lead co-author of the bestselling The Art of SEO (now in its third edition from O’Reilly Media), and a sought-after keynote speaker, as well as a regular columnist for Search Engine Land. Eric’s groundbreaking studies have become industry standards, regularly cited in major publications.
Thank you for sharing a conversation about machine learning and AI.
Most of the startups are working on machine learning and using AI and This article is very helpful for them thanks for sharing this valuable information.
With all of Googles machine learning, I wonder if there would be a way to have a website that uses it, sort of like a split test, but far more complex, changing pieces of itself around until it finds just the right balance to attract people and keep them there.
Thanks for sharing about the machine learning and Ai which helpful for me.