The demise of the keyword and the future of SEO

The last few years have been a tumultuous time in search. Google, the dominant force in web search (accounting for nearly 90% of all web searches in the UK), have been making earth-shaking changes to the way they rank and display search results.

Some have downplayed the significance of these changes, some have declared SEO a dead discipline. All have recognised that what we expect from search engines (both from a user and marketing perspective) has changed radically.

Which leaves one nagging question: what does this mean for your online strategy in 2014 and beyond?

When keyword was king

Until fairly recently the way that the major search engines ranked and displayed search results for a given search query was based on two major factors: the content of the page in question and the links pointing to that page from elsewhere on the web.

The way the search engines decided how relevant the content of a page was to a search query was based almost entirely on text. If text matching the search query turned up in the title of a page, the headings, body copy etc it was a pretty good bet that the page would be relevant to the query.

The way the search engines measured how authoritative a page was in relation to that query was based on the number of ‘citations’ that page had. In other words, how many links from elsewhere on the web pointed to that page.

(That’s a grossly oversimplified way of describing how search engines work but bear with me.)

Obviously this set up was pretty easy to manipulate by early search marketing types. Stuff a page full of the keywords you wanted it to rank for, build a bunch of links that pointed to it and hey presto: your page would rank.

This quickly started harming the quality and relevance of the pages that would show up when you searched for e.g. Auto Insurance Comparison (one of the most valuable keywords in 2009). Spammy pages with terrible user experience and awful content became the norm on vast swathes of search engine results pages (SERPs).

Google leads the fight back

In response to the spammers Google mobilised its webspam team. It’s their job to engineer changes to the search algorithm which improve the quality and relevance of its SERPs. Most of the initial changes were reactionary, designed to penalise sites which displayed behaviour that Google deemed spammy.

The updates that SEOs have been flustered about for the past couple of years (Panda, Penguin, Exact Match Domain) have been updates of this type. But while these reactionary, spam-fighting algo updates were grabbing the headlines, Google was quietly revolutionising its search engine. Under the skin, the search giant has been quietly moving away from keywords towards a more semantic search model.

What’s semantic search?

Semantic search isn’t a new idea. In the scientific and mathematical fields they’ve been around for some time. A semantic search engine, rather than relying on matching words in a search query to words in documents (e.g. web pages) instead tries to interpret the intention and context of what a searcher is asking for to return results that are more appropriate.

You may have noticed a recent feature of Google’s called Knowledge Graph. This information box pops up to the right of your search results whenever Google thinks it’s recognised an ‘entity’ that you’re looking for information on. It displays info on this entity from a number of sources and often has links to other related entities.

This means Google is no longer just using the words you put into the search bar to turn up results but is attempting to recognise the ‘things’ that those words refer to.

Bill Slawski recently wrote an excellent piece on how Google might be building its knowledge graph of entities and the relations between them. It’s based on a paper Google published in October 2013 and is full of insight.

From search engine to answer engine

Google’s attempt to build a social network to rival Facebook and Twitter has, on the face of it, been a spectacular failure. But the benefits to Google of getting everyone using Plus go far beyond owning a social network on which they can sell ads. One of them is that Google gets to gather a tremendous amount of data on the people who do sign up and remain signed in.

The more Google knows about you – your age, location, occupation, professional and social connections, tastes, preferences, hobbies and search history – the better able it is to work out what you’re searching for, and the more able it is to return appropriate search results. When you’re logged into your Google account the cache of personal data that Google holds for you grows and grows.

What this means is that Google is rapidly increasing its ability to answer more of your search queries on the results page itself, rather than sending you to an external link. This started slowly with widgets giving you the answers to unambiguous queries like “cm to in” or “weather tomorrow” or “how do I get to Cardiff”.

These only utilise data like your location but over time it’s easy to see how Google will pull in more and more personal data – and utilise the connections you’ve made on Google Plus – to provide search results that it thinks will be more suitable for you. (Results that have been +1ed by your connections are already favoured.)

The future is Now

Google Now is probably one of the most important indicators of how this trend is set to develop. Little known by its product name outside of Google, Now is part of Google on your mobile phone, delivering pertinent information in cards underneath the search bar.

Now is designed to answer your questions before you’ve even answered them, using the incredibly personalised data you generate as you use your mobile phone.

Driving directions and traffic data for your journey home, news based on your previous reading habits and reminders of dinner reservations are already part of it. Soon they might be augmented with reminders to record your favourite TV shows, alerts that tell you when friends are nearby and, once connected to the internet of things, perhaps even recipes based on what you have in the fridge.

What does this all mean for SEO?

The upshot of all this is that Google no longer has to rely so heavily on the content of your website and what other websites say about it (via links) to work out what’s there. Rather it can use its own map of the world built up from user behaviour and the emergent associations.

In addition, Google is holding back more and more of the data we need to use the old tactics of optimising pages for particular keyword phrases. Search marketers have been seeing the proportion of keywords displayed in Analytics as (not provided) growing larger and larger.

SEO specialists are thus finding themselves having to think much more strategically than tactically. The challenge becomes associating your brand with a particular subset of searcher intent rather than your webpages with a particular keyword; providing evidence that users are finding what they need at your site rather than building inbound links; providing answers to searchers questions rather than echoing the questions themselves.

How to become a provider of answers

Improving user experience becomes a big part of the equation. Lowering bounce rates, improving time spent on site and raising click through rates all send positive signals to Google that your site is delivering the goods and deserves to be ranked higher. But all those things require that you provide genuine answers to search queries in a way that doesn’t annoy users.

Providing answers (content) and providing them in a timely (site speed), easy to navigate (UX) and easy to digest (readability) manner are becoming the new cornerstones of SEO.

Is it making the web a better place? Maybe. We certainly see a lot less spam in our search results these days. (I do have major qualms about the near monopoly that Google has over web search though – can it be good for a single company to have so much influence over the development of the web?)

Is it making SEO a more difficult job? Absolutely. The days of extensive automation of SEO tasks are over. Today’s search professionals need to be creative, strategic, multi-disciplinary thinkers with almost infinite adaptability.