Keyword research is the heart of internet marketing.
It’s almost always a keyword phrase – several words in a specific order – rather than individual words because it’s rarely possible for a single word to convey the intent of what you’re searching for.
There are occasional exceptions and they’re usually short lived. For instance, when Hurricane Katrina hit the headlines in 2005 you could search for that one word and relevant results would show up.
It still shows at the top of the results for a search for Katrina but there’s now Google’s equivalent of a disambiguation page: the top three results currently are for the hurricane itself, a Wikipedia disambiguation page and an actress called Katrina Kaif:
Part of the reason for that change in the results is that Google has a freshness factor in its algorithm which allows its computers to include certain results whilst that news is popular and then downplay them again once people lose interest.
But normally you’d need to add a qualifying word (so you’d search for hurricane katrina or whichever other hurricane you were interested in) to get more relevant results.
This applies to almost every keyword you’re likely to be targeting – you need several words to make up a keyword phrase.
It used to be the case that you needed to target an exact phrase in order to rank for it but Google now uses something called latent semantic indexing (LSI) to include synonyms in the search results. So when I searched for hurricane arthur (which I found on a Wikipedia page when I was researching this article) the top result was for Tropical Storm Arthur (2008).
That means that Google “knows” that a hurricane can also be called a tropical storm.
Interestingly, although that storm name goes back 8 years, it’s about to be re-used. And there was a news article that talked about storm names on the first page of the search results.
As a general rule, the shorter the keyword phrase, the more difficult it will be to rank high in the search results.
That’s because there are more pages competing for that search term – either deliberately or just because the word is included on the page.
So, in theory, this article is competing for the word hurricane.
But, in practice, it’s not. I’m just using it as an example and the page will get counted in with the 43 million other pages that include the word or the almost 11 million pages that include hurricane arthur either as a phrase or scattered somewhere on the page.
How Google interprets people’s searches
It’s important to have at least a bit of knowledge as to how Google interprets the searches people type in.
Whilst there are around 200 factors involved in ranking a web page, some are more important than others.
You’ll know from your own searches that the words you typed in are bolded in the search results. It often also bolds LSI words – lose weight and losing weight are usually bolded in a search for weight loss – so if your page is considered good enough, it can get included in the search results even if you’re not targeting the precise phrase that was searched for.
That’s important to remember when you’re targeting longer keyword phrases as far fewer people are optimising their pages for those phrases and Google is clutching at straws for even half way decent results to show on the first page.
In theory, putting quote marks around the phrase you’re searching for tells Google that you’re looking for that precise page with all those words in the order you’ve typed – that’s often quoted as being a way to find out how many other pages you’re competing with for any given phrase.
In practice, that’s not always true. It depends on the phrase as you can tell from this search:
The top two results don’t have the phrase I searched for anywhere on the page. Which means the figure of 556 results is wrong.
And if you click through to page 2 of the results, you get this message:
In order to show you the most relevant results, we have omitted some entries very similar to the 13 already displayed.
If you like, you can repeat the search with the omitted results included.
Clicking on the option to repeat the search still brings back 13 results.
Which is another important lesson in keyword research – just because a computer shows you a figure doesn’t mean that figure is even vaguely accurate.
By any stretch of the imagination, 13 is a lot less than 556.
And once this page is indexed, the total is likely to go up by one, even though I’m not targeting that phrase on this page, I’m just including it as an example.
As a general rule, weight loss is a competitive niche and difficult to get ranked for. I chose that phrase as an example because it came up in the suggestions as I typed and narrowed down my search – more about doing that soon – but it didn’t show up on the first page of suggestions in the Keyword Planner, which is probably why it’s not being targeted.
What this means is that keyword research isn’t a precise science.
But you can get enough clues from a variety of places to make sure that you’re going in the right direction.
That said, if you get your target keywords wrong, you’re at an immediate disadvantage in the search results.
Too broad and you’re competing with millions of pages, lots of which are from stronger sites than yours.
Plus you’re probably not targeting your real customers.
Too narrow and you can get to the top of the results quite fast but there are only a handful of searches a year.
Pre-research keyword research
You’ve probably heard the phrase “garbage in, garbage out”.
That is very true for keyword research.
Regardless of the keyword tool you use, if you give it the wrong instructions in the first place you won’t get meaningful results back.
If you know your niche well, you may be able to this on your own.
Using Wikipedia for keyword research
If you’ve decided on a broad topic but don’t know where you want to home in on, Wikipedia can be a good way to find topics.
For instance, I like targeting personal development and their main page for that subject includes a list of 16 possible areas I could target, almost all of which have links to more pages:
Browsing like this usually needs a timer, otherwise hours disappear and you’re no further ahead.
But it can work well if you’ve decided to enter a new niche or want to make sure that you’re going in the right direction in a niche you’re already in.
The sheer number of pages that Wikipedia has and the amount of interlinking it does between those pages can be useful or it can be a distraction.
Most of the time I find it useful but I also need to be disciplined – if I’m researching keywords, I’ll usually set an alarm on my phone so that I don’t spend all day when I’d originally planned to spend 30 minutes.
Brainstorming keyword research
Using your own mind is one of the best ways to do keyword research. Even if you’ve only got a passing knowledge of your chosen subject, chances are this method will work well.
You need some quiet space to do this – it works best if you’re not interrupted.
Get a piece of paper and a pen.
Step away from the screen and set a timer on your phone for 15 minutes.
Start writing potential keywords and keyword phrases.
Anything and everything that springs to mind.
Don’t edit at this point – just write.
For example, if you were looking for potential weight loss keywords you might write down:
weight loss
lose weight
shed pounds
drop a dress size
look thinner
losing weight safely
lose weight and still eat chocolate
And lots more.
Keep writing until your alarm goes off.
The idea is to come up with as many possible topics related to your keyword as possible.
If you get stuck part way, don’t worry. Just tell yourself to let the ideas flow.
Sometimes when I do this, I get a block at around 8 to 10 minutes but it soon passes and I get back into full flow again.
I’ve run the exercise with other people and even the most sceptical get excellent results. Don’t let the simplicity of the idea put you off or cause your mind to think that it won’t work.
You’ll get words and phrases that would never be suggested by a keyword tool because it doesn’t think the same way as a human does.
Most times when I do this I’ll get about two or three columns of potential phrases filling most of one side of a sheet of paper. Other people I know have gone on to more than one sheet.
Once you’ve got your list, set it to one side for an hour or two (or ideally overnight) and then come back to edit it.
The reason for doing this is that the creative part of your mind is completely separate from the analytical part of your mind and they mix about as well as oil and water.
Keeping up with the oil analogy, our minds are a bit like supertankers. They take time to slow down (which is why being too active just before you go to bed is a bad idea and keeps you awake at night).
Taking a break after you’ve been creative gives your mind time to adjust and gives you better results than trying to flick a switch between being creative and being more analytical.
Score each of your phrases. The scale is up to you – I tend to use a 1 to 10 scale or even just a yes/no reaction.
You could even use a tick or a cross if you prefer.
Don’t take forever doing this – your first reaction is almost always right.
So if a phrase stands out when you come back to it, that gets a high score or a tick.
If your reaction is “why on earth did I write that phrase” then it gets a low score or a cross.
It shouldn’t take you very long to go through and score your list – if it’s more than a few minutes, you’re not going with your first instincts and you need to pay more attention to those.
If several variations of a keyword phrase keep cropping up, pay attention to those first.
Those are the ones your subconscious mind has been drawn back to.
Amazon Books for Keyword Research
Go into the main books section for your chosen keyword – Amazon usually offers the option to search in books in the drop down suggestions:
Choosing books will help the next stage of your keyword research.
If you just want some quick suggestions then, much the same as Google, the suggestions that come up as you type work well.
Amazon’s suggestions are similar to Google in that they’re based on popularity (they’re not alphabetical). But they have the added advantage that by definition they’re based on things people want to buy – people don’t go to a site like Amazon to browse for information. And whilst there are some free books on Kindle, an Amazon search often ends up with a purchase.
Depending on your niche, the other selections may bring up books anyway.
You can also select books from the drop-down list to the left of the search box – I just find it quicker and easier to select it from selections that show up as I type and then it will change the selection to books for me.
Even when you’ve selected books, Amazon may still give you a mix of options:
I usually prefer to choose real (paperback) books but you can choose Kindle as well if you want to.
The reason for choosing printed books is that for the most part they’re better produced than Kindle books which seem to have a much greater variation in quality.
Amazon defaults to Kindle as that format now outsells printed books, so it makes sense from a commercial point of view.
Use the Look Inside feature to examine the chapter headings in the table of contents.
Sometimes it’s useful, sometimes it’s not.
I thought Steve Pavlina’s book Personal Development for Smart People sounded promising but the table of contents chapter titles didn’t help:
Then I looked inside a Tony Robbins book Awaken the Giant Within and got this table of contents:
Slightly better but not fantastic.
Or the regular standby, the Dummies series:
This process takes a bit longer than some of the other methods but can bring up some keyword phrases that you’d not thought about or that you’d forgotten about.
Understandably, copying and pasting from the look inside doesn’t work.
Instead, jot down the main keywords used in the chapter headings.
I find keeping a spreadsheet for these allows me to easily sort them if I need to at a later stage in the process.
Or you could use a simple sheet of paper and a five bar gate or similar tally system to give a quick visual cue about the popularity of each phrase in the books you’ve examined.
Once you’ve gone through a few books in your niche you should have a decent size list of keyword phrases to target and probably some other areas to include as well.
Other People’s Meta Tags
If you’ve got the time and inclination, this can be a way to get other keyword ideas. It’s not a method I personally use very often as I find it takes too long to get meaningful information.
Although the meta keywords tag is mostly ignored by search engines, it’s often still completed by webmasters and sites – YouTube puts the first few tags into the meta keywords field, other sites do similar things.
I’ve included it more for completeness than anything else.
It’s not something I do unless I’m really stuck for ideas as it’s boring and I’ve not found a tool that I’m happy with to extract them.
But if you’re really stuck, it’s worth doing on the first few results for your keyword search. Just be aware that not every site uses meta keywords.
If you’re doing this by hand, choose the View Source option on your browser and then use the find function (Ctrl-F on Windows machines, Command-F on Macs) to search for meta name=”keywords”.
Then look along the line for content= and you’ll then get the list of meta keywords the site considers relevant for the page.
Be aware that historically the meta keywords tag has been used as an attempt to spam search engine results so not every site uses them ethically.
Some large sites – Wikipedia springs to mind – don’t use them at all.
Other large sites such as Amazon will make up their own keywords depending on how you reached the search result.
For instance, when I clicked the table of contents image above for the Tony Robbins book, I got this:
<meta name=”keywords” content=”Anthony Robbins,Awaken the Giant Within : How to Take Immediate Control of Your Mental, Emotional, Physical and Financial Destiny!,Free Press,0671791540,Psychological aspects,Success,Psychology,Self-Help,BUSINESS & ECONOMICS / Entrepreneurship,Entrepreneurship,Business & Economics / Motivational,Motivational,General,Psychology / General,Self-Help / Personal Growth / Success,Self-Help / Personal Growth / General,Behavioural theory (Behaviourism),Careers guidance,Management & management techniques,Self-help & personal development,Advice on careers & achieving success,Personal Growth – Success,Popular Psychology,Motivation,Personal Guidance” />
Google kept giving me the audio book, even when I specified paperback, so the keywords I found aren’t comparable.
And when I checked it from the phrase I typed in, I only got this:
<meta name=”keywords” content=”personal development” />
Which means Amazon are serving up different meta keywords according to how you reached a particular page. Which in turn means that someone, somewhere, in their programming team still thinks that meta keywords count for something, but that’s a different topic entirely.
Ebay seems to use the listing title and their own internal categories for their keyword tag as in the audio version of the same book I found here:
<meta name=”keywords” content=”NEW 2 CD Awaken the Giant Within Anthony Tony Robbins nlp TAKE CONTROL YOUR LIFE, Books, Audiobooks” >
Forum software often makes up its own mind regarding keywords, based on the title of the post, any tags entered by people posting on the thread (most won’t add tags even if they have the option) and various other factors.
If I needed to use the meta keywords tag for keyword research, I’d choose a site that was built by a hobbyist as I think they’d be the most likely to use the tag as it was originally intended.
Google Suggest
For almost any search, Google will make suggestions as you type.
If you’ve got instant results switched on, you’ll usually get 4 suggestions.
If you’ve turned off instant results you’ll get up to 10 suggestions.
To turn off instant results, you’ll first need to have made a search.
Then a cog will appear near the top right of the screen and clicking on the cog will give you these options:
Choose the Search Settings option.
Then click the option that turns off Instant Predictions:
Once you’ve done that, most of the time Google will give you 10 options and it still seems to modify the suggestions every time you type in your search query.
Of course, like anything to do with search results, some people try to influence the suggestions that come up as you type. Which I guess explains some of the weirder tasks on sites like Microworkers.
The suggestions are quite up to date – they are based on recent searches across the globe – and they are in descending order of popularity – they are not alphabetical.
So the suggestions near the top of the list are the most popular for the keywords you’re typing:
That makes Google’s suggestions very powerful.
Most people are lazy and will either click one of the suggestions or keep typing until they’ve typed in the phrase, which can be quicker depending on how deft you are with a mouse.
Either way, those phrases are being searched for, mostly by real people looking for results that match their phrase.
I’ve picked up keyword phrases from it on a number of occasions that don’t show up in the keyword planner. maybe because the search volume isn’t high enough to make them attractive to advertisers, maybe because the phrase is too new to be there.
You can get longer tail suggestions in a number of ways:
Move your mouse cursor to the start of the phrase and put a space before the current keyword phrase
Move your mouse cursor between any two words in the list – you’ll often get extra suggestions
Type a space at the end of the phrase and then a letter (or more than one letter if you’re feeling keen) to see suggestions that include that start with that letter after your phrase
This method is quick, easy and intuitive.
It has an extra bonus if you click the search results – you’ll get an “at a glance” look at the first page of the results and will be able to tell how competitive the phrase is likely to be if you decide to create some content around it.
Once you get into phrases that have quite a few words (usually four or more but it depends on the niche) you’ll find less adverts.
In itself, that’s a relief from the ever present ads in the search results.
But it also means that you’re not competing with the paid-for ads before the organic results finally show up. So your position on the search results page really is that, not 3 places further down (plus any shopping or other sponsored results) as so often happens with more competitive phrases.
It doesn’t mean that the phrase doesn’t have buyers – I’ve often found keyword phrases with commercial intent with this approach – it just means that advertisers don’t know these phrases exist and aren’t allowing Google’s broad match to splurge their advertising budget wherever it thinks it can get away with that.
Most of the time, Google suggest is the only keyword tool I use.
Related searches
Related searches are quite closely related to the late lamented Wonder Wheel but without the need to make any effort to go to a separate section of Google.
Scroll to the end of the first page of results for your chosen keyword and you’ll often see a “Searches related to” list:
Google assumes (probably quite rightly) that if you haven’t found what you’re looking for by the end of the first page of the results then you need some help refining your search.
They even put the suggestions above the “next page” option.
The related searches are usually longer tail versions of your initial search but they sometimes include related terms. For instance, I just searched for “long tail keyword research” and one of the related terms shown was “long tail seo”.
This happens fairly regularly and can take you off in a new direction on your keyword research and works in a very similar way to the Wonder Wheel that Google used to provide. So if you come across any older keyword research books that mention the Wonder Wheel, you know where to get something similar.
I clicked the “long tail seo” link and one of the related searches was “organic seo company” – quite a long way from the initial search but definitely related and a nice way to find those terms. That search then led to “organic internet marketing” as a related term.
This can be really useful when your mind goes blank or you’ve been researching a niche for so long that you think you’ve run out of alternative keywords.
Another way to come up with potentially related terms is to use Amazon’s “Customers Who Bought This Item Also Bought” links – they can help you get into the mind of your potential audience.
Google Keyword Planner
You need to remember that the Keyword Planner tool is designed to help advertisers find more keywords to bid on.
That means it is unlikely to find really long tail keywords as they wouldn’t have sufficient traffic to make it worthwhile running an advertising campaign.
You also need to be aware that the search volumes are estimates and that those estimates are then put into bands.
It’s designed to look precise but that’s not the case:
There are two phrases on that screen grab with an estimated 480 monthly searches, another two with an estimate of 170.
The advertiser competition is also guesswork.
For instance, long tail keyword research says it’s medium. I did two searches for that phrase, one in my regular Firefox browser and another in Google Chrome but in an incognito window and got these results: 7 adverts in incognito Chrome, none in Firefox:
Some of this is because storing that kind of data so that you could interrogate it on demand would be incredibly heavy on storage, partly because even if the search data was correct it’s historic. And with regard to the advertising competition, your guess is as good as mine.
So ignore the advertising competition unless you’re buying ads!
The historic part of the figures is important to remember.
If you’re looking for search data in the summer about the toy that you’re certain will be a best seller this Christmas, the data doesn’t exist yet.
Google hasn’t got a time machine, so it can’t know what people will be searching for months or even days from now.
You need to remember that searches are influenced by all sorts of factors and take that into account when you use the historic results in the keyword planner.
The next variable is the number of people who click on the results. A study in 2013 found that the top organic link got 33% of the clicks from a search. The further you are from the top of the page, the less clicks you’ll get. And if you’re not on the first page, forget it to all intents and purposes.
Another recent study found that 40% of consumers are unaware the top results are usually adverts. There’s been a change in the design of the results since that study – adverts are currently shown with an orange “ads” text next to them – but it’s a fairly safe bet that Google will have tested how to stay the right side of the authorities but also protect its income from adverts so that figure is unlikely to have dropped much, if anything.
As well as typing in a keyword phrase, you can also put in a URL (your own or someone else’s) to come up with potential keywords to target.
Wordtracker
Wordtracker is a paid alternative to Google’s keyword tool.
It draws its information from various sources – these used to be Metacrawler and Dogpile but their site is vague about where they are currently getting their data from.
One thing is certain though – they’re not getting their data from Google. And as far as I can work out they’re not getting data from Bing either. Which means it’s based on a small sub-set of searches and I’m not personally convinced about how reliable it is.
Other keyword tools
A lot of keyword tools just put a fancy wrapper around the Google Keyword Planner.
This can be useful if you want to grab lots of data and then store or manipulate it.
But it does also mean that they are at the mercy of Google. This is especially so if they’re just scraping the results as every time Google changes its format their program breaks until their programmers catch up with the changes.
Google offer an API to allow software developers to integrate their programs with the Google results – if the keyword tool you’re thinking about using uses Google’s data it would be well worth checking that the program uses this option to get their data efficiently.
Spyfu scrapes data from web pages to come up with its keyword suggestions and according to Wikipedia the accuracy of its data varies between websites but if can be an extra source of information if you’re happy to pay for a subscription.
Webmaster Tools
If you’re using Google Analytics then you can get access to the search queries used to find your site.
This can be useful as Google and other search engines do not provide this data for most searches, so you can’t get the information from your site log files.
You can then use the information you glean to research related terms that you could create content for.
What to do with your keyword research
It can be very easy to get bogged down in keyword research.
You need to actually use the keywords you gather rather than leave them on a spreadsheet on your hard drive, never to be looked at again.
If you’ve got a team of writers then you can delegate the task of creating content to them.
If you’re the “team” then you will need to create the content yourself.
I create most of the content for my own sites and my personal approach over the years has moved from creating a long list of potential keywords – that I never actually did anything with – to finding one keyword phrase that appeals and looks like it could generate traffic. I then create some content for that, post it, maybe turn it into a video and audio (in which case I post those as well) and then do some promotion of the page. At a minimum that means I Tweet about it or similar.
You need to work out what works best for you – whether it’s a long list that you work on over a few days or weeks or whether you pick one phrase to work on at a time to keep your focus.
The post How to do Keyword Research appeared first on Trevor Dumbleton.