Keyword Research Tools – Build Your Own Adventure

Posted by Sam Crocker

Hi there Mozzers! My name is Sam Crocker and I work for Distilled. This is my first post here at SEOmoz and I am looking forward to your feedback!

Background

My mother used to scold me for misusing my toys, playing with my food and for having a bit too much energy. She was well within her rights, as I was a bit of a handful, but at the moment one particular phrase really sticks out in my mind

“Is that what that was made for Sam? Use it the right way, please.”

Whether I was riding down the stairs in a sleeping bag, having sword fights with paper towel tubes with my sister, or using my skateboard as a street luge- I’ve always been big on using things for purposes other than their intended design. It should be no surprise that I do the same with some of the fancy and powerful tools upon which we have become quite dependent in the SEO world. Much like when I was little, it seems like by using things the “wrong way” there’s scope to have a bit more fun and to discover some new and different ways of accomplishing the same goals.

Me As a Little Guy. Snow Scraper = Renegade Fighting Stick?

I spoke about my most recent adventures in using things the wrong way at SMX Advanced London. I don’t think too many people who came to the keyphrase research session expecting to hear about how a scraper like Mozenda could be used to save all sorts of time and effort and generate new keyphrase ideas. You may want to have a quick read through that before watching the screencast.*

It’s also important to point out that Mozenda is best used as a discover tool in the instance I provide here. If this method were a perfect solution to keyword research you could very easily build a tool that does it better. The beauty of Mozenda, however, is that it can be just about any tool you want. If you need to generate brand new content around a subject area you know nothing about, you can use it to explore tags on delicious or another social media platform.

Given a great deal of interest in this technique that I received from attendees at the presentation and in the twittersphere I decided it was worth providing a full walkthrough to cover some of the nuances I wasn’t able to cover in a 12 minute presentation and to share with the folks who weren’t able to attend the conference.

 

 *It’s worth noting that for the sake of consistency I used the same Google Suggest tool in the video as I used for my initial research and discussed at SMX London. Since then Rob Milard built his own keyphrase expander tool based on this work and it is considerably more versatile than the original tool (you can search Google.com or Google.co.uk and export the file as a CSV). The output of this version isn’t in XML and provides the “search volume” data missing from the first tool. So congratulations and a BIG thank you to Rob from me and the search community in general!

Next Steps

The above screencast is an introduction of a technique we have been experimenting with to broaden the keyphrases targeted on a site (particularly, it can be used to increase the number of longtail keyphrases and provide insights into terminology you may not be targeting in your current list of keyphrases). This can be particularly useful if you work for an agency dealing with clients from a number of different sectors. For the sake of demonstration I have only input 7 terms into the Google Suggest tool in an effort to pull out a workable dataset for the screencast and for my presentation but Mozenda is a pretty powerful tool, so there’s really nothing stopping you from using more keyphrases. As a matter of courtesy, however, I would suggest setting up some delays when running any large scraping task to prevent overwhelming servers or hogging bandwidth. For more information on this, please have a read through Rich Baxter’s latest piece on indexation.

One of the questions I was asked (by a number of people) was “what next?” As in: “what on earth am I going to do with these extra 10,000 keyphrases?” And although this presentation was intended as a proof of concept, I don’t want anyone to think we are trying to keep anything secret here so here are a few ideas about what you might consider doing next.

Option 1: Ask For Help!

For the people who find themselves thinking “I’m not really sure what to do with this data” I would suggest enlisting the help of a numbers guy or gal (Excel Wizards or other nerdy warriors). Odds are if you find looking at this sort of data daunting, you’re going to need their help making sense of the numbers later anyways.

Option 2: Outsource

The second option, for those of who know exactly what you want to do with this data, but don’t have the time to go through it all, I strongly suggest enlisting the help of cheap labour. Either find yourself an intern or make use of Amazon’s Mechanical Turks to find someone who can accomplish just what you need. The nice thing about services like this is that it’s a 24/7 workforce and you can get a feel for how helpful someone will be fairly quickly and painlessly.

Option 3: Jump Right In

Finally, the third option for those of you with some Excel skillz and a bit of time. There will definitely still be some manual work to be done and some weeding through for terms that are not at all relevant, the suggestions where you usually say aloud “no, Google I did NOT mean…” will clearly need to go.

The best use of this data will be the general themes or "common words" that you can quite easily sort through or filter for using Excel and that you may have been to oblivious to prior to starting.

 Feel Free to Sing Along if You Know The Words! (image via: Kottke)

Step 1: Remove all duplicates. In this example there were no duplicates created though I can only assume that with 10,000 keyphrases run through the tool there will be some duplicate output.

Step 2: Remove URL suggestions. I know we like to think otherwise, but if the user was searching for “gleeepisodes.net” they probably aren’t interested in TV listings from your site. It would also be a fairly cheeky move to try to optimise a page about someone else’s website.

Step 3: Remember your target audience. If you only operate in the UK “Glee schedule Canada” and “Glee schedule Fox” can probably be eliminated as well. Now would be a good time to eliminate any truly irrelevant entries as well (e.g. “Gleevec” – although some of your viewers may have leukemia this probably is not what most visitors to your site are looking for).

Step 4: With the remaining terms and phrases run them through the usual sense checking routines. This is a good time to check global/local search volume for these terms and look at some of the competitiveness metrics as well. Search volume will probably be quite high for most of these terms (at least enough for Google to think someone might be looking for them regularly), though competitiveness probably will be too, so choose wisely.

Identifying the patterns at this stage will be essential to the value of the research you are conducting. You can try to filter for common phrases or suggestions at this stage and if, as in this example you realise "rumors" is a relevant term you’ve not targeted anywhere on the site, it is high time you consider adding content targeting this area for all of the television shows on the site.

Last Step: Come up with a sensible strategy to attack all this new content. Look at these terms as jumping off points for new content, new blog posts, and new ways of talking about this and other related products/services/subjects on the site.

Conclusions

A lot can be learned through this sort of exercise. In addition to finding some new high volume search terms, it may help you identify trends in search for which you have not been competing and have implications across the whole site rather than on one page. For example, maybe you didn’t think about “spoilers” or “rumors.” For a site dedicated to television programmes this sort of terminology will likely be valuable for a number of other shows as well!

The moral of the story? If you build it they will come.

Sometimes it is worth developing your own tool to make use of existing technology. Whilst I still feel Mozenda is the right tool for the job for handling larger datasets, the tool Rob built is a perfect example of both how a little creativity and building on other’s ideas can lead to benefit for everyone. Rob’s tool effectively rendered my Mozenda workaround unnecessary for most small to medium sites, and that’s awesome.

Image via: Motivated Photos

A final word of warning: I’m not suggesting that you replace all other keyphrase research with this idea. This technique is best utilised either during creation of a site about an area you know very little about (it’s rare, but it happens), or when you’ve run out of ideas and tried some of the more conventional approaches. It’s all about thinking outside of the box and trying new things to save you time. Onpage optimisation, linkbuilding and more traditional keyphrase research needs to be done but sometimes the best results come from trying something a bit experimental and using things for purposes other than that which they were designed.

If you have any questions, comments or concerns feel free to shame me publicly either in the below section or on Twitter.

Do you like this post? Yes No