Smart SEOs and content marketers know that search engines have evolved beyond optimizing for keywords to optimizing for topics.
We’re told to take our list of keywords, identify core topics for pillar pages, build related content associated with the core topic in the form of spoke pages, and then internally link them together.
One of the major hurdles we’ve encountered is how to cluster keywords together accurately at scale.
Tools are starting to crop with the ability to do some form of keyword clustering by fetching and processing SERP data. Some of my favorite include:
This is great.
But it’s not enough.
None of the available tools answer: What should I do first? What is the best way to plan all of this out?
How to answers those questions (and more) are why we developed our own internal tool and are what we’ll provide a framework for in this article.
Where Topic Clusters Fit in Content Strategy
Planning a content editorial calendar from a huge list of unorganized keywords is a mess. Grouping them into clusters makes things easier. It lets you segment based on relevance, search intent, the customer’s journey, etc.
Grouping them manually is tedious and prone to errors. It can take months depending on the size of the data set.
While there are several tools available that help with this process, we had to develop our own because none of the public options were actionable enough.
We needed more than a set of clustered keywords. We needed direction. We needed to know which topics to create first, how to optimally link them together, and how to build authority in a brand new topic by leveraging the topical authority we already had.
Keyword Clustering Alone Isn’t Good Enough
Topic clustering, keyword clustering, keyword grouping, (or whatever you want to call it) is not a silver bullet. It’s a single step in the keyword research process on our quest to find the best content topics for organic search.
The best topics for organic search are those that are the easiest to make, rank, distribute, and promote with the highest conversion value to traffic potential ratio
Let’s say you’ve used a tool to help cluster your keywords together. You now have a good idea of what topics to cover in each piece.
What do you with it? How do you build a strategy from here?
You could use them to throw together an editorial calendar for the quarter, but that’s not a strategy. That’s just guessing.
This approach is only a slight improvement from what you were doing before. You went from relying on your gut to group topics together to relying on your gut to prioritize them.
Note: This assumes the keyword data you fed a program to cluster your topics is comprehensive. If it’s not, your clusters will miss out on opportunities and intents.
Most topic clustering tools have input limits on how many keywords you can run at once, ranging from 1000 to 25,000. When we build our clusters, we don’t want limits. We wanted a way to process any number of keywords we threw at it—whether that was 250,000 or 2.5 million.
So how do we build a strategy from this?
Everything is a Subset of Business Strategy
You need to think about how to map your clusters to align with the overarching strategy of the business.
If you align your clusters with the big-picture, you can plan a content calendar that focuses on value.
There’s no perfect way to determine how valuable a topic will be. But we can minimize our opportunity cost with more data.
Note: How a business determines value will vary. It’s not always related to organic search. The focus may be on things like customer retention or competitive positioning. In the context of keyword clusters though, the focus is on value from organic traffic.
For us, the best approach we’ve found includes accounting for things like:
- Search intent
- Traffic potential
- Topic difficulty
- Content difficulty
Combining this data, we’re able to derive what we call an opportunity score.
Is it perfect? No. But it is better than topic clusters alone. With this data, we can supply our Monte Carlo simulation with the inputs it needs to project the ROI on our content marketing efforts.
Prioritizing Content Topics with Data
There are no universal rules to calculate and weigh each of the following variables. We aggregate the data to build our opportunity score. Then we treat it as a science. We experiment, test, observe, and iterate.
You’re encouraged to do so as well.
Try to get them as dialed in as you can but forget about perfection. Directional accuracy is more important — and more pragmatic.
Remember, the goal is to take your clusters and organize them to focus on the best topics: those that are the easiest to make, rank, distribute, and promote with the highest conversion value to traffic potential ratio.
Search Intent & The Funnel
Determine search intent to know which topic clusters make sense for your business. Sensible topics are those that align search intent with business goals.
If you get search intent wrong, you’re going to have a difficult time ranking for that topic.
You can always revisit a page that misses intent and update it, but getting it right the first time is better.
At a high-level, you can segment search intent into four primary categories:
Once you’ve set the intent for each topic cluster, you can then map each of these to your marketing funnel.
In doing so, you’ll be able to add your average conversion rate by funnel stage. If you don’t know what this is, you can set it at 2% to establish a baseline.
Mapping your clusters to the funnel also makes it easier to plan other marketing tactics to pair with each piece of content later.
This can be a time-consuming process but it’s too important to ignore. Other tools couldn’t handle doing this at scale so we developed a way to do it in our own algorithm.
Tips for the Tactician: You’ll have to do this manually if your tool doesn’t calculate search intent for you.
You can speed things up using query statements in things like Google Sheets. If you’re working with a large data set, you can use a connected sheet with BigQuery.
Traffic Potential Trumps Search Volume
Sorting your keyword clusters by volume is a quick way to determine priority.
It’s also misleading.
Search volume is not indicative of business value. It’s better to calculate the relative traffic potential for the keyword cluster.
Start with search volume, then factor in ClickStream data for things like:
- Clicks per search (CPS)
- Click distribution
- Return rate (RR) is an interesting one too
Tools like Ahrefs and Semrush provide a CPS value that can export and map to your cluster data.
Note: Semrush calls this Click Potential. Their formula assumes you’re ranking in the first organic position. Ahrefs calls this CPS and does not have it for every keyword.
We take the estimated search volume for the cluster and and multiply it by a set of CTR values we have in a lookup table.
You’ll want to account for the range of potential ranking positions, intent, the industry you’re in, and the presence of SERP features when calculating this. These all impact the click-through rate organic results receive.
- Searches with commercial intent get more clicks than those with informational intent
- Branded searches beat non-branded
- Long-tail CTR is higher than short-tail
- Featured snippets + People also ask boxes reduce CTR
For Content Refreshes: Prevailing wisdom says you should look for keywords ranking on the second page and work to get them to the first page. That’s not always optimal. If you’re ranking on the first page for a target keyword, your primary focus should be getting it to the top.
The CTR delta between ranks 12 and 8 is barely over a 10th of a percentage point (0.11%+). Contrast that with going from rank 4 to rank 3 (doubling CTR from 4.21% to 8.13%) or from rank 4 to the first position (7x increase to a 30% CTR).
For a single keyword with an estimated 1000 monthly search volume that’s the difference between 42 clicks (ranking fourth) and 301 (ranking first). This is assuming every search performed gets at least one click — which isn’t the case.
If your method of keyword clustering does not pull in SERP feature data, you can get it from most popular tools (e.g. Semrush, Ahrefs). Be aware that Ahrefs does not provide display whether or not a local pack gets triggered.
Tips for the Tactician: You can build a set of standard CTR values for different scenarios using up-to-date data from AdvancedWebRanking to create a lookup table that takes position, features, and industry into account.
Topic difficulty isn’t the same as keyword difficulty.
With topic difficult you’re evaluating the SERP from a brand perspective.
You can weight topic difficulty by comparing yourself to other brands on the first page. We don’t base this off of DR or anything like that. We research and find the big brands that have name recognition (called icons internally). Then we add them as a parameter to our algorithm to account for them as it runs.
That doesn’t mean smaller brands can’t tackle a topic where big brands dominate the SERPs. It’s something to consider when calculating an opportunity score though. If we see a few smaller brands ranking on the first page, that makes that topic more viable.
Tips for the Tactician: We calculate our score for each SERP based on the count of big brands and their ranking position. We then take the average score across all keyword variants in the cluster weighted by the traffic potential for that keyword.
To find big brands, note domains appear a lot across your topics then get the search volume for their brand name. Those with the highest branded search volume are the ones to be aware of when calculating relative topic difficulty. Don’t ignore the fact that some brand names may have search intent that isn’t always branded either.
Creating good content is hard.
Some content is more difficult to create than others — and not just as a function of word count.
We calculate content difficulty by considering the actual resources and effort it will take to make the right type of content that aligns with the searcher’s intent.
This isn’t always written content.
Figure out if you’ll need video, custom imagery/diagrams, tools, calculators, tables, or other types of media.
Difficulty for each content type will vary based on the topic and the business it’s for. You can use a relative metric like 0-10 to weight content difficulty. Consider available resources for content marketing projects to add weight to that scale.
More difficult content takes longer to put together, but has a higher ROI when you get it right. It’s difficult for the competition to copy and also tends to attract links/shares.
Take this example by Wistia that provides free expert training to teach you what you need to know to create a compelling video series or podcast for your brand. Expensive to build this content to put together but will drive a ton of leads.
Other Important Factors: Shareability & Linkability
Don’t neglect to consider how you’re going to distribute and promote your content before you make it.
There will be topics with poor scores in all other metrics that are perfect for engagement on channels other than search.
Distribution & Shareability
Note: Brands lacking in topical authority and name recognition should weigh this score higher. A few good pieces that get traction can make a significant difference to your bottom line.
Do we want evergreen content that can rake in passive traffic and revenue over time?
Smaller brands have a lot to gain by building trust with an audience first though. The impact this will have on your total organic search performance is indirect but important.
Advice on Scoring: Don’t rely on aggregate metrics from tools like Ahrefs’ Content Explorer, Buzzsumo, etc. to find what type of content gets shared. These tools favor popular sites and don’t reveal the delta between normal and outstanding performance.
Instead, use that data to compare how it does to the average for that site during the past year divided into quartiles.
Promotion & Linkability
Here’s an a priori truth:
The best way to get links is to create content that people like linking to.
When determining priority, ask yourself, “How difficult will it be to get people to link to this piece?”
Again, weighting this score is relative to your business. A company with an in-house link building or PR team will be able to handle this better than one that doesn’t.
Note: Don’t underestimate the difference a highly linkable asset can add to your evergreen content either. A single highly-linked post can lift the rankings of your entire site.
Starting with this sort of piece can be a gamble but it pays off big — especially if it can generate links passively over time.
Well-known brands with a ton of brand or topic authority can weight this factor lower. They often receive links on autopilot from their already established audience.
Things can be more difficult to calculate for small brands.
For each cluster, we take into account things like:
- How many links are needed
- From what types of sites
- From what types of content
- At what velocity
- The true cost per link
We get this data by reverse-engineering top competitors ranking for our target topics.
This process is very labor-intensive so we built a program for that too.
Further Reading: How we conduct a competitive backlink analysis.
After Action Report: Strategy ≠ Execution
Topic clusters by themselves aren’t very useful. By combining data to understand the intent, traffic potential, difficulty, complexity, shareability, and linkability — you can make confident decisions when planning where to focus your efforts.
Having a plan is great on paper but useless if not implemented. Knowing what to do and how to prioritize everything is not enough for success either. You can use our recommendations as a framework to create a revenue-first editorial calendar.
From there, your content team can carry on with business as usual.
If you don’t have editors and writers already, you can either hire a content team to do it for you or begin building one.