Update #1: Ahrefs official response (Sept 10th 11:42am CST)
Update #2: SEMRush data investigation (Sept 11th 08:12am CST – developing situation)
Update #3: Moz calls SEMRush out (Sept 11th 09:36am CST)
[CRITICAL] Update #4: SEMRush To Provide Updated Data… (Sept 12th 09:19am CST)
To find out which is the best backlink checker, I devised a real test.
I decided to:
- Take the 5x best backlink checkers on the market
- Use them to collect data for the same 1x million links
The result? Well…
Find out who below…
What Will I Learn?
The Competitors – The Old Guard Face A New Test
This time round they were up against some fierce competition.
And, as you read in the intro there is a NEW king in town. But, who took the throne?
Read on to find out…
SEMRush has always played second fiddle to Ahrefs.
Because they’ve always aimed to be a complete digital marketing tool, some of their specific SEO tools – like their backlink checker – have just never been as good.
They invested a lot of resources and added some exceptional new features. But:
Moz Pro is the original backlink checker. Their Moz Link Explorer (previously Open Site Explorer) was one of the first backlink checkers available.
So, they’ve been at this much longer than anyone else.
Has age and experiencd helped them make a real challenge for the title?
Or, are they a washed-up prizefighter who needs to hang up their gloves?
Majestic is a seasoned veteran that makes some big claims about the size of their link database (even though we have proven it not to be true many times before)
Can their tool match their sales pitch? History says no.
SEO Spyglass started life as an ultra-slow desktop SEO Tool. Never much of contender.
But now they are back with a RAPID upgrade and new database. Is it enough?
Let’s take a look…
Before I reveal the winner I want to talk you through this test’s methodology.
The domains for this test were pulled from the Majestic Million.
These are the top 1x million websites based on their number of referring IPs and Subnets according to Majestic.
- Each of these sites should have a TON of data
- Each of these sites should have consistent numbers across all 5x tools
The 1x million domains were run through each the 5x tools. I then looked at which tool showed:
- The most referring IPs
- The most referring Subnets
That gave me 5x million rows of data – Yay! – to work with.
You can download all of that data right here.
The next 3x graphics show the results of each test. The winner is underlined in pink.
Total Wins By IP – SEMRush
Total Wins By Subnet – SEMRush
Whilst you are digesting that, it’s important to note that the graphs show the number of “wins” and not absolute numbers.
So, they do not fully represent database size. Let me explain…
Let’s say all 5x tools look at the same site. They each report:
- Ahrefs: 100 referring IPs
- SEMrush: 101 referring IPs
- Moz Pro: 99 referring IPs
- Majestic: 100 referring IPs
- SEO Spyglass: 100 referring IPs
So the graph only represents the number of times each tool beat its opponents.
Which shows us that it is rare for other backlink checkers to have more data than SEMRush does.
Another simple way to look at the size of the database is to look at the average number of:
That were found across the dataset-
Average Number Of IPs & Subnets Found Per Domain
But what does that mean?
It means that on average SEMRush finds more IP’s per domain than any other tool.
But it also means that SEO Spyglass finds more subnets than any other tool.
That is super important to point out because-
SEO Spyglass is a $124.75 one time fee for the first year. Then it’s only $4.17/month after that.
The cost of ownership couldn’t be any further apart!
So with that said…
Now there are a couple of flaws with my testing that I’ll discuss below.
And previously Ahrefs dominated all of my testing with the same flaws. They were untouchable. But this time..
They have rightly earned their place as the best backlink checker on the market right now in terms of database size.
It’s important you keep on reading because there are some flaws and other things to consider when looking at backlink checkers.
A Few Potential Flaws With This Experiment
I’ll be the first to admit…
My methodology for this test is far from perfect. So I wanted to highlight some of the potential flaws.
Flaw #1: Moz Only Provided Estimated Data
Moz asked to be included in this test a long time ago. But:
For some reason, they don’t track IPs or Subnets in their tool which I found bizarre.
Here is what they had to say about their estimated data (edited for ease of reading):
“Moz doesn’t collect IPs or Subnets. We had to build a unique model for this test. This was done by running a regression against our log of Root Linking Domains and against the percent of domains with unique IPs and Subnets.
Because these models have limited predictive power I added some constraints to the top and bottom level of predictions. For example; if the model predicted a number of Subnets that was more than 2s standard deviations from the mean, it defaulted to +/- 2x standard deviations of the mean.
The two regression models are…
IPS Y = -0.05x + 0.8599
SUBS Y = -0.1437x + 1.045
X is the log of the number of root linking domains. Y is the percent by which we need to multiply the RLDs to get the predicted number of IPs or Subnets.
The calculations should, therefore, be taken with a grain of salt.”
Flaw #2 – Each Tool Picks Up Different Links
Each tool has its own index. That’s 5x unique databases.
Not all databases share the same links. What appears in Ahrefs may not in SEMRush.
Let me offer an example.
If Ahrefs and Majestic are comparing the same site they may find the same number of links. But these links may not be from the same sources:
See what I mean?
If you wanted a full picture you would need to buy all 5x tools and cross-reference.
(I don’t recommend you do this. Keep it simple.)
Keep In Mind: Size Isn’t Everything
People always think that bigger is better, but as the old saying goes:
It’s not how big it is, it’s how you use it.
…sorry, I couldn’t resist…
But what’s more important than database size, is how you can use that data to increase your search traffic – because that’s the point of buying any SEO tool right?
This is often a personal decision and will require some time playing with different interfaces and features yourself.
Ahrefs has my favourite interface, for example.
It’s pretty intuitive and doesn’t take a lot of time to get used to. You can run searches and retrieve old data. It’s all there in the dashboard waiting for you.
They provide you with a lot of 1x click filters to help you find the information you need.
For example, you can filter by link type…
Or by platform:
They also generate lots of handy reports, like their best by links report:
Or their best by links’ growth one:
SEMRush operates the same intuitive dashboard-based system and easy to understand reports:
But it also has fewer filters and the ones they do have are not always useful.
See for yourself:
If neither of these seems right for you, you should also check out the much more budget friendly SEO Spyglass.
A lot of people like the overhauled version because it is-
You can also import backlink data from third-party tools like Google Search Console and SEMRush.
Take some time to play around and find the right tool for you.
Wrapping It Up
The results of the 5x million domain test are in:
But after reviewing the data and spending time using each of the tools, I can say with absolute confidence that-
Thanks to all of the backlink checking tools that took part in the test!
I look forward to repeating it again in the future because I never expected these results – but for now, the data has spoken.
Which is the best backlink checker for you? Let me know in the comments (but subscribe to my YouTube channel first)…
UPDATE #1: Ahrefs Official Response (Sept 10th 11:42am CST)
Here at Ahrefs we have 3 backlink indexes that vary in size and freshness:
- LIVE index. As the name suggests, this contains only live backlinks (to the best of our ability).
- HISTORICAL index. This contains every link we’ve seen since March 2013. Some are still live, some aren’t.
- RECENT index. This one is something in between the two. If we recrawl a page and see that the link was lost, we will keep it in this index for 90 days, because some links tend to reappear.
In this study, Matthew used the data that we provided to him from our LIVE index. Which led to the results that you’ve seen. We double-checked them too:
But if we use data from Ahrefs’ RECENT index, the number of “wins by Ahrefs” will grow rather significantly:
Both LIVE and RECENT Ahrefs’ indexes are designed to be (relatively) small, for the sake of containing a high percentage of links that are live.
While, to the best of our knowledge, our competitors don’t prioritise the freshness of their link indexes as much as we do (see this small study that compares freshness of different backlink indexes):
- SEMRush’s Fresh index: According to SEMRush, their Fresh index contains “backlinks that their bot saw in the last 6 months of crawling the Internet.”
- Majestic’s Fresh index: According to Majestic, their Fresh index now contains “120 days [~4 months] of crawl coverage.”
We don’t know the exact numbers for Moz, but this tweet from Russ Jones states that they don’t focus on freshness of their index rather explicitly:
Matthew Edit: When I spoke to Moz on the phone, they confirmed they would perfom badly in a freshness test. They also provided estimated data so take that with a pinch.
For that reason, we don’t think it’s fair to compare data from Ahrefs’ LIVE index to data from these indexes.
And the underlying message of this comment is to explain that Ahrefs doesn’t have a goal of showing big numbers via its LIVE and RECENT indexes. We have HISTORICAL index for flexing the “size” muscle, which neither Moz nor SEMrush have.
In other words, we might argue that the “Biggest Backlink Database” badge should probably be a battle between “Ahrefs HISTORICAL index” VS “Majestic HISTORIC index” as they are the two largest backlink indexes in the industry, with the biggest total coverage.
Another important factor to keep in mind when comparing data from different providers is the actual quality of that data. For example: Ahrefs tends to remove duplicate pages from our index and limits the number of pages we crawl from obviously spammy websites to increase signal/noise levels.
And finally (and this is what Matthew has mentioned himself), what also matters for a backlink tool is how usable the data is. And I’m happy to see that Matthew is choosing Ahrefs a winner in terms of actionable ways to research and apply our backlink data. :)
My Comments On Ahrefs Response
Tim makes a great point – every backlink database defines what is Live/Fresh/Historic differently and in this sense it is hard to compare the databases.
However the Ahrefs team were perfectly aware of the parameters of the testing and they supplied the data they wanted to be used in the test. But it has to be said, each tool defines what is fresh and historic data differently.
When reviewing the performance of the Live & Fresh databases of Ahrefs, they still lagged behind SEMRush-
But Ahrefs are right to call for a test of Majestic Historic vs Ahrefs Historic.
That is a test I happen to have ran last year–
As you can see based on historical data, Majestic absolutely destroyed Ahrefs but that was from last year and a lot can change! I am definitely intrigued to see how they shape up.
I also think there is a big problem with Ahrefs actively limiting the number of pages they crawl from spammy websites because those are precisely the links you want to know about if you are facing a manual link penalty.
With that said:
What is important is which tool helps you to build more links and increase your search traffic.
For one reason or another, anytime a conversation comes up about backlink checkers it always ends up being a conversation about size and it’s also a point that is widely used to market and sell a tool.
But the point of the tools is to help build links and neither “size of database” or “freshness of data” correlate to that.
At the moment no body dissects the data better than Ahrefs does.
They far exceed the competition when it comes to connecting the data to real life use cases which is why they won “most useful data” and will continue to be my “go to” tool for that reason.
I happen to know the product teams of their competitors are in absolute awe and amazement at how Ahrefs dissect the data and I don’t think Ahrefs stressed that enough in their response.
The battle continues!
UPDATE #2: SEMRush Data Investigation (Sept 11th 08:12am CST)
Each of the backlink checker tools provided the data for this test, which was then manually spot-checked to verify it’s accuracy.
We asked for a list of the total number of referring IP’s and referring subnets for each domain in the majestic million.
What we didn’t ask for – was the number of domains each tool counts for reasons explained here.
However Russell Jones from Moz called me this morning and pointed out that if you lookup a domain like Grepwords.com in SEMRush-
SEMRush reports that is has more referring IP’s than domains – which isn’t possible.
And because we didn’t collect referring domain data we didn’t see that either!
Russell has a pretty good idea of why this might be happening (Round Robin DNS) but I’m waiting to speak directly with the SEMRush team to get an official line.
Running through some manual checks this morning it is hard to find examples where there are more referring IP’s than domains so I dont think it’s a widespread issue, but it is an issue you should be aware of none the less.
This is a developing situation, I will update with more information as it comes.
UPDATE #3: Moz Challenges SEMRush’s Data (Sept 11th 09:36am CST)
Russell Jones has published an analysis of SEMRush’s data this morning which you really need to read–
In essence he believes SEMRush collects IP data at the link level instead of the domain level which is why you can find examples of sites having more referring IP’s than domains-
I am still awaiting an update from the SEMRush team.
UPDATE #4: SEMRush To Provide Updated Data Soon… (Sept 12th 09:19am CST)
SEMRush have acknowledged the issue and are going to provide an updated data set that only counts 1 IP address per domain.
They say that data will be ready in the next 4-5 days.
So until that happens…
I have striked out the entire post top to bottom because it could have a drastic impact on the results.