Twenty years of ranking factor studies, side by side

Every Moz and Searchmetrics correlation we could recover, the vendor studies that replaced them, and two decades of expert votes, set against what Google's own leaked documentation showed in 2024. Each number carries a grade for how we read it.

What experts believed against what correlated, in one year

Moz's 2015 edition is the cleanest test in the record. It asked its panel to score each factor and measured the same factors against rankings, then printed both. Plotting one against the other splits the factors into two failure modes that recur through every later study.

Top left are the social counts: Facebook, Google +1 and tweets, rated lowest of anything on the survey and correlated nearly as well as links. Matt Cutts said in 2013 that Google was not using +1s that way and that good content earns both shares and links (Hacker News). Bottom right are keyword in title and mobile, rated near the top and correlated at about zero. They are saturated: when nearly every ranking page has the keyword in its title, there is no variance left for a rank correlation to find. The leak's titlematchScore says titles are scored regardless (iPullRank).

Every correlation we could recover, 2011 to 2016

This is the only stretch in which two independent vendors measured overlapping factors with the same statistic. Moz values are the most reliable in the record: printed on the page, or printed as labels on its 2011 chart images. Searchmetrics values come from a press write-up (2012) and from chart labels matched by order in its 2015 whitepaper transcript. Its 2013, 2014 and 2016 coefficients survive only as search snippets or unread chart images, so they are left out.

Two patterns hold through every caveat. Link and social counts sit between 0.17 and 0.39 in both vendors' data, while on-page keyword measures sit between −0.02 and 0.15. And Moz's 2013 and 2015 numbers barely moved, which suggests the rise from 2011 owed at least as much to the switch from the top 30 to the top 50 results as to anything Google changed. On the 2015 sample, Ahrefs' referring-domain count correlated 0.34 against 0.30 for Moz's own count: the link index alone moves the coefficient (Moz 2015).

After 2016, every vendor brought its own ruler

Searchmetrics dropped its universal study in December 2016 for industry studies that report ratios instead of rho (press release; why it ended). The tool vendors that followed used different depths, statistics and outcomes, and several of their strongest factors are their own product metric.

Backlinko published no coefficients in either study, so its findings are directional. SEMrush 2017 reported that every correlation was below 0.3, which is why it switched to a random forest ranking. Its top four factors were traffic estimates, which raises the question of whether sites rank because they get traffic or the reverse. Two descriptive series need no coefficient at all: the average #1 page was 2 years old in 2017 and 5 years old in 2025 (Ahrefs), and domains passing Core Web Vitals at the Page Experience rollout gained 3.7% visibility against 2.7% for the average domain (SISTRIX).

What the experts voted for, 2005 to 2015

Moz asked its expert panel to score ranking factors in six editions. The scoring scale changed almost every time, so the raw numbers cannot be lined up. Ranks can: below, each factor is placed by where the panel put it that year.

Exact-match domains are the sharpest swing. Experts put them joint 8th of 9 in 2005, then made them the single highest-rated factor of 2011 (97.0 of 100), in the same edition whose own data showed the .com EMD correlation falling from 0.38 in June 2010 to 0.22 (Moz 2011). Anchor text went the other way: 1st in 2009, 12th of 18 by 2015, while its correlation sat at 0.24 to 0.28 throughout. Practitioners updated on penalties, not on the studies.

The 2013 survey also asked what would grow. Those forecasts can now be scored:

• 96 of 121 expected authorship to grow. Authorship display was dropped. Moz measured it at −0.03.
• 92 of 121 expected structured data to grow. Correlation 0.02, then 0.01. Backlinko 2020 found none.
• 89 of 121 expected google+ to grow. The network was shut down.
• 82 of 121 expected social signals to grow. Last-rated Moz category by 2015; Google said it does not use them directly.
• 87 of 121 expected exact-match domains to decline. Came true. The leak names an exactMatchDomainDemotion.

Local search: the beliefs that moved most were never measured

The Local Search Ranking Factors survey has run since 2008 on panels of 20 to 47 local specialists. No correlation study of the local pack exists to check it against, so every number in this section is belief.

Part of the profile's rise is relabeling, because proximity and keywords in the business name sit inside that cluster. Citations fell from the #2 specific factor of 2009 (LSRF 2009) to 6% of the pack in 2026 with no Google statement either way. Google's own documentation names relevance, distance and prominence (Google), which confirms the clusters exist and says nothing about their weights. The 2026 edition added an AI visibility dimension, in which on-page leads at 24% and citations rank third at 13%, ahead of the profile itself at 12%.

After Moz stopped asking

The mass poll and the expert panel disagree on links, 17.4% against 54.8%, which tracks who was asked and how more than any change in Google. First Page Sage publishes percentage weights with no stated method; it is listed because it circulates as a study.

Why these numbers cannot go on one chart

The tables invite trend lines. Most of those lines would chart methodology, not Google.

SERP depth changes rho mechanically. Restricting the range of positions shrinks a rank correlation, so a move from the top 30 to the top 20 lowers every coefficient before Google does anything.

Averaging changes the size of the number. Moz and Searchmetrics averaged per-keyword coefficients. SEMrush 2024 correlated position against each position's average, which smooths away variance, so its ~0.47 for text relevance cannot sit beside Moz's 0.13 for a language model. Marcus Tober's 2014 write-up has the cleanest warning: the share of links with the keyword in the anchor fell from about 40% to about 27% while its correlation rose, because rho measures the slope across positions, not the level of use (Moz blog).

Survey scales changed every edition. Moz went from 1 to 5, to percent, to 0 to 100, to 1 to 10, which is why the survey section shows ranks rather than scores. Whitespark's own over-time chart re-plots earlier local editions one to two points away from what those editions printed.

Vendors grade their own metrics. Page Authority was a link model fitted to rankings, so its place atop every Moz edition partly reflects its design. Surfer's Content Score, SEMrush's Authority Score and Ahrefs' DR each appear as factors in studies their makers published. Searchmetrics' 0.67 for click-through rate is the same problem in pure form: position sets CTR.

Two distortions apply to every study: saturation hides real signals (98% of top 20 pages use HTTPS), and authority confounds anything strong sites tend to have, such as shares, length and brand mentions (SEJ).

Fifteen factors, three readings each

Later evidence comes in three strengths: Google's public statements, sworn testimony in US v. Google, and the Content Warehouse API leak, which Google confirmed as authentic in May 2024 (Wikipedia). The leak lists 2,596 modules and 14,014 attributes with no weights, so it proves a feature exists, not how much it counts. The testimony is more specific. Asked in October 2023 whether "the navboost system memorizes past clicks that have been issued for past queries," Google's Pandu Nayak answered "Yes," and confirmed it covers "all queries received in the past 13 months" (trial transcript, p. 6405).

The record supports a sharper rule than "correlation is not causation." Google's denials held up for on-page folklore: meta keywords, keyword density, LSI and word count (Mueller on word count). They did not hold up for aggregates that can be gamed: clicks, site authority, host age and Chrome data. The correlation studies failed in the mirror pattern, over-reading confounded aggregates (social, traffic, length) and under-reading saturated basics (titles, HTTPS, mobile).

The same study, now pointed at AI answers

The template has moved to a new outcome. Ahrefs correlated brand signals with visibility in AI Overviews and found branded web mentions at 0.664, about twice Domain Rating at 0.326 (Ahrefs). That is the highest coefficient anywhere in this history, and it is a cross-brand correlation built on the publisher's own mention counts. Meanwhile, the share of AI Overview citations drawn from the organic top 10 fell from 76% to 38% (SEJ). Those numbers deserve the skepticism the 2013 +1 correlation eventually earned.

We ran the belief half of this page forward. Belief vs Measurement takes the September 2026 expert survey and checks it against 12,778 AI answers we logged, including how rarely an engine returns the same sources twice. The 18-Year Shift counts what was actually said on this show from 2009 to 2026, which is the practitioner record with the survey filter taken off.

How this was compiled

Every value is graded by the copy we actually read, not by the original study's quality:

Left out on purpose: Searchmetrics 2013 and 2016 coefficients (search snippets only; the originals 403 or 404), a widely repeated Searchmetrics 2014 "relevant terms 0.54" (we could not trace it to the study), percentage-change claims about the 2018 local survey that do not reconcile with the survey itself, and any "Moz 2017" study (Moz's last general edition is 2015). Still open: Searchmetrics 2015 values are order-matched and need checking against the original whitepaper, the SEMrush 2024 values marked C are read off its charts.

Sources