8 Quick Checks for Duplicate Content by Stephanie LeVonne #338

Ep. 33835 min2016-09-23

Duplicate content harms crawl budget and rankings; the eight checks cover HTTP/HTTPS versions, scraper sites, abandoned subdomains, staging environments, dynamic URL parameters, mirrored subdirectories, syndicated content, and appreciably similar pages.

Cover art for 8 Quick Checks for Duplicate Content by Stephanie LeVonne #338
Key takeaways
duplicate content is the equivalent of overdrawing your checking account. Except instead of paying costly fees each month, you'll be sacrificing your precious crawl budget.

About this episode

Do you have duplicate content? You may be getting penalized for dupe content right now and not even know it. Check out this week's podcast for 8 checks you can quickly do to see if you have duplicate content killing your ranking. This week's article is "Don't get duped by duplicate content: 8 quick checks for every SEO" by Stephanie LeVonne over at Search Engine Land.

Questions this episode answers

What are the most common sources of duplicate content on a website?

According to this episode, the eight main sources are: having both HTTP and HTTPS versions live, scraper sites copying your content, forgotten subdomains, unblocked staging environments, dynamically generated URL parameters from faceted navigation, mirrored subdirectories for multiple locations, syndicated content without canonical tags, and pages with appreciably similar content.

How do I check if my site has duplicate URL versions?

The episode recommends manually visiting your site using HTTP, HTTPS, www, and non-www combinations, and also testing with and without index.php. In WordPress you could end up with as many as eight live versions of the same site if none of these are redirected or consolidated.

How should I handle syndicated content to avoid duplicate content issues?

The episode says the publisher syndicating your content should add a rel canonical tag pointing to your original page, noindex the syndicated copy to prevent it from ranking, and link back to you within the article. Following these steps means neither party faces a duplicate content penalty.

What should I do if two articles on my site cover similar topics?

The episode advises merging them into one piece of content if a reader can get the same answer from both. If the two articles address genuinely different issues within a topic, they can coexist, but if there is too much overlap and the content is appreciably similar, consolidation is the right move.

In their words

if you have these duplicate content issues and other type of issues affecting these bots, then technically your site won't get crawled as much on her terms. You're sacrificing your crawl budget.

Host

the key is making sure your content is different enough to warrant multiple pages.

Host
Terms defined in this episode
Crawl budget
The capacity Google Bot and other crawlers allocate to indexing a site; duplicate content and other technical issues reduce how frequently a site gets crawled.
Absolute URL
A full URL including the domain, such as ewebresults.com/seo-podcast, as opposed to a relative URL like /seo-podcast; using absolute URLs helps Google identify the original source if content is scraped.
Rel canonical tag
An HTML tag used to tell Google which URL is the original or preferred version of a page, used to consolidate duplicate or syndicated content under one credited URL.
Dynamically generated parameters
URL variations created automatically by faceted navigation or category structures in e-commerce or CMS platforms, which can produce multiple URLs for the same page and cause duplicate content.

Full transcript is being added.

This episode answers

Matthew Bertram, host of The Best SEO Podcast
Matthew Bertram
AI keynote speaker · Owner & CEO, EWR Digital · President, ModalPoint

Matthew Bertram is an AI keynote speaker, creator of DIG™ (Digital Information Governance), and owner and CEO of EWR Digital. He helps energy and industrial leaders win visibility in AI search (GEO and AEO), is President of ModalPoint and CMO of the Oil & Gas Global Network, hosts The Best SEO Podcast, and has authored eight books including LLM Visibility. More about Matthew Bertram.

Related episodes

The show explains it. EWR Digital does it.

The Best SEO Podcast breaks down SEO and AI search; EWR Digital, the agency behind the show, implements it for operators. Start with a free, no-pitch audit of where you stand in Google and in AI answers.

Get a free AI visibility audit