As an SEO Agency, we are very aware of the problems raised by duplicate content.
Most web users will be blissfully unaware that duplicate content even exists but are very unlikely to really care about it even if they do see similar content on different urls.
Why is duplicate content a problem?
Search engines do not wish to have their results pages listing pages that are all identical, as it would not be a good experience for their users. For this reason, ‘duplicate content’ filters will strip out multiple instances of the same content.
It can be easier than you may think to create duplicate content on your own site, without meaning to.
Whilst it is not immediately apparent, many websites serve the same content on both http://www.domain.com and http://domain.com. This is a good example of duplicate content appearing on the same site and introduces the issue of canonicalisation, where Google will try to determine which url to use and then only show that version in its index.
There is a good summary of the issues surrounding canonicalisation at http://www.mattcutts.com/blog/seo-advice-url-canonicalization/.
Another example is having a ‘print friendly’ version of the page which exists on a separate url but which contains the same content.
The difficulty that faces the search engines is deciding which version of the content is the ‘right’ version.
There are several ways that webmasters or web site owners can diffuse this problem, such as “normalizing” the URLs, picking a canonical url and always linking to that url in the internal links and they can add permanent 301 redirects from the non-canonical url to the canonical one.
Webmasters do not, however, have complete control. They cannot control which urls other website owners use to link to them, which can help confuse the search engines.
Google has designed the ‘canonical link element’ to help address the issue and it has been confirmed that Yahoo!, Microsoft and Ask will be supporting it.
The new element works like a mini 301 redirect and works by placing a small line of html code between your head tags. For example, on http://www.example.com/page.html?=ghjk7450324587, the element would be written in as:
<head>
<link rel=”canonical” href=”http://example.com/page.html”/>
</head>
Whilst this code works very similarly to a 301 redirect, there are some differences. For example you cannot redirect to different domains with the new element. You can, however, redirect to different subdomains and from http:// to https://.
The element has been said to be a ‘hint’, which will subsequently be taken as a ‘strong hint’ by Google, but it is not mandatory and they will reserve the right to choose when they use the suggestions.
We would always recommend that is far better to avoid duplicate content and normalise URLs from the start, but it is hoped that the new canonical element will work well.
Only time will tell…
