SEO - Duplicate content

Hi guys,

Because of the way URL re-writing is done...  there is a huge duplicate content issue, and the fact that nopCommerce doesn't come with canonical tags makes this worse..

Because I can access a page like this...

Products/178-actual-product-url.aspx

Or like this

Products/178-blahblahvoiweh;iouwef'irj235945uiefjwkefjslkdjfsdfhelllooooooo.aspx

This means that once search engines have deemed them to be duplicates, the one with the lowest page ranking will be dropped...  Which doesn't sound too bad.. but take a case in point;

Say I have lots of people linking my pages and improving my page ranking...

Then EVIL massive competitor company comes along..  and does the following link on their massively well ranked website;

http://www.mysite.com/178-a.aspx.">click here</a>

Can you see the problem?  My high quality link with all the keywords will be dropped, and my page ranking will be obliterated!

This is something that can and does happen and essentially means competitors can affect your SEO.


I'm going to modify my version of nop so that it checks the URL stored in the database against the ID number..  and if it doesn't match up.. it will throw a 404..

I'm also trying to find an easy way to implement canonical URLs as well

thanks :)

what are canonical URLs ???

a canonical URL tag is in the header

it tells search engines what the preffered URL to use for a page is

otherwise search engines will try to index EVERY querystring for a particular URL, and find they are all duplicates

thanks i will investigate it..
why dont you you promote it in order to vote it=? in codeplex?

I know it seems like having LESS url's landing on 404 pages is better... but duplicate content is more of an issue for SEO reasons which are more important for traffic...

I have fixed the category page with 6 lines of code, am going to apply these to the product / topic etc.. pages as well..

This is my suggested code to fix this problem (this example is for category) :)


       protected void Page_Load(object sender, EventArgs e)
        {
            if (!Page.IsPostBack)
            {
                CommonHelper.EnsureNonSSL();
            }

            if (category == null || category.Deleted || !category.Published)
                Response.StatusCode = 404;

            string catURL = SEOHelper.GetCategoryURL(category);
            string currentURL = Request.Url.AbsoluteUri;

            if (!currentURL.Contains(catURL))
                Response.StatusCode = 404;


Tested and working!  woop

xerath wrote:

I'd like to see this incorporated into nopCommerce.

Here's a good reference.

http://www.seomoz.org/blog/canonical-url-tag-the-most-important-advancement-in-seo-practices-since-sitemaps

Here's what I've done to fix the problem...

First check out the code in my last post..


We compare the SEO link with the current URL, if they don't match we throw a 404, we also change the default response which deals with invalid ID numbers by changing it from redirecting to the home page... to throwing a 404.

I've managed to do this for category, blog, manufacturer, product and news.


Next thing is the URL re-writer...  remove anything which you aren't using... if you don't have a blog, remove it from the re-writer

The re-writer processes all requests IN ORDER...  so at the bottom I put this

    <add name="CatchRemaining" virtualUrl="^(.*).aspx"
           rewriteUrlParameter="ExcludeFromClientQueryString"
           destinationUrl="~/NotFound"
           ignoreCase="true" />  

"NotFound" won't appear in the browser address bar, but it will throw a genuine 404 error!   I also have a 301 map for old .html files on the previous site, so I put a catch all rule in for .htm requests as well...   Anything else will get dealt with by IIS.

Once I sort out canonical URLs, I'll have a completely SEO friendly website :)

Vote for on CodePlex, if important to you.

http://nopcommerce.codeplex.com/workitem/8731

done

as well as canonical tags it would be good for all invalid URL's to be 404'd

which can easily be done using the code I supplied

solved!

Here is how I did it...

My previous code was designed to 404 invalid requests, I have added the following onto it...  This example is for product pages, it would have to be adjusted to whatever page you were working on..


//If it's invalid throw a 404 instead of re-direct

            if (product == null || product.Deleted)
                Response.StatusCode = 404;



//get the SEO friendly link from the database

            string prodURL = SEOHelper.GetProductURL(product);


//get the current URL

            string currentURL = Request.Url.AbsoluteUri;


//GO COMPAARRREEEEEEE

            if (!currentURL.Contains(prodURL))
                Response.StatusCode = 404;


//Normal META STUFF
            
            string title = string.Empty;
            if (!string.IsNullOrEmpty(product.MetaTitle))
                title = product.MetaTitle;
            else
                title = product.Name;
            SEOHelper.RenderTitle(this, title, true);
            SEOHelper.RenderMetaTag(this, "description", product.MetaDescription, true);
            SEOHelper.RenderMetaTag(this, "keywords", product.MetaKeywords, true);

//Building the canonical link from the variable I already declared representing the SEO friendly URL

            HtmlLink canonical = new HtmlLink();
            canonical.Href = prodURL;
            canonical.Attributes["rel"] = "canonical";

//DO IT!
            Page.Header.Controls.Add(canonical);




All together now;

            if (product == null || product.Deleted)
                Response.StatusCode = 404;

            string prodURL = SEOHelper.GetProductURL(product);
            string currentURL = Request.Url.AbsoluteUri;

            if (!currentURL.Contains(prodURL))
                Response.StatusCode = 404;

            
            string title = string.Empty;
            if (!string.IsNullOrEmpty(product.MetaTitle))
                title = product.MetaTitle;
            else
                title = product.Name;
            SEOHelper.RenderTitle(this, title, true);
            SEOHelper.RenderMetaTag(this, "description", product.MetaDescription, true);
            SEOHelper.RenderMetaTag(this, "keywords", product.MetaKeywords, true);

            HtmlLink canonical = new HtmlLink();
            canonical.Href = prodURL;
            canonical.Attributes["rel"] = "canonical";
            Page.Header.Controls.Add(canonical);