About Google Webmaster Tool Soft 404 Error

Hi,

Anyone experiences about Google Webmaster Tool below error ? How we can solve it ? Project version default 3.10 and product size 16.000.

"
While crawling your site, we have noticed an increase in the number of transient soft 404 errors around 2014-01-27 08:00 UTC (London, Dublin, Edinburgh). Your site may have experienced outages....."

Hi,
Yes, we purchased a domain with Magento on it. More than 40K of urls which caused 404 errors. I customized the robots.txt mechanism to load a file with more disallow stuff. Then Google is still not happy. You need to rewrite. I used urlrecord with IISrewrite. In IISRewrite I replaced the existing products to be rewritten to NOP to allow better SEO.

The other urls throwed 404 errors. I wrote static maps in IIS to have them 301 redirects. I also added some dynamic maps for instance for SID=. Then I refreshed on a daily basis my web.config with the rewrite rules and urlrecord. Now there are a couple of errors. Maybe due to the fact that the robots.txt is for all sites instead of each particular site. So I cannot do perfect disallows.

You are talking about soft 404. I believe they vanish or are also rewritten in the above metholody.

J.

https://www.nopcommerce.com/boards/t/28211/how-to-add-a-page-to-the-robotstxt-file.aspx about own robots.txt file

Quantis001 wrote:

Hi,

Thank you for help i ll try to test now .

Post you PM with our rewrite rules. You can define them as well with IIS Rewrite module (need to install this on IIS). Please do not publish them online since on the long term I want to remove those rewrites for performance reasons.

Quantis001 wrote:

Ok . Thank you so much .