I discovered that Google is crawling unpublished product pages, index those pages and I can see the content of those pages without being logged as Admin.
As a Result, in Google Search Console, I have 3 291 pages send and indexed (sitemap) and 3 991 indexed but not send from Sitemap. Many of those 3 991 page are unpublished products.
- Our site is in FR / EN and we submit FR/sitemap.xml to Google.
- Unpublished pages are not in the sitemap
1. What is the normal behavior for our site for unpublished pages ?
It's not fun for customers to consult out of stock product pages.
2. Is there an SEO mecanism in NopCommerce to stop the Crawler to index unpublished pages ?
1. It depends on your business needs. By default nopCommerce allows to visit unpublished products
2. You can disable it in admin area > configuration > settings > catalog settings > "Allow viewing of unpublished product details page". You can also check the "Discontinued message for unpublished products" checkbox as well (related to the first question)
Thank you very much A.M. for your answer.
Actually, an SEO expert contracted for an SEO audit because we have bad ranking is telling to my client that NopCommerce is a bad framework for SEO. As long as I live, Nobody will insult NopCommerce. I think that he is not an expert at all of e-commerce SEO.
For a Web FR / EN site of 90 categories, 160 manufacturers, 1500 published products (out of a total of 4600), Google Search console shows 7200 valid pages (almost twice as the real number because google index unpublished products) and 45 000 exclusions (25 000 blocked by robots.txt, 10 000 correct url refered by canonical URL , 6000 pages with redirection (many /changelanguage) and 1200 explored, but not indexed + some other exclusions).
Related to question 2 (Allow viewing of unpublished product details page):
1- what will happen in a Google point of view for those pages if 1 day, they are missing and next day, they are available because new stock is received. Will google will give us bad ranking or something like that.
BTW, I submited a question (from contact us form) to validate if your premium support service convers SEO support / Google ranking investigations. Just let me know if you know it.
Thank you very much for your help.