duplicate content login?returnUrl=%

1 month ago
version 4.2
we are seeing huge volumes of duplicate contnet in google search on our product pages and content pages.
its seems that google see the pages twice
once the normal page
then a duplicate version of the page with
login?returnUrl=% in front of the page
is there a way to stop this duplication happening ?
1 month ago
NopCommerce generates a robots.txt file that tells Google what it can/cannot crawl, you can view your site's file by going to:
http://yoursite.com/robots.txt

If you haven't made any modifications it does not explicitly disallow "login?", but you can add a custom "robots.additions.txt" file that NopCommerce will append to the one it generates.

E.G. create a txt file in notepad named "robots.additions.txt" with:

Disallow: /login?returnUrl=*


And save it in your site's root directory (where Nop.Web.exe lives).

If you want to bypass the built-in robots.txt and roll your own, name it "robots.custom.txt" and NopCommerce will use it instead, but I don't recommend doing that.  There are a lot of default links you don't want any bot crawling.

You may also want to sign up for Google's search console so you can see what it sees:
https://search.google.com/search-console/welcome

Google should be assigned to a built-in search-engine account, but if it's being considered a Guest account and you have Guest navigation disallowed in ACL settings then it's going to be redirected to login for every single link it finds.
1 month ago
thanks so much for the very helpful answer.  can i add

Disallow: /login?returnUrl=*  

to the existing robots.txt file ?

cheers !
1 month ago
sorry i see now. its auto generated.
ive added the additonal file.

another question for you if thats ok ?

google alos see the list views and grid views as duplicate content  i.e.

thewebsiteurl.co.uk/ then >>>>>   ?orderby=5&viewmode=grid
thewebsiteurl.co.uk/ then >>>>>   ?orderby=5
thewebsiteurl.co.uk/ then >>>>>   ?viewmode=list&orderby=11
etc etc ?orderby=5&viewmode=list

is there a way to exclude these?
1 month ago
Not sure off the top of my head, but this might address that:
https://support.google.com/webmasters/answer/6080548?hl=en&ref_topic=10267931

And they have a great tool for testing:
https://www.google.com/webmasters/tools/robots-testing-tool