to my robot.txt document using the web Gui in the "additional rules" box in the robots.txt section however what I get in the robots.txt file is "User-agent: RSiteAuditorAllow: /" all on one line, how do I get around this, I have also tried the adding a robots.additions.txt file but this doesn't show any changes when I look at www.mywebsite.co.uk/robots.txt
Yes, this behavior is wrong, I've made adjustments to the code, see this commit for more details.
As for the robots.additions.txt file, it must be located in the wwwroot directory at the root of your site, and then its contents will be included in the robots.txt data
logikjas wrote:
I am trying to add
"User-agent: RSiteAuditor Allow: /"
to my robot.txt document using the web Gui in the "additional rules" box in the robots.txt section however what I get in the robots.txt file is "User-agent: RSiteAuditorAllow: /" all on one line, how do I get around this, I have also tried the adding a robots.additions.txt file but this doesn't show any changes when I look at www.mywebsite.co.uk/robots.txt
You can make changes yourself to your source codes and rebuild your site. Or you can just use the robots.additions.txt file for now instead of editing the "additional rules" setting
logikjas wrote:
Great thank you, how do I get this sorted on my end?
I have created a robots.additions.txt file in the wwwroot folder but the new rule is not displaying on the www.mywebsite.co.uk/robots.txt, is there something i have done wrong or is this another issue?
This is very strange, I checked the functionality of this block on the 4.60.0 sources and everything works without problems, please check your logs for errors, perhaps the file does not have permissions, or it is blocked by another application, or your data may not be saved in it. If you do not find any problems with the file and rights and there are no errors in the logs either, then: indicate your version of nopCommerce and send your file for analysis
logikjas wrote:
I have created a robots.additions.txt file in the wwwroot folder but the new rule is not displaying on the www.mywebsite.co.uk/robots.txt, is there something i have done wrong or is this another issue?
I have also create a txt file, named robots.additions.txt with the follow string: Disallow: /product/estimateshipping/
I have uploaded it in the wwwroot folder but when I check the www.mysite/robots.txt I cannot find the rule Disallow: /product/estimateshipping/. (cleared the cache and restarted the application)
Make sure it is a text file called robots.additions OK DONE
"Also it needs to be in the Website root directory i.e C:\inetpub\wwwroot\nopCommerce4.40.4 not in /wwwroot"
Ok I have done also this one but my site is stored in e:\website\mysite\ and there isn't any inetpub folder in the e:| directory....... There is in the c:\inetpub\wwwroot\ I have pasted there but nothing changed