Start a new line in robot.txt

I am trying to add

"User-agent: RSiteAuditor
Allow: /"

to my robot.txt document using the web Gui in the "additional rules" box  in the robots.txt section however  what I get in the robots.txt file is "User-agent: RSiteAuditorAllow: /" all on one line, how do I get around this, I have also tried the adding a  robots.additions.txt file but this doesn't show any changes when I look at www.mywebsite.co.uk/robots.txt

Yes, this behavior is wrong, I've made adjustments to the code, see this commit for more details.

As for the robots.additions.txt file, it must be located in the wwwroot directory at the root of your site, and then its contents will be included in the robots.txt data

logikjas wrote:

Great thank you, how do I get this sorted on my end?

You can make changes yourself to your source codes and rebuild your site. Or you can just use the robots.additions.txt file for now instead of editing the "additional rules" setting

logikjas wrote:

I have created a robots.additions.txt file in the wwwroot folder but the new rule is not displaying on the www.mywebsite.co.uk/robots.txt, is there something i have done wrong or is this another issue?

This is very strange, I checked the functionality of this block on the 4.60.0 sources and everything works without problems, please check your logs for errors, perhaps the file does not have permissions, or it is blocked by another application, or your data may not be saved in it. If you do not find any problems with the file and rights and there are no errors in the logs either, then: indicate your version of nopCommerce and send your file for analysis

logikjas wrote:

Hi

I have also create a txt file, named robots.additions.txt with the follow string:
Disallow: /product/estimateshipping/

I have uploaded it in the wwwroot folder but when I check the www.mysite/robots.txt I cannot find the rule Disallow: /product/estimateshipping/.
(cleared the cache and restarted the application)

Using nop 4.404

Make sure it is a text file called robots.additions
i.e. Not a file called robots.additions.txt

Also it needs to be in the Website root directory
i.e C:\inetpub\wwwroot\nopCommerce4.40.4
not in /wwwroot

Make sure it is a text file called robots.additions OK DONE

"Also it needs to be in the Website root directory
i.e C:\inetpub\wwwroot\nopCommerce4.40.4
not in /wwwroot"

Ok I have done also this one but my site is stored in e:\website\mysite\ and there isn't any inetpub folder in the e:| directory....... There is in the c:\inetpub\wwwroot\ I have pasted there but nothing changed

Sorry but I'm not very expert
Lorenzo

uciocan wrote:
It does not go in a public folder - should be in the website root directory
i.e. the same directory as web.config file for the website