Robot.txt Disallow

Currently there is no way to block search indexes. Have a look at adding a new setting in order to block search engine crawling which will output the robots.txt file as

User-agent: *
Disallow: /

The Sitemapseo has a similar problem, to disable the sitemap.xml you need to CommonSettings.SitemapEnabled which will hide your Sitemap html page which should not be effected by removing crawler access.

This will prevent google from indexing test sites.

Confirm that the problem is relevant.
For the test site requires appropriate robots.txt


By the way, with robots.txt there is another problem.
In multilingual sites are generated in the form of records:
Disallow: ru/checkout

Without first "/"

I believe you can edit what the robots.txt will create by editing the Nop.Web.Controllers.CommonController.RobotsTextFile()

This is what is used to dynamically generate the robots.txt

Same here. How can i edit the robot.txt file.

HI
is anyone has an idea
I am using nop 3.90 facing an issue on the staging site
In my robot.txt

User-agent: *
Disallow: / 

so it means in the dev site it will stop crawling ,
but as I am searching with my product it displaying

can any one guide for the same
Thanks in advance