I noticed that my site have command for Google and other bots INDEX, FOLLOW.
My robots.txt file have line:
Disallow: /admin
As I can see most of backend URLs are:
https://site.com/Admin/Setting/GeneralCommon
https://site.com/Admin/Product/List
...
So, my question is, should I add following command into robots.txt file?
Disallow: /admin/*
I am just worry not to block Google bot where I shouldn't.
Thanks a lot for looking into.