can you post (via copy and paste) one of the errors that you are getting from the NOP log so that maybe we can figure out now what the issue may be?
Just a hunch, but you'll probably need to work on the SEO tab's "Search engine friendly page name" field for whatever type of content that you're working with.
Log level Error Short message Error 404. The requested page (/en/audi) was not found Full message IP address 207.46.13.217 Customer builtin@search_engine_record.com Page URL https://*********.com/page-not-found
Ok..... as stated earlier, be weary of searchbot related errors in your logs, since often they are trying page combinations that are impossible for users to navigate to, or they could simply be changes to your site's structure that you've initiated that no longer exist and the bot is using a cached URL. If it doesnt find it, it will remove it from it's records eventually.
Is the page that you listed in the error log existing and accessible from a browser by your customers?
With that being said, there's nothing you can do to stop the bots from doing this. Assuming they're working like they're supposed to, they should be removing those failed page load attempts from their records so that they'll no longer attempt to try and access them in the future.
This is exactly how the search engine crawlers are supposed to work.
As for your logs, you can do some cleanup in the DB if you want, or you can clear them periodically (which is what I do after reviewing them). You can also utilize the exclude setting in the admin section to eliminate various things from being logged.
In my case this page (category) exists and accessible by me - why it thrown the 404 error page was not found?
Log level Error Short message Error 404. The requested page (/ru/рюкзак-для-дрона) was not found Full message IP address 46.229.168.150 Customer builtin@search_engine_record.com Page URL https://****.com/page-not-found Referrer URL Created on 3/15/2019 8:22:16 AM
This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.