Hello All,

I also faced the same issue with Google indexing below pages without a view;
~/tag/products?tagId=54
~/manufacturer/products?manufacturerId=2
~/category/products?categoryId=45

As New York mentioned on the earlier post,
~/tag/products?tagId=39
~/mytagname
are two different things. 1st one is the AjaxCall for 
GetTagProducts
and the second one is the actual view for a product tag page. While we want google see and index the second page, we do not want it index the first one as it creates partially duplicate page and this page may end up on the SERP instead of the actual tag page.

Adding the 1st URL to robots.txt will not make much of a difference as the robots file is used to help bots navigate. Disallow rule will not simply "deindex" a page from its index. A page with crawl disallowed may still be in Google's index for years to come. As per Google Search Console help, only way to get these pages removed from the index is to use "noindex" tag.

However, as these Ajax call pages do not actually have <head> tags, it is not possible to add the noindex tag directly on the pages. And you don't want to add the noindex tag to partial view for the tags page.

So this is what I did instead. I have added 

Response.Headers.Add("X-Robots-Tag", "noindex, nofollow");
to "GetTagProducts" action just before it returns the "_ProductsInGridOrLines" partial view.

This is done on the \Presentation\Nop.Web\Controllers\CatalogController.cs file around line 300 (Nop version 4.40.4 with some modifications so your line may be different. Just search for "GetTagProducts" on your CatalogController.cs page)


public virtual async Task<IActionResult> GetTagProducts(int tagId, CatalogProductsCommand command)
        {
            var productTag = await _productTagService.GetProductTagByIdAsync(tagId);
            if (productTag == null)
                return NotFound();

            var model = await _catalogModelFactory.PrepareTagProductsModelAsync(productTag, command);

            // Added Response Headers to De-Index Ajax Call pages from Google 
            Response.Headers.Add("X-Robots-Tag", "noindex, nofollow");

            return PartialView("_ProductsInGridOrLines", model);
        }


This change seems to has worked as I can see the "noindex, nofollow" as response header when I call these Ajax call pages.

After the Change:


I hope this helps.
Thanks
Volkan