Hi Guys,
Just wanted to share some issues i faced with the way how sitemap.xml implemented right now with nop commerce.
As you all know sitemap.xml is generated on the fly on nop commerce if you type in http://demo.nopcommerce.com/sitemap.xml
This will work if you have fewer products. But for my store with over 20K products, it took considerable time to serve this file. And we had 5 sites running on same installation. But google/bing bots wont wait that long.
So i worked out my way,
01. Created a method called "createsitemap" which points to http://demo.nopcommerce.com/createsitemap. This will be the exact code which executed under http://demo.nopcommerce.com/sitemap.xml method call. Instead of serving back to client, this will save the sitemap for that particular store and name it on hard drive as sitemap-<storeid>.xml name format. Example sitemap-1.xml.
02. Created a Scheduled task, which calls this "createsitemap" for every store in a given time frame.
03. Modified the code under http://demo.nopcommerce.com/sitemap.xml method to pick the correct sitemap-<storeid>.xml instead of generating one.
So sitemap will be kept up-to-date automatically and will be served faster to clients as well.
I don't think we need to do same for robots.txt as its pretty fast.
If you guys think this would help you also, please reply.