Well, I found out that the slowest operations in generating the feed is getting the localized values for the Name and Description of the products (as those are really slow operations) and generating the XML. If you could eliminate those operations, for instance, if you have only 1 language in your store you won't need to get the localized value for the Name and Description but rather get the name from the Product object you've gotten from the database you would greatly decrease your generation time. I am not sure about the text file as I've not tested that myself.
Regards, Anton
You might be on to something here. Ill try and change the original plugin and ill test.
Locally im down to 200 ms per product after removing all localization. With 5000 products that 16 minutes. The thing taking time now is the pictureUrl.
Any ideas here? We are using azure storage for blobs.
So, i found out that web apps in azure have a set timeout for 230 seconds, which cannot be adjusted.
"No, you cannot increase the timeout for Azure App Services (it is 230 seconds). You can move to a cloud services or IIS hosted on a VM where you have control over those settings. You can also move to an async model where the client makes a request, gets some sort of ticket or identifier that is can poll back to see if the processing is done. There are many examples of async type models for web apps out there you can choose from. Ref:https://social.msdn.microsoft.com/Forums/en-US/05f254a6-9b34-4eb2-a5f7-2a82fb40135f/time-out-after-230-seconds?forum=windowsazurewebsitespreview"
I too noticed the picture url taking up too much time. That is because it takes 2 or 3 requests to the database (if I remember correctly) to get the picture path. I couldn't think of a better way to do that though. My timeframe for my project made it impossible for me to dive into that so I don't have any advice on that for you.
The other option I was looking at was to make the action method accepting the request by async and get all of the logic into another method (synchronous this time) that would be called by the first method. The trick here is that the async action method won't wait for the synchronous one to end it's work. That means that the async action method will return immediately and the sync method will continue its work in the background resulting in no timeout. This requires a lot of customization to the plugin though. Also, you cannot use the nop object context but you will have to get the connection string to your database and make a new connection. I suggest you try everything else before you try that since this method is really hard to implement and time-consuming.
I too noticed the picture url taking up too much time. That is because it takes 2 or 3 requests to the database (if I remember correctly) to get the picture path. I couldn't think of a better way to do that though. My timeframe for my project made it impossible for me to dive into that so I don't have any advice on that for you.
The other option I was looking at was to make the action method accepting the request by async and get all of the logic into another method (synchronous this time) that would be called by the first method. The trick here is that the async action method won't wait for the synchronous one to end it's work. That means that the async action method will return immediately and the sync method will continue its work in the background resulting in no timeout. This requires a lot of customization to the plugin though. Also, you cannot use the nop object context but you will have to get the connection string to your database and make a new connection. I suggest you try everything else before you try that since this method is really hard to implement and time-consuming.
Regards, Anton
Yeah, i think changing the picture method will take too much time atm.
Do you know if a scheduletask will time out as well in a webapp?
After version 4.0 the schedule tasks use requests so they will timeout. If you are using an older version, though, that is not the case and you won't have such issues.
After version 4.0 the schedule tasks use requests so they will timeout. If you are using an older version, though, that is not the case and you won't have such issues.
Regards, Anton
Explains why this has been an issue since we upgraded directly from 3.9 til 4.1.
How do people using web apps and long running tasks get around this though? This has to be considered a big flaw in nopcommerce, dont you agree?
API methods to fetch stores\products in nopcommerce solution Azure Function to generate feed.
Details Ive created 3 endpoints in a nopcommerce plugin: /stores - get all stores /product/count/{storeId} - get total count of products in store /product/{storeid}/{take}/{pageIndex} - get X products from store with page index.
The function app runs every 6 hours, and it does the following:
Gets info of all stores Loops stores by x Gets productcount for x store Loops productcount/100 and fetches 100 products each get request until all products are fetched Generates xml doc based on product data Uploads xml to azure storage (blob)
So you sell the plugins or can you share the code? Thanks hkbits wrote:
So here is my solution to this issue:
API methods to fetch stores\products in nopcommerce solution Azure Function to generate feed.
Details Ive created 3 endpoints in a nopcommerce plugin: /stores - get all stores /product/count/{storeId} - get total count of products in store /product/{storeid}/{take}/{pageIndex} - get X products from store with page index.
The function app runs every 6 hours, and it does the following:
Gets info of all stores Loops stores by x Gets productcount for x store Loops productcount/100 and fetches 100 products each get request until all products are fetched Generates xml doc based on product data Uploads xml to azure storage (blob)
Question: Does Azure allow console apps to run? I ran into the same problem, but since we host on our own server, I was able to write a console app that generates multiple feeds (Google, Bing, etc) every so often (via Windows Scheduler) with thousands of products. The data is formatted from within a Stored Procedure, then assembled into whatever feed type is necessary (XML,TSV,PSV,CSV,etc) and saved to a folder called Feeds (off wwwroot). I've never seen any cart system that can handle this without weird timeouts, slowing the site down, etc. when there's a lot of products. I had a client with 92,000 products, a decade ago, on another cart system, and this was the only way to do without timeouts.
This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.
