Performance optimization needed - NOPCOMMERCE IS TOO SLOW !

A lot of performance optimization changesets have been added to codeplex for the last several days. And there are a lot of new chnagesets are coming.

Today I added changeset 51cf96ce8d4d related to memory consumption . Can anybody please apply it and test it out on your hosting? Please let me know if memory consumption is dropped (of course, in case if you experienced any memory issues)

Hi Andrei,

Regarding this changeset:

https://nopcommerce.codeplex.com/SourceControl/changeset/92bd1dd9cf28

Can we have the embeded view infrastructure in place so that it can still be used by the plugin vendors.
I can see that the files for it are deleted already and this is going to create a lot of problems for us.

Thanks

Nop-Templates.com wrote:
Hi Milen,

After playing with some performance profilers I've found out that these classes made a lot of extra work. Now it's just not required. You have two ways to go:

1. Modify the plugins to support new paths. For example, see how it was done in the built-in "Facebook shop" plugin.

2. Just add this embedded view infrastructure to SevenSpikes.Core. Then register EmbeddedViewVirtualPathProvider there on application start like it was previously done in Global.asax

If it does not help please describe in details what exactly issues you experience. We'll find a way to resolve it

Hi Andrei,

Thanks for the answer!
I have not looked at it in detail but I guessed it was performance related.
If it could only be left as an option for the admin views it would be perfect.
We would try to do this in the SevenSpikes.Core as we do not want the core to affect the performance of the public part in anyway.
I will let you know if we encounter any issues.

Best Regards

I'm hacking CatalogController to the max. I have split out the product page into its own separate controller, as I felt the CatalogController is a bit too monolithic. Before I re-wrote the Controller, I had refactored the Manufacturer() and Category() actions to use a lot more shared code by extracting methods that were common to both actions. Not to mention custom catalog filtering, and displaying product counts to the side of the page etc.

This is very custom product page stuff and it was easier to just practically do the whole thing from scratch to match our specific needs. Otherwise we'd have these viewmodels with so many properties that weren't being used etc and lots of database calls to fetch unused data.

Then I added the ability to browse by Manufacturer AND Category and things got all kinds of interesting :) (a useful feature too, by the way)
However, it was really hard to integrate this with site. I got it working so that you could navigate products using URLs in the format of /manufacturer/category, dynamically resolving them, e.g:
Ford/Cars
Ford/Parts
Fiat/Cars
Fiat/Accessories

The way I'm looking at it, is that Manufacturer, Category and ManufacturerCategory (my own thing) are essentially the same view, with different filters applied. So my CatalogController reflects that, and is about 200 lines of code. Its constructor takes in about 4 services to perform and receives a model that is parsed by the Route and contains all relevant domain objects about the requested URL.

Just a thought into how to possibly break things down, although it's tricky inside nopcommerce's core website as it caters to so many needs, whereas we have pretty specific use case. I've monitored DB calls with Glimpse and have gone from over 100 calls to display 10 products per page, to 40 database calls to display 30 products. The database has 1200 manufacturers, 3000 categories and 65000 products (15,000 grouped and 40,000 simple linked to the grouped ones) and all the grouped products are linked to their categories.

Page response time is about 600ms for showing 30 products, with most of the database calls on the page coming from fetching each lazy loading each product's manufacturer (we also changed the Product has many Manufacturer relationship to Product has a Manufacturer). Fetching the manufacturers is fast, at about 1ms each call but I feel I can eagerly load them all, struggling how though. I've started my own custom 'SearchProduct' implementation which is giving me filtered products in around 50ms vs 250ms for the SP version.

....Phew ...that was some brain dump there. Sorry if it's all irrelevant baffle, I'm just trying to show it is possible to get fast page loads with a decent amount of data using nop's architecture (if slightly tailored to my needs). I'm trying to make it even faster too.

sproaty wrote:
Looks promising. Is it possible to share your implementation?

I have another question / idea concerning website loading process :

When I restart the website on my development computer, it's very much faster than on production. On production, memory is growing to 2Go for the pool, then is reduced to 1Go when the site is on.

So here is my suggestion : if many users are accessing the website while it's loading, just after the compilation process, each request find nothing in the cache, and each request try to load all the settings, localized values etc... at the same time. Only one will be stored in cache but they are all loaded in parallel. That seems to be the reason why it's so long on production, even longer than in development.

Is this suggestion possible?

Regards,
Nicolas

PS : I'm sorry I was not able to test you changeset concerning memory because time is missing...

nicolas.muniere wrote:
Not sure that it's the real reason. Anyway, how? Locking?


nicolas.muniere wrote:
There were about 20 commits related to performance for the last several days. If you're going to test it, then please try the latest version 3.40 from codeplex

a.m. wrote:

Sorry Andrei, it's too business-specific for nop, as we're not using all product information, and have changed some domain objects so that they map to our company's existing structure (e.g. a Product has one Manufacturer).

sorry :(


We've got pages rendering in about 250ms, displaying 90 products per page, with about 15 database calls to actually render the page (with an empty shopping cart)

a.m. wrote:

I don't really know what would be the best way to solve this, but something is sure, if nothing is done to avoid each thread to load the same data to initialize the same cache, we are doing a lot of work for nothing!

Perhaps we could lock. It's better to wait results from another process than to wait (probably longer) for the same useless and duplicate result! This is the way ASP already manage compilation process, the first hit launch compilation, other hits are waiting.

To test the efficiency of locking, it could be simple to manage a global and simple lock, not on each resource, it would alow the fisrt request to work alone. We could use the event beginhttprequest (or something like that), a global boolean (ApplicationInitialized) could remember if the application is initialized... it's just an idea.