Reducing the memory usage on startup

Hi guys,

First to give you some background information before going into more details (if you don't want to read it all you can simply jump to the bottom).
We have had complains from our customers that their nopCommerce websites restart from time to time.
All these customers turn out to be using a shared hosting and the reasons for the restarts is that their website application pools hit their memory limit, which on a shared hosting is usually between 300-500 MB with some shared hosting offering up to 800 MB.
Since we run more than 30 nopCommerce websites ourselves on a single server and it is also important for us to use as little memory as possible and we already know the fact that changing the application pool to run as a 32 bit application decreases the memory usage twice. Actually all our 30+ theme demos run as 32 bit applications without any issues and consume very little memory around 200-300 MB.
Actually most of the hosting companies are already aware of this improvement and should have already done this in advance. But even after ensuring that the website is running as 32 bit application they still hit the memory limits from time to time.

Why is this happening when the average memory consumption is around up to 300 MB (when running as 32 bit application)?

The simple answer is that there are cases when there is a really high memory consumption that causes the memory to go far beyond the 800 MB memory limit. If you are like us running your websites without any memory limits, you won't even notice that because once the Garbage Collector is triggered it will remove any unused memory and your website memory consumption will be back to normal. But if you are using a shared hosting and have a memory limit on your application pool you will definitely know when this happens as your website will be restarted.

So I decided to do some memory profiling to see why this really happens.
Since the customers complained that the memory limit is hit after installing our plugins I run two nopCommerce websites - a brand new nopCommerce 3.80 (no source code version) with none of our plugins installed and another one with all our plugins intalled.
Both sites performed almost the same with a slight difference around 15 MB, which means that these memory spikes are not directly caused by any of our plugins. So the problem seems to be somewhere else and not directly caused by a specific plugin.

Why do these customers experience these issues after installing the plugins when there are no any memory consumption differences with and without the plugins?

The only thing that happens when installing a new plugin is that nopCommerce is being restarted after the plugin is installed.
So I went in the administration and hit the Restart Application button, which is the same that happens after a plugin is being installed and then I noticed that the memory goes up and it starts growing faster if you try to access the website while it is being restarted.
I notice the memory goes from around 200 MB to up to 700MB and if you keep doing this you might end up with more than 1 GB (see the end of the video).
Since all out themes come with a least 10 plugins, then this means that our clients will have their website restarted at least 10 times during the installation of the theme and then the probability that they will hit the memory limit is quite high.
I recorded a video that easily illustrates this with a nopCommerce running without any plugins and simply restarting it a few times and making requests to the website while it is being restarted.

So this already proved to me that the issue with hitting a high memory usage is not caused by any of our plugins since as shown in the video we don't have any plugins (instead of the default ones) installed. But this still does not change the fact that shared hosting users may experience frequent restarts of their websites after installing several plugins one after another.

Why nopCommerce uses so much memory after a restart?

I decided to get to the bottom of this and downloaded the trial version of dotMemory and snapshot the memory usage during the peak time memory consumption (you need to be very quick as the GC may reduce the memory at any point).
It turned out the there are 500 MB ... yes 500 MB in Strings allocated in the memory heap.
When investigated who allocated so much stings in the memory it turned out that it is the constructor of the BrowscapXmlHelper class.
You can easily spot why this happens as there is a huge 40 MB file App_Data\browscap.xml that is being loaded in a String into the memory. Then there are several Replace methods that actually create new strings rather than modifying the existing one.

By simply commenting the the Initialize method invocation in the constructor of the BrowscapXmlHelper, then I couldn't make my nopCommerce to take more than 200 MB even after several restarts.

So the implementation of the BrowscapXmlHelper class needs to be corrected to ensure all nopCommerce users will run smoothly their nopCommerce websites on a shared hosting.

Hope this help!

Thanks,
Boyko

Hi Boyko,

Thanks a lot for investigation and so detailed report. We'll investigate it and get back to you soon

Thank you for the detailed analysis and a full description of this issue. I did the code analysis  and made all the necessary changes, you can view them here. If you ll have time please test the changes and write then wheter all problems are solved.

Sergei-k wrote:

Hi Sergei,

Thank you for looking at this!

At first glance I see in the commit that the 40 MB file will still be loaded in memory but I will give it a try with the profiler and will let you know.

My opinion on this matter is that as far as I can see you are using this 40 MB browscap.xml file for determining if the current agent is a search engine or not.  After a quick search in this file I can see that there are 3558 nodes that are for search engines

<item name="Crawler" value="true"/>

and the rest 58366 are not really needed. If you remove the not needed 58366 records the 40 MB file will become 18 times smaller and will be around 2 MB only. This way you don't really need to worry about the implementation details and nopCommerce will take far more less memory.

I have implemented the feature to create and use a stripped-down version of the user agents file. Implementation details here. Thank you for the idea.

Hi there,

I just pulled in the latest changeset and back-ported this to 3.8.  I did notice though that the slimmed down xml file is not in the App_Data folder, just the full 40 MB version.  I see the git line that shows it being added in src/Presentation/Nop.Web/Nop.Web.csproj but the physical file seems to be missing.
Can you add browscap.crawlersonly.xml file to the solution?

Thanks
Chuck

Small update, I see the file is created and deployed during a publish. I was thinking it was a supplied file, not a generated one. All good on this file now. Looks to be about 450 Kb

I have just tested this in a custom 3.8 environment. Results are impressive.

Bit of background, On average, my app pools were running anywhere from 600 to 900 MB (running in 64 bit mode for app pools).  I had flipped this to 32 bit mode and saw a good reduction.  Averages were now floating around 400 to 500 MB for a worker process.

Now, after implementing these change-sets, ...hard to believe, but I am running my app pool under 200 MB now!
And I am an ultimate customer of Nop-Templates, so I have all their plugins running during this test and about 300 products and 50 categories in the database.

Conclusion: Great job to both the Nop team and the Nop-Templates team for chasing this down. This is by far the most significant performance update to memory I have seen in a very long time.

Thanks
Chuck

Is there a quick way or patch that existing 3.80 nops can implement this. I use a no source code version of nop 3.80.

Would really appreciate if someone can guide me on how to implement this patch.

Its not neccessary fot you to make any changes in the code for reducing the burden on the system, you can just take the already trimmed UserAgents file (saving the name browscap.xml and its structure)

Sergei-k wrote:
Maybe you could provide a link to the already trimmed version of browscap.xml since it's generated dynamically there's no way for non-technical users to get it without compiling the development version of nop from GitHub.