Serious IIS Application Pool recycling bug/design flaw

angeor wrote:

Then how you will handle the third-party plugin without source plugin.

Any news on the final fix? I have just deployed 4.2 onto Azure and same issues. Is there going to be any resolution to this bug?

I experienced the same issue after updating to v4.20 in Azure.

First check that you app is set to 64 Bit. This seemed to be the resolution after testing for the last month. The platform is stable.
Configuration > General Settings > Platform Settings > Platform > 64 Bit

Additionally removing everything from the production environment and publishing from VS after you have merged you nop versions is important. You are probably already doing that, but just in case...

We have a NOP 4.1 site deployed to multiple environments in Azure (as App Services). The sites seem to go down randomly and there is nothing useful in the stdout log files to help us track down the issue. However, the symptoms are very similar to what was described by OP.

In azure there is no way to recycle the app pool, only start and stop the app service. I have tried triggering a recycle by simply making a small change to the web.config file. The site restarts but it always seems to come up with no issues (albeit after about 5 mins).

Has anyone experienced this in Azure App services?

Here are some settings as they are currently are in our appsettings.json file:

"RedisCachingEnabled": true,
"RedisCachingConnectionString": "HIDDEN",

//Do not edit this element. For advanced users only
"DisableSampleDataDuringInstallation": false,
"UseFastInstallationService": false,
"PluginsIgnoredDuringInstallation": "",

//Indicates whether to ignore startup tasks
"IgnoreStartupTasks": false,

//Enable if you want to clear /Plugins/bin directory on application startup
"ClearPluginShadowDirectoryOnStartup": true,
//Enable if you want to copy "locked" assemblies from /Plugins/bin directory to temporary subdirectories on application startup
"CopyLockedPluginAssembilesToSubdirectoriesOnStartup": false,
//Enable if you want to copy plugins library to the /Plugins/bin directory on application startup
"UsePluginsShadowCopy": true,
//Enable if you want to load an assembly into the load-from context, by passing some security checks
"UseUnsafeLoadAssembly": true,

//Enable for backwards compatibility with SQL Server 2008 and SQL Server 2008R2
"UseRowNumberForPaging": false,

Pay attention to this parameter 'UsePluginsShadowCopy'
And read this topic.
I did custom build.

tekguy82 wrote:

tekguy82:
What is the error?
What do you see in Azure when you go to Diagnose and solve problems > Availability and Performance > View Full Report?
Your 500 errors should provide some indication as to which is the failing module.

I am experiencing the same issue in Azure. The problem is that even though I have

        "ClearPluginShadowDirectoryOnStartup": true,

set to true, the plugins/bin folder continues to hold a copy of the .dlls.  The exception I different each time depending on which .dll it has the issue with, but here is an example:

An error occurred while starting the application.
FileLoadException: Could not load file or assembly 'D:\home\site\wwwroot\Plugins\bin\Nop.Plugin.Payments.AuthorizeNet.dll'. Access is denied.

So I decided to dig further into the code to find out exactly where this is happening, which is a bit tough because I can't reproduce the issue regularly. It happens about 25% of the time the application restarts.  The best I can figure is that the sequence in the code is:

ApplicationPartManagerExtensions.InitializePlugins - this is where it all starts.

At line 419, it checks the ClearPluginShadowDirectoryOnStartup value and, if it is true, attempts to delete each file in the directory. If it fails, it just gobbles up the exception.

A little further down, it runs PerformFileDeploy for each of the plugin assemblies.   Once there, it checks to see if it should be using the shadow copies or not. I have that set to false, so it steps into

AddApplicationParts

at line 193.

Here is what I don't understand.  Its failing at this on line 160 within the AddApplicationParts method:

                    assembly = Assembly.UnsafeLoadFrom(assemblyFile);

And the exception I get shows that it is trying to load the DLL from the Plugins/bin folder --- which is the very folder it was supposed to have deleted the files from earlier in the code.  So if I have use shadow copies set to false, why is it trying to load a file from that folder?

Hey Guys,

we're facing this issue since we recently migrated from NopCommerce v3.8 to v4.2. The store is running as Azure App Service and we are desperately searching a solution.

The web app goes down approximately once a day - whenever the gods of Azure recycle the AppPool - and does not recover without restarting the Azure App Service manually.

Here's what we found in our Application Insights logs. Maybe this helps and "Sharing is caring"...

The message:

Plugin 'Seven Spikes Core'. Could not load file or assembly 'D:\home\site\wwwroot\Plugins\bin\SevenSpikes.Nop.Plugins.Core.dll'. Access is denied.
Could not load file or assembly 'D:\home\site\wwwroot\Plugins\bin\SevenSpikes.Nop.Plugins.Core.dll'. Access is denied.
 Plugin 'Seven Spikes Core'. Could not load file or assembly 'D:\home\site\wwwroot\Plugins\bin\SevenSpikes.Nop.Plugins.Core.dll'. Access is denied. Could not load file or assembly 'D:\home\site\wwwroot\Plugins\bin\SevenSpikes.Nop.Plugins.Core.dll'. Access is denied. 

The stacktrace:

System.Exception:
   at Nop.Web.Framework.Infrastructure.Extensions.ApplicationPartManagerExtensions.InitializePlugins (Nop.Web.Framework, Version=4.2.0.0, Culture=neutral, PublicKeyToken=nullNop.Web.Framework, Version=4.2.0.0, Culture=neutral, PublicKeyToken=null: C:\Users\devusr\nopCommerce_42\src\Presentation\Nop.Web.Framework\Infrastructure\Extensions\ApplicationPartManagerExtensions.csNop.Web.Framework, Version=4.2.0.0, Culture=neutral, PublicKeyToken=null: 579)
   at Nop.Web.Framework.Infrastructure.Extensions.ServiceCollectionExtensions.ConfigureApplicationServices (Nop.Web.Framework, Version=4.2.0.0, Culture=neutral, PublicKeyToken=nullNop.Web.Framework, Version=4.2.0.0, Culture=neutral, PublicKeyToken=null: C:\Users\devusr\nopCommerce_42\src\Presentation\Nop.Web.Framework\Infrastructure\Extensions\ServiceCollectionExtensions.csNop.Web.Framework, Version=4.2.0.0, Culture=neutral, PublicKeyToken=null: 75)
   at Nop.Web.Startup.ConfigureServices (Nop.Web, Version=4.2.0.0, Culture=neutral, PublicKeyToken=nullNop.Web, Version=4.2.0.0, Culture=neutral, PublicKeyToken=null: C:\Users\devusr\nopCommerce_42\src\Presentation\Nop.Web\Startup.csNop.Web, Version=4.2.0.0, Culture=neutral, PublicKeyToken=null: 43)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Microsoft.AspNetCore.Hosting.ConventionBasedStartup.ConfigureServices (Microsoft.AspNetCore.Hosting, Version=2.2.7.0, Culture=neutral, PublicKeyToken=adb9793829ddae60)
   at Microsoft.AspNetCore.Hosting.Internal.WebHost.EnsureApplicationServices (Microsoft.AspNetCore.Hosting, Version=2.2.7.0, Culture=neutral, PublicKeyToken=adb9793829ddae60)
   at Microsoft.AspNetCore.Hosting.Internal.WebHost.Initialize (Microsoft.AspNetCore.Hosting, Version=2.2.7.0, Culture=neutral, PublicKeyToken=adb9793829ddae60)
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Microsoft.AspNetCore.Hosting.Internal.WebHost.BuildApplication (Microsoft.AspNetCore.Hosting, Version=2.2.7.0, Culture=neutral, PublicKeyToken=adb9793829ddae60)
Interne Ausnahme "System.Exception" behandelt bei "Nop.Web.Framework.Infrastructure.Extensions.ApplicationPartManagerExtensions.InitializePlugins":
   at Nop.Web.Framework.Infrastructure.Extensions.ApplicationPartManagerExtensions.InitializePlugins (Nop.Web.Framework, Version=4.2.0.0, Culture=neutral, PublicKeyToken=nullNop.Web.Framework, Version=4.2.0.0, Culture=neutral, PublicKeyToken=null: C:\Users\devusr\nopCommerce_42\src\Presentation\Nop.Web.Framework\Infrastructure\Extensions\ApplicationPartManagerExtensions.csNop.Web.Framework, Version=4.2.0.0, Culture=neutral, PublicKeyToken=null: 568)
Interne Ausnahme "System.IO.FileLoadException" behandelt bei "Nop.Web.Framework.Infrastructure.Extensions.ApplicationPartManagerExtensions.InitializePlugins":
   at System.Runtime.Loader.AssemblyLoadContext.LoadFromPath (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Runtime.Loader.AssemblyLoadContext.LoadFromAssemblyPath (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Reflection.Assembly.LoadFrom (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at System.Reflection.Assembly.UnsafeLoadFrom (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
   at Nop.Web.Framework.Infrastructure.Extensions.ApplicationPartManagerExtensions.AddApplicationParts (Nop.Web.Framework, Version=4.2.0.0, Culture=neutral, PublicKeyToken=nullNop.Web.Framework, Version=4.2.0.0, Culture=neutral, PublicKeyToken=null: C:\Users\devusr\nopCommerce_42\src\Presentation\Nop.Web.Framework\Infrastructure\Extensions\ApplicationPartManagerExtensions.csNop.Web.Framework, Version=4.2.0.0, Culture=neutral, PublicKeyToken=null: 160)
   at Nop.Web.Framework.Infrastructure.Extensions.ApplicationPartManagerExtensions.PerformFileDeploy (Nop.Web.Framework, Version=4.2.0.0, Culture=neutral, PublicKeyToken=nullNop.Web.Framework, Version=4.2.0.0, Culture=neutral, PublicKeyToken=null: C:\Users\devusr\nopCommerce_42\src\Presentation\Nop.Web.Framework\Infrastructure\Extensions\ApplicationPartManagerExtensions.csNop.Web.Framework, Version=4.2.0.0, Culture=neutral, PublicKeyToken=null: 208)
   at Nop.Web.Framework.Infrastructure.Extensions.ApplicationPartManagerExtensions.InitializePlugins (Nop.Web.Framework, Version=4.2.0.0, Culture=neutral, PublicKeyToken=nullNop.Web.Framework, Version=4.2.0.0, Culture=neutral, PublicKeyToken=null: C:\Users\devusr\nopCommerce_42\src\Presentation\Nop.Web.Framework\Infrastructure\Extensions\ApplicationPartManagerExtensions.csNop.Web.Framework, Version=4.2.0.0, Culture=neutral, PublicKeyToken=null: 532)

Our current appsettings.json

...
    "ClearPluginShadowDirectoryOnStartup": true,
    "CopyLockedPluginAssembilesToSubdirectoriesOnStartup": false,
    "UsePluginsShadowCopy": true,
    "UseUnsafeLoadAssembly": true,
...

Guys, from my opinion this is serious and the bug seems to be quite old...

Read topic about Azure.
You will find solution.

foxnetsoft wrote:

Sorry, can't seem to find the topic. Where is it?