Help is needed Error: Timeout expired.The timeout period elapsed prior to obtaining a connection from the pool.

Hi,

I have a nopCommerce 3.20 which has been working for years without any problem. Recently my dedicated server died. Fortunately, I have backups and moved to a new server and strange things begin: Application starts without any problem, then after a while an error occurs:

An unhandled exception occurred and the process was terminated.

Application ID: /LM/W3SVC/1/ROOT

Process ID: 4408

Exception: System.Data.Entity.Core.EntityException

Message: The underlying provider failed on Open.

StackTrace:    at System.Data.Entity.Core.EntityClient.EntityConnection.Open()
   at System.Data.Entity.Core.Objects.ObjectContext.EnsureConnection()
   at System.Data.Entity.Core.Objects.ObjectContext.ExecuteInTransaction[T](Func`1 func, IDbExecutionStrategy executionStrategy, Boolean startLocalTransaction, Boolean releaseConnectionOnSuccess)
   at System.Data.Entity.Core.Objects.ObjectQuery`1.<>c__DisplayClassb.<GetResults>b__9()
   at System.Data.Entity.SqlServer.DefaultSqlExecutionStrategy.Execute[TResult](Func`1 operation)
   at System.Data.Entity.Core.Objects.ObjectQuery`1.GetResults(Nullable`1 forMergeOption)
   at System.Data.Entity.Core.Objects.ObjectQuery`1.<System.Collections.Generic.IEnumerable<T>.GetEnumerator>b__0()
   at System.Lazy`1.CreateValue()
   at System.Lazy`1.LazyInitValue()
   at System.Lazy`1.get_Value()
   at System.Data.Entity.Internal.LazyEnumerator`1.MoveNext()
   at System.Linq.Enumerable.FirstOrDefault[TSource](IEnumerable`1 source)
   at System.Data.Entity.Core.Objects.ELinq.ObjectQueryProvider.<GetElementFunction>b__1[TResult](IEnumerable`1 sequence)
   at System.Data.Entity.Core.Objects.ELinq.ObjectQueryProvider.ExecuteSingle[TResult](IEnumerable`1 query, Expression queryRoot)
   at System.Data.Entity.Core.Objects.ELinq.ObjectQueryProvider.System.Linq.IQueryProvider.Execute[TResult](Expression expression)
   at System.Data.Entity.Internal.Linq.DbQueryProvider.Execute[TResult](Expression expression)
   at System.Linq.Queryable.FirstOrDefault[TSource](IQueryable`1 source)
   at Nop.Services.Tasks.ScheduleTaskService.GetTaskByType(String type)
   at Nop.Services.Tasks.Task.Execute(Boolean throwException)
   at Nop.Services.Tasks.TaskThread.Run()
   at Nop.Services.Tasks.TaskThread.TimerHandler(Object state)
   at System.Threading.TimerQueueTimer.CallCallbackInContext(Object state)
   at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
   at System.Threading.TimerQueueTimer.CallCallback()
   at System.Threading.TimerQueueTimer.Fire()
   at System.Threading.TimerQueue.FireNextTimers()
   at System.Threading.TimerQueue.AppDomainTimerCallback()

InnerException: System.InvalidOperationException

Message: Timeout expired.  The timeout period elapsed prior to obtaining a connection from the pool.  This may have occurred because all pooled connections were in use and max pool size was reached.

StackTrace:    at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection)
   at System.Data.ProviderBase.DbConnectionInternal.TryOpenConnectionInternal(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)
   at System.Data.ProviderBase.DbConnectionClosed.TryOpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)
   at System.Data.SqlClient.SqlConnection.TryOpenInner(TaskCompletionSource`1 retry)
   at System.Data.SqlClient.SqlConnection.TryOpen(TaskCompletionSource`1 retry)
   at System.Data.SqlClient.SqlConnection.Open()
   at System.Data.Entity.SqlServer.DefaultSqlExecutionStrategy.<>c__DisplayClass1.<Execute>b__0()
   at System.Data.Entity.SqlServer.DefaultSqlExecutionStrategy.Execute[TResult](Func`1 operation)
   at System.Data.Entity.SqlServer.DefaultSqlExecutionStrategy.Execute(Action operation)
   at System.Data.Entity.Core.EntityClient.EntityConnection.Open()

Then a lot of similar errors occur:

    Exception type: InvalidOperationException 
    Exception message: Timeout expired.  The timeout period elapsed prior to obtaining a connection from the pool.  This may have occurred because all pooled connections were in use and max pool size was reached.
   at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal oldConnection, DbConnectionInternal& connection) ......

and finally w3wp.exe is crashed:

Faulting application name: w3wp.exe, version: 10.0.14393.0, time stamp: 0x57899135
Faulting module name: KERNELBASE.dll, version: 10.0.14393.1198, time stamp: 0x5902845a
Exception code: 0xe0434352
Fault offset: 0x000da9f2
Faulting process id: 0x1138
Faulting application start time: 0x01d31b61b34acff6
Faulting application path: C:\Windows\SysWOW64\inetsrv\w3wp.exe
Faulting module path: C:\Windows\System32\KERNELBASE.dll
Report Id: 4d48a969-7578-4563-8a52-4ccafd285d35
Faulting package full name: 
Faulting package-relative application ID: 

And it happens all over again and again.

OS: Windows Server 2016 Datacenter 6.3 <X64> (Build 14393: ) (Hypervisor)
SQL: Microsoft SQL Server 2016 (SP1) (KB3182545) - 13.0.4001.0 (X64)   Oct 28 2016 18:17:30   Copyright (c) Microsoft Corporation  Web Edition (64-bit) on Windows Server 2016 Datacenter 6.3 <X64> (Build 14393: ) (Hypervisor)

Have anyone faced with the similar problem? Any help would be highly appreciated!

Are you able to access data from your website at all? If not, I'd suggest ensuring you have correct db access permissions and correct presentation/nop.web/app_data/settings.txt configuration.

AdamK wrote:

Yes, everything is working well at the beginning and after error occurs everything start working slowly because app poll tries to resolve connection to SQL server and could not. Then app pool crashes and restarted. This dramatically slowing down user experience and user start to get custom error. Any other thoughts? Could it be somehow related to a new SQL server 2016? Does anybody have it running on SQL 2016?

I had a very similar problem once. For me, it was a temporary situation.

We had two dedicated servers in California. One for the websites and one for the SQL server. They were adjacent to one another in a server center. We migrated our websites to another dedicated server located in Arizona a couple of weeks before we migrated our SQL server to the same new server center in Arizona..during that 2 week period, we were having this same issue occur fairly regularly...everything pointed to the fact that there was too much potential latency and packet loss between the two servers, and once we migrated our database to a new/nearby server, the issues ceased.

I don't know if that helps you any, but that was my issue.

Steve