Hi Jason,
I’d prefer not to disclose the exact version here for security reasons, but happy to explain the setup more generally or discuss privately if needed.
All of our websites are experiencing a high number of recycles due to ConfigurationChange
, with shutdown logs like this:
Application shutdown. Details: "ConfigurationChange"
Application shutdown. Details: “ConfigurationChange” _shutDownMessage=“CONFIG change HostingEnvironment initiated shutdown CONFIG change CONFIG change HostingEnvironment caused shutdown” _shutDownStack=" at System.Environment.GetStackTrace(Exception e, Boolean needFileInfo) at System.Environment.get_StackTrace() at System.Web.Hosting.HostingEnvironment.InitiateShutdownInternal() at System.Web.Hosting.HostingEnvironment.InitiateShutdownWithoutDemand() at System.Web.HttpRuntime.ShutdownAppDomain(String stackTrace) at System.Web.Configuration.HttpConfigurationSystem.OnConfigurationChanged(Object sender, InternalConfigEventArgs e) at System.Configuration.Internal.InternalConfigRoot.OnConfigChanged(InternalConfigEventArgs e) at System.Configuration.BaseConfigurationRecord.OnStreamChanged(String streamname) at System.Web.Configuration.WebConfigurationHostFileChange.OnFileChanged(Object sender, FileChangeEvent e) at System.Web.DirectoryMonitor.FireNotifications() at System.Web.Util.WorkItem.CallCallbackWithAssert(WorkItemCallback callback) at System.Web.Util.WorkItem.OnQueueUserWorkItemCompletion(Object state) at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context(Object state) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem() at System.Threading.ThreadPoolWorkQueue.Dispatch() at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback()"
In some cases, we’ve seen sites recycle over 20 times a day, often with multiple recycles in quick succession. A particularly critical issue occurs when the AppDomain is recycled just 18–20 seconds after claiming MainDom, which leaves Umbraco unresponsive and requires a manual App Service restart.
Our websites use SqlMainDomLock
, and I suspect there’s an unresolved race condition that’s triggered by these closely timed recycles. I briefly reviewed the latest Umbraco version and it looks like this issue might still be present.
Here’s a diagram that outlines the race condition, based on a real incident where Umbraco was left unresponsive:
Setting fcnMode="Disabled"
immediately stops the ConfigurationChange-related recycles, which confirms that File Change Notifications (FCN) are the trigger. I then isolated the source: moving the /media
folder back into wwwroot
and removing the virtual directory (while using fcnMode="Single"
) eliminated the recycles entirely.
My working theory is that hidden metadata updates or background processes on the Azure File Share are being interpreted as file changes, which then trigger FCNs.
For reference, Shazwazza shared years ago that FCN and Virtual Directories (especially pointing to network shares) don’t mix well:
As it turns out, Virtual Directories in IIS are treated differently as well! The only FCN setting that affects Virtual Directories is “Disabled” (…) and as you can see, if it’s not disabled, it will be creating a directory watcher for every sub directory in the Virtual Directory. This problem is compounded if the Virtual Directory is pointing to a network share.
Thanks again for engaging with this — happy to go deeper on any part of the setup.