MainDom Lock Timeout on Azure App Service - Looking for Solution

Questions
1

Environment:

  • Umbraco CMS 15.4.3

  • .NET 9 on Windows (Azure App Service)

  • Azure App Service: 2 separate App Service Plans

    • Backoffice: SchedulingPublisher (minimal traffic)

    • Public: Subscriber (~100k requests/day, traffic doubled recently)

    • One instance each, no scaling out

  • Configuration: FileSystemMainDomLock with LocalTempStorageLocation = EnvironmentTemp

Problem:

Subscriber server crashes on restart when under load with:

System.TimeoutException: Cannot acquire MainDom
at Umbraco.Cms.Core.Runtime.MainDom.Acquire()

What we observe:

  • Backoffice server (SchedulingPublisher): Always acquires MainDom successfully

  • Subscriber server: Times out after exactly 40 seconds when restarting under traffic

  • Workaround: If we switch all traffic away from Subscriber before restart, it acquires MainDom successfully

Example from logs:

05:55:19 - Acquiring MainDom.
05:55:59 - Cannot acquire MainDom (timeout).
05:55:59 - Unhandled exception in AppDomain (terminating).

Configuration (per Umbraco docs):

Umbraco:CMS:Global:MainDomLock = FileSystemMainDomLock
Umbraco:CMS:Hosting:LocalTempStorageLocation = EnvironmentTemp

What we need:

A solution that works for all restart scenarios (deployments, IIS auto-recycle, manual restarts, platform maintenance) - not just deployments.

Has anyone solved this on Azure App Service?

Docs: https://docs.umbraco.com/umbraco-cms/fundamentals/setup/server-setup/load-balancing/azure-web-apps

2

Have you tried turning off overlapped recycling with the following environment variable in your Azure Web App?

WEBSITE_DISABLE_OVERLAPPED_RECYCLING=1
3

Thank you! WEBSITE_DISABLE_OVERLAPPED_RECYCLING=1 got us past the MainDom timeout issue.

We’re still seeing the server not come online on its own after restart (requires manual intervention), but that appears to be a separate issue we’re investigating.

Appreciate the help!

1 Like
4

@JasonElkin would you say that this is a good setting to always have when hosting in web apps in Azure, regardless of if you are using load balancing?

5

Not in general, no.

IME it’s only necessary if this problem is encountered.

It really shouldn’t happen. When Overlapped Recycling is enabled two different VMs will temporarily run the site at the same time. So it’s a bit like you’re “load balancing”.

But in that scenario, so long as you are using FileSystemMainDomLock and EnvironmentTemp that shouldn’t matter - we should be talking about two separate machines with two separate sets of temp files so I really don’t know what’s going on.

I’ll give a shoutout to @Jeavon at this point, as he found out that during slot swaps temp files can kinda persist. IDK if the same sort of thing is happening here.

I feel like the proper way to address this is to have an extra nerdy look at the FileSystemMainDomLock and AspNetCoreHostingEnvironment.LocalTempPath and see if there’s something that can be done.

Frankly, I’m not sure why we have IMainDom and require a lock at all…

AFAIK in .NET5+ you can’t even have multiple App Domains running…