Umbraco 17.5.3 hosted on Azure PaaS

srikanth-orbit · August 20, 2026, 2:54pm

getting this error every 5 seconds in Production - any help on resolving this error is greatly appreciated.

System.InvalidOperationException: This SqlTransaction has completed; it is no longer usable.
at Umbraco.Extensions.NPocoDatabaseExtensions.GetCurrentTransactionIsolationLevel(IDatabase database)
at System.Runtime.ExceptionServices.InternalCalls.<RhpSfiNext>g____PInvoke|1_0(StackFrameIterator* __pThis_native, UInt32* __uExCollideClauseIdx_native, Boolean* __fUnwoundReversePInvoke_native, Boolean* __fIsExceptionIntercepted_native)
at Umbraco.Cms.Infrastructure.Scoping.Scope.get_Database()
at Umbraco.Cms.Infrastructure.Persistence.Repositories.Implement.DistributedJobRepository.GetAll()
at Umbraco.Cms.Infrastructure.Services.Implement.DistributedJobService.TryTakeRunnableAsync()
at Umbraco.Cms.Infrastructure.BackgroundJobs.DistributedBackgroundJobHostedService.RunRunnableJob(CancellationToken stoppingToken)
at Umbraco.Cms.Infrastructure.BackgroundJobs.DistributedBackgroundJobHostedService.ExecuteAsync(CancellationToken stoppingToken)

justin-nevitech · August 20, 2026, 3:20pm

Hi @srikanth-orbit

Are you running in a load balanced setup and did this only happen after updating to a certain version?

Justin

srikanth-orbit · August 20, 2026, 3:36pm

Hi Justin, thank you for reaching out. No, this is not a load balanced setup - this error in Logs has started appearing only after we upgraded from Version 13 to Version 17 recently. thanks

justin-nevitech · August 20, 2026, 9:17pm

Hi @srikanth-orbit

Thanks for confirming.

That error comes from a background job runner that’s new in v17, so it wouldn’t have existed on 13. Looks like the upgrade didn’t fully apply to the database and it’s failing on the first poll after startup, then repeating every 5 seconds.

Can you check the Umbraco logs to see if you have any other errors, especially from when you migrated (if you still have them).

Also, can you run this SQL to see if you have the distributed jobs table:

SELECT name FROM sys.tables WHERE name LIKE 'umbraco%Job%'

You may also be able to change the logging for the Umbraco.Cms.Infrastructure.BackgroundJobs and Umbraco.Cms.Infrastructure.Scoping namespaces to Debug to see if you get any further information in the logs.

Justin

srikanth-orbit · August 21, 2026, 7:11am

Thank you for your detailed response. Yes, there’s a ‘umbracoDistributedJob’ table - the logviewer has only this particular error appearing every 5 seconds - no other error reported.

thanks

justin-nevitech · August 21, 2026, 7:23am

Hi @srikanth-orbit

If you change the log levels to Debug for these two namespaces, do you see anything else that would help diagnose the issue?

Umbraco.Cms.Infrastructure.BackgroundJobs

Umbraco.Cms.Infrastructure.Scoping

Justin

srikanth-orbit · August 21, 2026, 7:34am

Hi Justin, how can i selectively assign log level to ‘Debug’ for the 2 namespaces ? please clarify. thanks.

justin-nevitech · August 21, 2026, 7:38am

Hi @srikanth-orbit

In appSettings.json:

"Serilog": {
  "MinimumLevel": {
    "Default": "Information",
    "Override": {
      "Microsoft": "Warning",
      "System": "Warning",
      "Umbraco.Cms.Infrastructure.BackgroundJobs": "Debug",
      "Umbraco.Cms.Infrastructure.Scoping": "Debug"
    }
  }
}

Justin

srikanth-orbit · August 21, 2026, 8:55am

thanks, Justin. I just applied the ‘Debug’ setting for the namespaces. One of the errors i’m seeing in logs now is

System.InvalidOperationException: 

This SqlTransaction has completed; it is no longer usable.
   at Microsoft.Data.SqlClient.SqlTransaction.ZombieCheck()
   at StackExchange.Profiling.Data.ProfiledDbTransaction.get_IsolationLevel() in C:\projects\dotnet\src\MiniProfiler.Shared\Data\ProfiledDbTransaction.cs:line 36
   at Umbraco.Cms.Infrastructure.Scoping.Scope.get_Database()
   at Umbraco.Cms.Infrastructure.Persistence.Repositories.Implement.DistributedJobRepository.GetAll()
   at Umbraco.Cms.Infrastructure.Services.Implement.DistributedJobService.TryTakeRunnableAsync()
   at Umbraco.Cms.Infrastructure.BackgroundJobs.DistributedBackgroundJobHostedService.RunRunnableJob(CancellationToken stoppingToken)
   at Umbraco.Cms.Infrastructure.BackgroundJobs.DistributedBackgroundJobHostedService.ExecuteAsync(CancellationToken stoppingToken)

mistyn8 · August 21, 2026, 9:09am

@justin-nevitech

I’m not sure if that will work as intended…

Serilog’s MinimumLevel.Default acts as a global floor for event creation. An override can only raise the restriction (e.g., from Information up to Warning or Error to quiet things down). It cannot promote a namespace’s logging level lower than the default floor. Because your default is set to Information, Serilog drops Debug events before they are ever generated, meaning your Umbraco background job and scoping Debug overrides will be completely ignored.

@srikanth-orbit was that new log entry actually tagged as debug ?

I have the feeling you need…

"Serilog": {
  "MinimumLevel": {
    "Default": "Debug",
    "Override": {
      "Microsoft": "Warning",
      "System": "Warning",
      "Microsoft.AspNetCore": "Warning",
      "Umbraco": "Information",
      "Umbraco.Cms.Infrastructure.BackgroundJobs": "Debug",
      "Umbraco.Cms.Infrastructure.Scoping": "Debug"
    }
  }
}

but you might have to add other namespaces too back to Info/warning if the logs are still too chatty from other debug namespaces.

you also need to watch for minimumlevels set in the sinks as they might also stop output of debug at the sink level.

eg

  "Serilog": {
    "MinimumLevel": {
      "Default": "Information"
    },
    "WriteTo": [
      {
        "Name": "UmbracoFile",
        "Args": {
          "RestrictedToMinimumLevel": "Information"
        }
      }
    ]
  },

justin-nevitech · August 21, 2026, 9:23am

Thanks for clarifying @mistyn8 I missed that part.

@srikanth-orbit

Are you able to update to the latest version of v17?

There have been some fixes to isolation scopes since the release you are using which may solve this.

github.com/umbraco/Umbraco-CMS

Scoping: Isolate the ambient scope stacks across execution context flows (closes #23366) (#23492)

main ← Ron-Brouwer:v18/bugfix/23336-ambient-scope-stack-isolation

merged 01:55AM - 05 Aug 26 UTC

Ron-Brouwer

+319 -5

### Description All three ambient scope stacks share a `ConcurrentStack` inst…ance between unrelated execution flows, so concurrently-running hosted services can observe each other's ambient scope. When that happens `Scope.Dispose()` throws before it releases anything, stranding the transaction and whatever distributed lock it held until the process restarts. Related: #23366, which reports the same defect becoming reachable after #22331. #### Root cause `Push` does `(_stack.Value ??= new ConcurrentStack<IScope>()).Push(scope)`, and `Pop` never resets `_stack.Value` to `null` — popping the last entry leaves the instance in place. So the first scope used on any execution context leaves behind a **non-null but empty** stack. `AsyncLocal` copy-on-write protects the *reference*, not the instance it points at. Every flow that later branches off that context therefore inherits and mutates the **same** `ConcurrentStack`. Because a `ConcurrentStack` is globally LIFO rather than per-flow, pushes and pops from unrelated hosted services interleave. This is why the failure is timing-sensitive. If `_stack.Value` happens to be `null` when a service starts, that service allocates its own stack and behaves correctly; if something had already used a scope on that context — `DistributedJobService.EnsureJobsAsync` during start-up, for instance — it silently shares. #### Why a failed dispose strands the lock When the mismatch is detected, `Scope.Dispose()` throws *before* `Locks.ClearLocks(InstanceId)`, before `Locks.EnsureLocksCleared(InstanceId)`, and before `_scopeProvider.PopAmbientScope()`. So a failed dispose leaves the locks held, the transaction uncompleted, **and** the stack still corrupted for everything that follows. On SQL Server that strands the lock row; on SQLite it leaves an open write transaction. There is a quieter second symptom. `Scope.Database` fetches the *parent's* database when `ParentScope != null` and calls `GetCurrentTransactionIsolationLevel()`, which touches `Transaction.IsolationLevel`. A scope wrongly parented to a foreign scope whose transaction has since completed throws `InvalidOperationException: This SqlTransaction has completed`. Worth noting because such a scope also does not own its transaction, so its `scope.Complete()` becomes a no-op against the real one — a correctness exposure, not only a liveness one. #### All three stacks need the fix ``` src/Umbraco.Infrastructure/Scoping/AmbientScopeStack.cs src/Umbraco.Infrastructure/Scoping/AmbientScopeContextStack.cs src/Umbraco.Cms.Persistence.EFCore/Scoping/AmbientEFCoreScopeStack.cs ``` **A partial fix is worse than no fix.** We deployed a build with only `AmbientScopeStack` fixed to an affected production site and immediately got a new exception: ``` System.InvalidOperationException: No AmbientContext was found. at Umbraco.Cms.Infrastructure.Scoping.AmbientScopeContextStack.Pop() at Umbraco.Cms.Persistence.EFCore.Scoping.EFCoreScopeProvider`1.PopAmbientScopeContext() at Umbraco.Cms.Persistence.EFCore.Scoping.EFCoreScope`1.HandleScopeContext() at Umbraco.Cms.Persistence.EFCore.Scoping.EFCoreScope`1.Dispose() at ...AIUsageHourlyAggregationJob.ProcessMissingHoursAsync() at Umbraco.Cms.Infrastructure.HostedServices.RecurringHostedServiceBase.ExecuteAsync() ``` Whether a scope pushes a scope *context* depends on whether it sees an ambient scope. Before the fix, flows sharing a stack wrongly saw each other's scopes and became nested children, which do not push a context. Once the scope stack is isolated those scopes correctly become root scopes and start pushing contexts of their own — onto a context stack that is *still* shared between flows. They then pop each other's entries until one finds it empty and throws. The corruption moves one layer down rather than disappearing. #### The change Identical in all three: when the inherited stack is empty there is no genuine nesting to preserve, so allocating a fresh one lets `AsyncLocal` copy-on-write isolate the flow. A non-empty stack is real nesting within the same flow and is kept, so a child scope still nests under the current ambient one. ```csharp ConcurrentStack<IScope>? stack = _stack.Value; if (stack is null || stack.IsEmpty) { stack = new ConcurrentStack<IScope>(); _stack.Value = stack; } stack.Push(scope); ``` #### Why not restore `ExecutionContext.SuppressFlow()` instead #22331 removed `SuppressFlow` from `RecurringHostedServiceBase` and simultaneously added an equivalent to `RecurringBackgroundJobHostedService.StartAsync`. The protection was relocated rather than deleted, so the `IRecurringBackgroundJob` wrapper stayed safe while external subclasses of the public `RecurringHostedServiceBase` lost protection they had in 17.4. Restoring it there would close that regression, but it is not sufficient: - `DistributedBackgroundJobHostedService` has **never** had `SuppressFlow` in any version, including current `main`, and it participates in most of the traces we collected. - `AmbientScopeStack.cs` is byte-identical across `release-17.4.2`, `release-17.5.3` and `main`, so the defect is version-invariant rather than a regression. Fixing the stacks covers every caller at once, including third-party hosted services, instead of requiring each entry point to remember to suppress flow. It is also the only option available to affected sites, since `IAmbientScopeStack` and `AmbientScopeContextStack` are `internal` — the DI registration cannot be overridden from a consumer project, so there is no supported workaround. #### Affected versions Confirmed in production on 17.5.2 and 17.5.3, and present unchanged on `main`. This PR targets `main` per the contributing guide, but it cherry-picks cleanly to the v17 branches: - `AmbientScopeStack.cs` and `AmbientScopeContextStack.cs` are byte-identical between `main` and `v17/dev`. - `AmbientEFCoreScopeStack.cs` differs only by the `IEfCoreScope` → `IEFCoreScope` rename, so the v17 variant needs the lower-case spelling in both the fix and the new test. ### How to test Run the new tests: ```bash dotnet test tests/Umbraco.Tests.UnitTests/Umbraco.Tests.UnitTests.csproj --filter "FullyQualifiedName~AmbientScope" dotnet test tests/Umbraco.Tests.Integration/Umbraco.Tests.Integration.csproj --filter "FullyQualifiedName~AmbientEFCoreScopeStackTests" ``` To confirm the tests actually catch the defect, revert the three `Push` methods to `_stack.Value ??= new ConcurrentStack<…>()` and re-run. Each stack has one isolation test that fails and one nesting guard that passes either way, so the guards cannot mask an over-correction: | Run | Without the fix | With the fix | |---|---|---| | Unit, `~AmbientScope` | Failed 2, Passed 2 | Passed 4 | | Integration, EF Core stack | Failed 1, Passed 1 | Passed 2 | Each isolation test reproduces the defect deterministically: it leaves a non-null empty stack on the execution context, starts two flows from it, has both push and wait for each other, then asserts each flow still sees its own entry as ambient. Wider suites on this branch: **6,393 unit tests** and **106 scoping integration tests** passing, 0 failures (6 and 3 skipped respectively, unrelated). For a real-world reproduction, run two hosted services concurrently — `DistributedBackgroundJobHostedService` plus any `RecurringHostedServiceBase` subclass that opens a scope — on a site where a scope was used during start-up. Before the fix, one of them eventually throws `"The Scope … is not the Ambient Scope …"` from `Scope.Dispose()`, after which every server polling the `-347` DistributedJobs lock times out every few seconds indefinitely, because the poll interval and the write-lock timeout are both 5 seconds. Fixes #23336

The latest version is currently 17.6.2.

Justin

srikanth-orbit · August 21, 2026, 9:39am

thanks both. i’ll try to upgrade to latest version and get back to you, thanks

srikanth-orbit · August 24, 2026, 7:33am

The upgrade from 17.5.3 to 17.6.2 has resolved the Isolation scope error and the other errors related to Distributed Background jobs - CMS errors have reduced drastically. thanks

justin-nevitech · August 24, 2026, 7:37am

Hi @srikanth-orbit

That’s great to hear! Thanks for letting us know.

Justin