Panic, exception while loading cache data

Questions
, ,
1

Hi All:

TL:DR;

is there a way to check for data integrity in v8 schema and up? The DB schema is valid after migration, but there is something IN the data itself that kicking an edge case and very weird bug in the NuCache service.

The story:

I am working on a migration from V7 to V13 from a few months now. It’s been a bumpy but funny road and when we finally made it to create a migration checklist for DB, migrated views and custom code, I am having trouble migrating the production database.

The staging test DB (which really is a snapshot from production DB about 4 years ago) was migrated successfully and the final migrated V13 version works beautifully in staging. But when it is time to migrate production DB I cannot get past V8 because of this error during v8 first launch:

{
	"@t": "2026-03-08T18:02:27.2086443Z",
	"@mt": "Panic, exception while loading cache data.",
	"@l": "Fatal",
	"@x": "System.ArgumentOutOfRangeException: Specified argument was out of the range of valid values. Parameter name: handle
   at CSharpTest.Net.IO.TransactedCompoundFile.Read(UInt32 handle)
   at CSharpTest.Net.Storage.BTreeFileStoreV2.TryGetNode[TNode](IStorageHandle handleIn, TNode& node, ISerializer`1 serializer)
   at CSharpTest.Net.Collections.BPlusTree`2.StorageCache.FetchFromStore`1.CreateValue(IStorageHandle key, Object& value)
   at CSharpTest.Net.Collections.LurchTable`2.InternalInsert[T](Int32 hash, TKey key, Int32& added, T& value)
   at CSharpTest.Net.Collections.LurchTable`2.Insert[T](TKey key, T& value)
   at CSharpTest.Net.Collections.LurchTable`2.AddOrUpdate[T](TKey key, T& createOrUpdateValue)
   at CSharpTest.Net.Collections.BPlusTree`2.StorageCache.TryGetNode[TNode](IStorageHandle handle, TNode& tnode, ISerializer`1 serializer)
   at CSharpTest.Net.Collections.BPlusTree`2.NodeCacheNone.Lock(NodePin parent, LockType ltype, NodeHandle child)
   at CSharpTest.Net.Collections.BPlusTree`2.Enumerator.SeekNext(NodePin thisLock, TKey key, NodePin& pin, Int32& offset, TKey& nextKey, Boolean& hasMore)
   at CSharpTest.Net.Collections.BPlusTree`2.Enumerator.MoveNext()
   at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
   at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
   at System.Linq.OrderedEnumerable`1.<GetEnumerator>d__1.MoveNext()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Umbraco.Web.PublishedCache.NuCache.PublishedSnapshotService.LoadEntitiesFromLocalDbLocked(Boolean onStartup, BPlusTree`2 localDb, ContentStore store, String entityType)
   at Umbraco.Web.PublishedCache.NuCache.PublishedSnapshotService.LoadContentFromLocalDbLocked(Boolean onStartup)
   at Umbraco.Web.PublishedCache.NuCache.PublishedSnapshotService.LockAndLoadContent(Func`2 action)
   at Umbraco.Web.PublishedCache.NuCache.PublishedSnapshotService.<EnsureCaches>b__36_0()
   at System.Threading.LazyInitializer.EnsureInitializedCore[T](T& target, Boolean& initialized, Object& syncLock, Func`1 valueFactory)
   at Umbraco.Web.PublishedCache.NuCache.PublishedSnapshotService.EnsureCaches()
   at Umbraco.Web.PublishedCache.NuCache.PublishedSnapshotService.Notify(JsonPayload[] payloads, Boolean& draftChanged, Boolean& publishedChanged)
   at Umbraco.Web.Cache.ContentCacheRefresher.NotifyPublishedSnapshotService(IPublishedSnapshotService service, AppCaches appCaches, JsonPayload[] payloads)
   at Umbraco.Web.Cache.ContentCacheRefresher.Refresh(JsonPayload[] payloads)
   at Umbraco.Core.Sync.ServerMessengerBase.Deliver[TPayload](ICacheRefresher refresher, TPayload[] payload)
   at Umbraco.Web.Cache.DistributedCache.RefreshByPayload[TPayload](Guid refresherGuid, TPayload[] payload)
   at Umbraco.Web.Cache.DistributedCacheExtensions.RefreshAllContentCache(DistributedCache dc)
   at Umbraco.Web.Cache.DistributedCacheExtensions.RefreshAllPublishedSnapshot(DistributedCache dc)
   at Umbraco.Core.Migrations.MigrationPlan.Execute(IScope scope, String fromState, IMigrationBuilder migrationBuilder, ILogger logger)
   at Umbraco.Core.Migrations.Upgrade.Upgrader.Execute(IScopeProvider scopeProvider, IMigrationBuilder migrationBuilder, IKeyValueService keyValueService, ILogger logger)
   at Umbraco.Core.Migrations.Install.DatabaseBuilder.UpgradeSchemaAndData(MigrationPlan plan)",
	"SourceContext": "Umbraco.Web.PublishedCache.NuCache.PublishedSnapshotService",
	"ProcessId": 33980,
	"ProcessName": "w3wp",
	"ThreadId": 84,
	"AppDomainId": 4,
	"AppDomainAppId": "LMW3SVC16ROOT",
	"MachineName": "ASUSPONTI",
	"Log4NetLevel": "FATAL",
	"HttpRequestNumber": 7,
	"HttpRequestId": "95a2b584-0d34-4490-ade9-5f4cf1a09797"
}

And, as you can imagine right now, the migration does not finishes, rollsback and back to square one, I mean, v7.

At some point, I don’t know exactly what I did, I managed to get past v8 and got to v10, but even then it failed miserably with this same exact error.

So, obviously, there is something weird with the data itself that is making NuCache lose its mind, but I don’t know how to pin-point it. And I am sure that it is the data instead of the structure because it says (in the log) that this is a post-migration task and it really starts to care about the data when populating the cache (I think).

Then, I need help creating/finding/figuring out a way or tool to check the validity or correctness of the content of each field of every content node in order to find the culprit and take action.

For the time being, it is being hosted locally with IIS and SQL Server 19 during the migration tasks.

Any ideas would be really appreciated.

Thanks in advance,
Daniel.

2

Hi Daniel,

This is a known NuCache issue others have hit the exact same stack trace during migration. A few things to try:

1. Find the problematic node (https://our.umbraco.com/forum/using-umbraco-and-getting-started/109590-boot-failure-nucache):

sql

SELECT TOP 10 nodeId, LEN(data) as dataLength 
FROM cmsContentNu 
ORDER BY dataLength DESC

If any row is 1M+ characters, clear that entry and delete App_Data/TEMP/NuCache/ before retrying.

2. If that doesn’t fix it (https://our.umbraco.com/forum/using-umbraco-and-getting-started/109590-boot-failure-nucache), also check umbracoDocument / umbracoNode for broken records — someone hit the same error where cmsContentNu wasn’t the issue and tracked it down to 5 specific nodes there.

3. To get past v8 temporarily (https://github.com/umbraco/Umbraco-CMS/issues/12103), disable local DB entirely to force loading from the database instead:

xml

<NuCache IgnoreLocalDb="true" />

The reason staging worked but production didn’t is almost certainly a specific content node added in the last 4 years.

Note: This answer was created with the assistance of AI and is mostly correct please verify against the linked references.

EDIT: Daniel himself came back with the actual root cause and a great solution worth highlighting:

3

Thanks, @BishalTimalsina12

Although your suggestions are for v8, and the thing was that it cannot finish the upgrade to v8, you did gave me the idea about what to look for. And I thank you for that.

The problem were about 5 nodes with content bigger than 1mb because those were filled with an RTE that somehow allowed to insert images in data format, so those posts had images that made the node way bigger than it should. In fact, what was exploding was not the NuCache population, but its loading afterwards. All I did was to remove the images in those nodes and then everything worked like a charm, I was able to finish the upgrade to v13.

Daniel.

ps. The solution you mention that I did was to another problem, although I used almost the same steps to find the culprit.