Examine index file becomes corrupted and deleted

Questions
1

With Examine, there’s a site with over 2 million quotes. The index often becomes corrupted, and then it displays this error until the whole index is deleted from the file system and rebuilt.

{
    "@t": "2025-03-10T01:47:34.8002163Z",
    "@mt": "An error occurred during the index commit operation, if this error is persistent then index rebuilding is necessary",
    "@l": "Error",
    "@x": "System.InvalidOperationException: Failed to compare two elements in the array.\n ---> Lucene.Net.Util.LuceneSystemException: Could not find file '/var/www/vhosts/quotetab.com/httpdocs/umbraco/Data/TEMP/ExamineIndexes/QuotesTextsIndex/_se_37.del'.\n ---> System.IO.FileNotFoundException: Could not find file '/var/www/vhosts/quotetab.com/httpdocs/umbraco/Data/TEMP/ExamineIndexes/QuotesTextsIndex/_se_37.del'.\nFile name: '/var/www/vhosts/quotetab.com/httpdocs/umbraco/Data/TEMP/ExamineIndexes/QuotesTextsIndex/_se_37.del'\n   at System.IO.FileInfo.get_Length()\n   at Lucene.Net.Store.FSDirectory.FileLength(String name)\n   at Lucene.Net.Store.NRTCachingDirectory.FileLength(String name)\n   at Lucene.Net.Index.SegmentCommitInfo.GetSizeInBytes()\n   at Lucene.Net.Index.MergePolicy.Size(SegmentCommitInfo info)\n   at Lucene.Net.Index.TieredMergePolicy.SegmentByteSizeDescending.Compare(SegmentCommitInfo o1, SegmentCommitInfo o2)\n   --- End of inner exception stack trace ---\n   at Lucene.Net.Index.TieredMergePolicy.SegmentByteSizeDescending.Compare(SegmentCommitInfo o1, SegmentCommitInfo o2)\n   at System.Collections.Generic.ArraySortHelper`1.PickPivotAndPartition(Span`1 keys, Comparison`1 comparer)\n   at System.Collections.Generic.ArraySortHelper`1.IntroSort(Span`1 keys, Int32 depthLimit, Comparison`1 comparer)\n   at System.Collections.Generic.ArraySortHelper`1.Sort(Span`1 keys, IComparer`1 comparer)\n   --- End of inner exception stack trace ---\n   at System.Collections.Generic.ArraySortHelper`1.Sort(Span`1 keys, IComparer`1 comparer)\n   at System.Array.Sort[T](T[] array, Int32 index, Int32 length, IComparer`1 comparer)\n   at J2N.Collections.Generic.List`1.DoSort(Int32 index, Int32 count, IComparer`1 comparer)\n   at J2N.Collections.Generic.List`1.Sort(Int32 index, Int32 count, IComparer`1 comparer)\n   at J2N.Collections.Generic.List`1.Sort(IComparer`1 comparer)\n   at Lucene.Net.Index.TieredMergePolicy.FindMerges(MergeTrigger mergeTrigger, SegmentInfos infos)\n   at Lucene.Net.Index.IndexWriter.UpdatePendingMerges(MergeTrigger trigger, Int32 maxNumSegments)\n   at Lucene.Net.Index.IndexWriter.MaybeMerge(MergeTrigger trigger, Int32 maxNumSegments)\n   at Lucene.Net.Index.IndexWriter.PrepareCommitInternal()\n   at Lucene.Net.Index.IndexWriter.CommitInternal()\n   at Lucene.Net.Index.IndexWriter.Commit()\n   at Examine.Lucene.Providers.LuceneIndex.IndexCommiter.CommitNow()\n   at Examine.Lucene.Providers.LuceneIndex.IndexCommiter.TimerRelease()",
    "SourceContext": "Examine.BaseIndexProvider",
    "ProcessId": 32840,
    "ProcessName": "dotnet",
    "ThreadId": 133,
    "ApplicationId": "699b97a4ca6c75d445a6fc9e32437c3b9172c616",
    "MachineName": "quotetab",
    "Log4NetLevel": "ERROR"
}

I’ve been tracking this issue for weeks now, but I cannot pinpoint its origin because there are no related log entries. It seems to be working fine at first, and then it just breaks.

Anyone has any ideas or suggestions?

2

I absolutely hate Umbraco indexes. They cause way too many issues. We also sometimes have indexes just dying, or indexes being corrupt after a release. I think Umbraco should out of the box support at least one serious alternative, like ElasticSearch or Solr. There is a reason why Sitecore ditched Lucene for Solr some time ago.

Having said that, where are you hosting? If it’s Azure, make sure you set these settings correctly: Running Umbraco On Azure Web Apps | Umbraco CMS.

1 Like
3

Just going to point you here to join the discussion:

4

Thanks Sebastiaan, I know that rfc :slight_smile: Although I think it’s more about abstractions.

5

Hey Luuk, I’m hosting it in Linux, using Plesk as control panel.

For now, any other suggestions?

6

I’m having similar Lucene/Examine issues on a new Umbraco 13 site that impacts everything we are doing—obviously when the indexes don’t work then just about every feature of your site will fail too.

It’s led to so much frustration that I’m going through a small but serious refactor to move away from using custom indexes, and anything using the content delivery api.

Examine feels like a loose thread on a shirt and every time I pull on it I embarrassingly lose 10% more of my shirt.

7

For now it seems that Examine 3.7.0 fixes a lot of indexing issues

And when using the SyncedTempFileSystemDirectoryFactory for the Lucene directory setting, seleting the umbraco/data/temp/examineindexes folder seems to help as well.