We are experiencing an issue with Umbraco across all of our domains.
Some pages that were previously published and working correctly suddenly appear to no longer be published. When users try to access them, they are redirected to a 404 page.
If we go into Umbraco and publish the affected page again, it starts working normally.
This seems to be happening intermittently, and we haven’t identified what is causing the pages to become unpublished.
Has anyone experienced something similar, or can suggest where we should look to identify the cause?
Could this be related to scheduled publishing, the Examine/indexes, cache, database issues, deployments, or something else?
Any advice on how to troubleshoot this would be greatly appreciated.
Yes, this is a rather unfortunate thing that you are not alone in seeing.
I have not been able to figure anything out as to a direct cause but have definitely seen a number of threads that are all tip-toeing around URL issues that are not immediately obvious as related though, I believe, are related to some shared core hiccup.
I have not recreated or seen the issue yet this week so, perhaps, it was resolved with 17.6.2? Or, perhaps that is just good fortune so for. I’m definitely on the lookout for causes/patterns and will share back if anything useful presents..
Thank you so much for taking the time to share this — this is incredibly helpful!
The examples you provided sound very similar to what we’re experiencing, especially the cases where the content is still published but suddenly becomes unroutable or loses its URL. It’s reassuring to know that we’re not the only ones seeing this behaviour.
I will try the upgrage to 17.6.2 and i hope that move will give the solution!
If we spot any patterns or find a root cause on our side, I’ll share them here too.
Really appreciate you sharing your experience and the additional context — it gives us some very useful directions to investigate. Thanks again!
I know you posted this about a week ago, but thought I would reply with my own experience of the same. I don’t have an issue with the URLs being routable when the page is first published, but it seems to be for pages that aren’t touched for long periods of time. It mostly happens to my clients that make use of Cloudflare. I think the amount of pages held in memory becomes too much, and somehow that data becomes “stale”.
Once a user clears the Cloudflare cache, and the server is hit directly, is when the pages start to give issues. I’m not sure if it is the pages being in memory, and then too many other pages get loaded, or if the Lucene index needs to be rebuilt. If you log into the BackOffice and republish the website, the issue goes away, usually for quite a while (unless the user is clearing the Cloudflare cache often, some of our users don’t realize that they should only do this if page content changes, and only the current page should be cleared rather than the entire website).
I can say that the issue occurs far less frequently with v17 websites than v13 and prior versions. It’s not always clear, as sometimes there is an error in the logs about the Lucene index, but usually there is nothing in the logs pointing to the cause.
Thank you very much for taking the time to share your experience and for such a detailed reply — I really appreciate it!
What you’re describing actually makes a lot of sense and definitely gives me another angle to investigate. My case is slightly different, though, as the issue seems to occur with recently created/published pages rather than older pages that haven’t been touched for a long time.
The part about Cloudflare and the cache potentially being involved is particularly interesting, and I’ll definitely keep that in mind while investigating. I’m also wondering whether there could be some kind of caching or indexing issue behind it, especially since the problem seems to resolve itself after republishing the website.
In the meantime, I’ve updated the site to the latest Umbraco v17(17.6.2) version as pc-pdx mentioned above, hoping that perhaps the issue has already been addressed there. I deployed it to production a couple of days ago, so I’ve been keeping a close eye on it to see if the problem comes back.I hope that the update will do the trick, but if it happens again I’ll have a much better idea of where to look thanks to your experience.
Thanks again for taking the time to reply and of course i will inform here if the solution is given by the update or it is something else that needs more investigation.
Bad news guys. We’ve also updated Umbraco to the latest version, but unfortunately the issue still persists.
We are continuing to see previously published pages intermittently become unavailable and return a 404. Re-publishing the affected page from the backoffice immediately resolves the issue, but the problem can occur again later.
At this point, it doesn’t seem to be related to the specific content or an isolated publishing action, as we are seeing the behaviour across multiple domains/pages.
I was wondering if it is related to the Cloudflare purge but i cannot understand the reason if any.
We’re currently investigating whether this could be related to URL routing, the published snapshot/cache, Examine/indexes, scheduled publishing, database state, or potentially something happening during deployments.
Any additional troubleshooting steps, logs, or areas we should check would be greatly appreciated. Or any ideas about the solution of that issue?
Instead of “guys”, may we suggest, for example: “folks", “Umbracians”, “all”, or “everyone”? We use gender inclusive language in this forum (read more)
Hi @giannisMarv13 what environment are you using? Are you running in Azure? Redis Cache? Load balancer? If its running on Azure in a single webapp that’s load balance, that could explain it
For the Umbraco Backoffice, we have a single pod / single Umbraco instance running. We are also using Redis Cache.
The pod restarts obviously when we deploy a new version, but we haven’t observed unexpected/repeated restarts outside of deployments.
The important part of the issue is that the affected pages are initially published and working correctly, but intermittently they later become unpublished in the Umbraco Backoffice itself. Once that happens, the frontend returns a 404. If we publish the page again manually, it immediately starts working.
So I’m not sure whether the typical multiple-instance/load-balancing scenario would apply to us, given that we only have one Umbraco pod. However, since we are using Redis, perhaps there could still be something related to caching, MainDom, NuCache, or the publishing process.
I’m also looking at the changes between 17.6.2 and 17.8.0, especially the publishing/cache/MainDom-related fixes, to see if any of them could be related.
Would you recommend checking anything specific on those cases?
Have you got a version of 17.6.2? If so I would try that and see if that solves the issue. If it does it could be Umbraco and not Kubernetes
Are you hitting Kubernetes resource limits. It does feel like, everything is working but then something is having a hiccup and not quite completing.
ChatGTP suggest this which I think is a good idea
1. The NuCache Desync on Pod Restarts
The root cause is usually a NuCache (the internal memory/database cache) desynchronization that triggers when a Kubernetes pod restarts. [1]
The Cause: If your pod experiences minor node maintenance, an idle downscale, or restarts due to a memory limit breach, Umbraco boots up and attempts to reconstruct its routing cache. Bug reports for recent minor upgrades of Umbraco 17 show that during a startup/maintenance restart, the cache initialization can drop the links/URLs for randomly selected nodes (often pages updated or scheduled since the last major upgrade). [1]
How to fix:
Ensure your pod has a Persistent Volume Claim (PVC) mapped to /umbraco/Data/ so that the local NuCache files (NuCache.Content.db) survive pod restarts rather than forcing a broken cold-rebuild every time the container spins up.
Review K8s Restarts: Run kubectl get pods. Look at the RESTARTS column. If that number is greater than 0, your unpublishing events line up exactly with whenever Kubernetes kills and brings back the pod.
(