Umbraco backoffice cache busting when UmbracoVersion changes

[mistyn8 (Mike Chambers)]

I’ve been noticing that on each deployment of umbracoVersion updates, having done 17.4.0, 17.4.1 and 17.4.2 deployments since friday last week.

That each deployment the backoffice workspaces are blank with console errors logging 404’s for manangement api endpoints.
A hard reset, or cache purge in the browser seems to fix things.

I had conicidentally seen cloudflare increase it’s cached files percentages, so thought that could be related (now have a cache rule to ignore the /umbraco/* urls..)

However, just observed it running locally when updating to uForms 17.3.2 as well as (umb 17.4.2) that first run the workspaces are blank…

opening developer tools and setting cache disabled on the network tab again sorted the issue.

It seems to suggest that the entrypoint js is cached, and so refereing to now none existent hashed manifests?

With smidge gone.. (which used to add a cachebusting querystring param based on the umbracoversion for the backoffice bundles) I know we now have lit compiling with a hash when it thinks files have changed (I believe based on a file contents hash) However I only see that for bundled manifests and associated files.. the entrypoint being set in the package-json doesn’t get the same treatment..

Are we missing a trick in the backoffice for core packages/ HQ extensions etc.. that also means we aren’t cache busting packages between versions?

eg the recommendation from Kevin Jumps Battle scared developers guide springs to mind..
Battle scarred developer’s guide to Umbraco v17 - Setup - DEV Community
(see below for code sample to add a cache busting version)

Before I start digging any deeper is anyone else seeing this? Or have workarounds?
(ps the CI/CD deployments include adding app-offline.htm file so the instances are forced to recycle, though seeing it locally after a stop and rebuild cycle suggests something cache wise is occuring)

public Task<IEnumerable<PackageManifest>> ReadPackageManifestsAsync()
{
  var version = Assembly.GetAssembly(typeof(DoStuffPackageManifestReader))?
   .GetName()
   .Version?.ToString() ?? "1.0.0";

  return Task.FromResult<IEnumerable<PackageManifest>>(new[]
  {
    new PackageManifest
    {
      Id = "DoStuff.Client",
      Name = "DoStuff with Umbraco client",
      AllowTelemetry = true,
      Version = version,
      Extensions = [
        new {
          name = "DoStuff Client Bundle",
          alias = "DoStuff.Client.Bundle",
          type = "bundle",
          js = "/App_Plugins/DoStuffClient/do-stuff-client.js?v=" + version
        }
     ]
   }
  });
}

[justin-nevitech (Justin Neville)]

Hi @mistyn8

It may be worth checking what Cache-Control headers are being returned for those assets and whether you are setting any Cache-Control headers that is also affecting the backoffice. I’ve not noticed this but will check next time I do an update.

Also, you may want to adjust your rule as the packages (i.e. Umbraco Forms) would be served form /App_Plugins not /umbraco, so you may also need a rule to exclude /App_Plugins/*.

Justin

[markusjoha (Markus Johansson)]

I’ve also noticed issues with client files during upgrades (especially extension entry points, bundles).

There is currently no built in way to automatically handle cache busters for extensions so it’s up to the package developer to ensure that a cache buster is added to entry points, bundles and import maps when shipping a new version.

I know I pointed out ideas around this, eg that the core would use the package version or something as a cache buster automatically. At the same time I saw that the core has mechanisms that should use a hash based on the installed version but it could be something wrong with that.

Are you saying that you have these problems also when updating ONLY the Umbraco version without touching any packages?

Did you notice which kind of files that where effected? Files for the “core backoffice” or any extension?

One could use a different cache-control but that would potentially slow down loading of the backoffice for editors - but might be a decent work around while investigating the underlying issue.

It very much sounds like something needs a cache buster if this is just minor updates of the CMS and no package changes I would definitely post a issue on GitHub for this because that “should” work

[jacob (Jacob Overgaard 🚀)]

@markusjoha

There is currently no built in way to automatically handle cache busters for extensions so it’s up to the package developer to ensure that a cache buster is added to entry points, bundles and import maps when shipping a new version.

You can add %CACHE_BUSTER% to the string to apply the generated version-hash:

{
  "name": "MyPackage"
  "extensions": [
    {
      "type": "backofficeEntryPoint",
      "alias": "my.entrypoint",
      "name": "My Entrypoint",
      "js": "/App_Plugins/MyExtension/entrypoint.js?v=%CACHE_BUSTER%"
  ]
}

Edit: Follow-up. Since you yourself suggested it in this issue, which got fixed for 17.3: Make `BackOfficeCacheBustHash` usable for packages · Issue #16893 · umbraco/Umbraco-CMS · GitHub, I am not sure if you were talking about something else when you said there is no built-in way to handle cache-busting?

[marcemarc (Marc Goodson)]

@jacob

cache-busting

“When there’s something the same… in the neighbourhood…”

[mistyn8 (Mike Chambers)]

I think I read that as in previous versions of umbraco we have the runtime minification settings that allowed the consumer to set cache busting requirements on the backoffice and all packages..

Cache buster

Specifies mechanism for cache invalidation.

The options are:

• Version - Caches will be busted when your assembly version changes, when the upstream Umbraco version changes and when the version string specified in Configuration changes.

• AppDomain - Caches will be busted when the app restarts.

• Timestamp - Caches will be busted based on a timestamp of the bundled files.

with v17 we’ve lost that overarching functionality, and have to rely on the package developer who didn’t have anything in core till v17.3 and I can’t see that the docs have been updated to reflect this which presumably should now be best practice?

%CACHE_BUSTER% looks great… does it follow the previous version implementation?

        // Assembly Name adds a bit of uniqueness across sites when version missing from config.
        // Adds a bit of security through obscurity that was asked for in standup.
        var prefix = _runtimeMinificationSettings.Value.Version ?? _entryAssemblyMetadata.Name;
        var umbracoVersion = _umbracoVersion.SemanticVersion.ToString();
        var downstreamVersion = _entryAssemblyMetadata.InformationalVersion;

        _cacheBusterValue = $"{prefix}_{umbracoVersion}_{downstreamVersion}".GenerateHash();

Can it also be affected by DEBUG, or maybe allow a timestamp\appdomain by setting again like the previous implementation?

Looks like it’s just the umbraco semantic version..

    public static string GetCacheBustHash(IHostingEnvironment hostingEnvironment, IUmbracoVersion umbracoVersion)
    {
        // make a hash of umbraco and client dependency version
        // in case the user bypasses the installer and just bumps the web.config or client dependency config

        // if in debug mode, always burst the cache
        if (hostingEnvironment.IsDebugMode)
        {
            return DateTime.Now.Ticks.ToString(CultureInfo.InvariantCulture).GenerateHash();
        }

        var version = umbracoVersion.SemanticVersion.ToSemanticString();
        return $"{version}".GenerateHash();
    }

Umbraco-CMS/src/Umbraco.Web.Common/Extensions/UrlHelperExtensions.cs at main · umbraco/Umbraco-CMS
[nice it still refers to client dependency version in the comment… ;-)]

[jacob (Jacob Overgaard 🚀)]

The value of %CACHE_BUSTER% depends on the version, as you found out. It’s the same value that we use for Backoffice assets in the /umbraco/backoffice/<hash> part of the URLs, only now made available directly in the umbraco-package.json files for developers to specify. I don’t recall which options were available to package developers before the new Backoffice - perhaps we appended a ?v=123 string to all files? Seems dangerous if we did. Does anyone remember?

[mistyn8 (Mike Chambers)]

So as a package developer it’s cache is only busted on an umbraco core version update when using this.. don’t we really need to include the package version too?

Sorry for all the questions.. just trying to understand what the current implementation is so I can either leverage that, or indeed find why my core workspaces are blank after a release deploy until browser caches are manually deleted.

On previous <14 smidge bundled all backoffice including packages so we had the choice (timestamp/apprecycle/assembly+umbracosemver).. but I guess you are asking post smidge and prior to v17.3? (I’ve only jumped LTS to LTS so don’t know on that one.)

[jacob (Jacob Overgaard 🚀)]

@mistyn8

So as a package developer it’s cache is only busted on an umbraco core version update when using this.. don’t we really need to include the package version too?

Yes, you are right. It doesn’t make much sense to use the Backoffice’s cache-busting in this case, if it’s just based on the version. If it was more dynamic, i.e., resetting after each deploy, then it would be a different case. Would be interesting to have some more configuration in this area, like we used to have with Smidge.
Your initial post, taking the assembly version and using that both for the “version” field as well as the “?v” parameter, is quite ideal for packages, at least.

@mistyn8

On previous <14 smidge bundled all backoffice including packages so we had the choice (timestamp/apprecycle/assembly+umbracosemver).. but I guess you are asking post smidge and prior to v17.3? (I’ve only jumped LTS to LTS so don’t know on that one.)

Do you know if Smidge also loaded and bundled the assets defined in the then package.manifest files? If I recall correctly, those were still loaded directly in the backoffice, but there may have been a way to append the cache-busting hash to those, too.

[mistyn8 (Mike Chambers)]

Yep.. Package.Manifests got bundled into

<script src="/sb/umbraco-backoffice-extensions-js.js.vea5882032811acf4ffc8475bfdacfb15411a154c" class="lazyload" charset="utf-8"></script>

I guess something more customisable or overridable would require some heavy lifting..
because extension methods are compile-time sugar, not DI-replaceable behavior…

public static class HtmlHelperBackOfficeExtensions

In the mean time I guess with
\Umbraco.Cms.StaticAssets\umbraco\UmbracoBackOffice\Index.cshtml being a RCL I could override that file and call a different
@await Html.BackOfficeImportMapScriptAsync(JsonSerializer, BackOfficePathGenerator, PackageManifestService, CspNonceService)

and play with

var importmapScript = sb.ToString()
    .Replace(backOfficePathGenerator.BackOfficeVirtualDirectory, backOfficePathGenerator.BackOfficeAssetsPath)
    .Replace(Constants.Web.CacheBusterToken, backOfficePathGenerator.BackOfficeCacheBustHash);

[jacob (Jacob Overgaard 🚀)]

@mistyn8

I guess at present with
\Umbraco.Cms.StaticAssets\umbraco\UmbracoBackOffice\Index.cshtml being a RCL I could override that file and call a different
@await Html.BackOfficeImportMapScriptAsync(JsonSerializer, BackOfficePathGenerator, PackageManifestService, CspNonceService)

and play with

var importmapScript = sb.ToString()
    .Replace(backOfficePathGenerator.BackOfficeVirtualDirectory, backOfficePathGenerator.BackOfficeAssetsPath)
    .Replace(Constants.Web.CacheBusterToken, backOfficePathGenerator.BackOfficeCacheBustHash);

That’s just for the <script type="importmap"> generation. Those also use cache-busters. But for extensions, they are being lazily loaded in the Backoffice directly after being served through the /extensions/private endpoint. There isn’t really any RCL you can overwrite to modify that behavior, unfortunately.

[mistyn8 (Mike Chambers)]

Oh.. probably need to look deeper then as that was the only place I could find for

.Replace(Constants.Web.CacheBusterToken, backOfficePathGenerator.BackOfficeCacheBustHash);

in use.

What about??

using System.Text.Json;
using Microsoft.AspNetCore.Mvc;
using Umbraco.Cms.Api.Management.Routing;
using Umbraco.Cms.Api.Management.ViewModels.Manifest;
using Umbraco.Cms.Core;

namespace Umbraco.Cms.Api.Management.Controllers.Manifest;

/// <summary>
/// Serves as the base controller for manifest operations in the management API.
/// </summary>
[VersionedApiBackOfficeRoute("manifest")]
[ApiExplorerSettings(GroupName = "Manifest")]
public abstract class ManifestControllerBase : ManagementApiControllerBase
{
    protected static void ReplaceCacheBusterTokens(
        IEnumerable<ManifestResponseModel> models, string cacheBustHash)
    {
        foreach (ManifestResponseModel model in models)
        {
            if (model.Extensions.Length == 0)
            {
                continue;
            }

            var json = JsonSerializer.Serialize(model.Extensions);
            if (json.Contains(Constants.Web.CacheBusterToken) is false)
            {
                continue;
            }

            json = json.Replace(Constants.Web.CacheBusterToken, JsonEncodedText.Encode(cacheBustHash).ToString());
            model.Extensions = JsonSerializer.Deserialize<object[]>(json) ?? model.Extensions;
        }
    }
}

and it’s use in (all|public|private)ManifestControllers?

[jacob (Jacob Overgaard 🚀)]

You got it, that’s the base method the manifest controller calls. The manifest controller is the one the backoffice calls to find its third-party extensions.

[mistyn8 (Mike Chambers)]

Just the hurdle that it relies on Constants.Web.CacheBusterToken for a find and replace…

[markusjoha (Markus Johansson)]

Thank you for clarifying @jacob!

I think I was more referring to that there is no “magic” thing that is automatically applied for updated packages.