Replacing the Default Preview Option in Umbraco 18

We hit the same thing. The fix is in the manifest, not in the URL provider registration. Your previewOption is added alongside the built-in one, so the built-in one (umbDocumentUrlProvider) still runs and throws the 400. Tell your option to replace the default with overwrites:

{
  "type": "workspaceActionMenuItem",
  "kind": "previewOption",
  "alias": "Custom.Extensions.CustomUrl.PreviewOption",
  "name": "Custom URL Preview Option",
  "forWorkspaceActions": "Umb.WorkspaceAction.Document.SaveAndPreview",
  "overwrites": "Umb.Document.WorkspaceActionMenuItem.SaveAndPreview",
  "weight": 100,
  "meta": {
    "icon": "icon-eye",
    "label": "#buttons_saveAndPreview",
    "urlProviderAlias": "customUrlProvider"
  }
}

You can leave the default URL provider registered. Just keep builder.AddUrlProvider<CustomUrlProvider>().

If you use Umbraco Workflow: its in-workflow preview button only respects overwrites since Workflow 17.4.0 / 18.1.0, so update if you’re on an older version.