Never mind, I found the culprit. I had implemented an IUmbracoBuilder extension method called AddBackgroundJobs() to inject my custom background jobs. I call this method inside an IComposer, but apparently Umbraco.Cms.Infrastructure.DependencyInjection.UmbracoBuilderExtensions also has a method with that same name, and I was accidentally using that method instead of my own. So this caused the duplicate injection of the WebhookFiring job. 
I previously hadn’t noticed that my background jobs weren’t triggered at all, because they were disabled (by an app setting) on the development and staging environments. So the background jobs not running actually was expected behaviour.