Well, @warren came to the rescue with this solution.
Add to program.cs
var customContentTypes = new FileExtensionContentTypeProvider();
customContentTypes.Mappings[".vtt"] = "text/vtt";
services.PostConfigure<StaticFileOptions>(opt =>
{
opt.ContentTypeProvider = customContentTypes;
});