Limits on concurrent API requests for Umbraco 13.5.3

Questions
1

Hi,

We’re using Umbraco 13.5.3 in our web application, in which we’re exposing an endpoint through a Surface Controller. We noticed and verified, that receiving 50 or more concurrent requests through this API (even though the API does not do anything and returns right away) these requests bring the site down and make it unloadable.

Is this a known issue in Umbraco 13? If so, is there a patch that we can apply?
Are there any known limits on concurrent requests for Umbraco 13?
Is there a specific configuration that can help improve performance under load?
What troubleshooting steps can be taken to further investigate this issue?

Any suggestions or advice to tackle this issue is very appreciated.

Thank you,
Sam

2

I don’t know if it makes a difference, but you can start to update Umbraco to 13.8.1. That will also fix a number of vulnerabilities.

3

As Luuk notes, you should be on the very latest version at least. This is not a known bug though. I would suspect your server setup is probably not able to handle the load.

  • Is this a known issue in Umbraco 13? If so, is there a patch that we can apply?
    • Not a known issue, but upgrade anyway
  • Are there any known limits on concurrent requests for Umbraco 13?
    • It completely depends on how you set up your infrastructure
  • Is there a specific configuration that can help improve performance under load?
    • Nothing specific, you need to load test, identify bottle necks and tweak accordingly, for both Umbraco and general .NET apps - best practices documentation for Umbraco could help: Good practice and defaults | Umbraco CMS
  • What troubleshooting steps can be taken to further investigate this issue?
    • See previous answer
4

Thank you Luuk and Sebastiaan for your replies. We’ll try your suggestions and will post an update later.

5

I have updated to umbraco 13.8.1 and that did not solve the issues.

We tested the limitation on our production servers, then we created a separate VM that had the site/db on the same machine. This still did not resolve the issue.

Here is the code we use to create members and login. Anything that touches the database from umbraco’s side seems to be a chokepoint and causing large amount of deadlocks.

All ADGUmbracoMember.Init is doing is setting properties on the model.

public IADGUmbracoMember CreateMember(string username, string email, string name, string type)
{
IMember? member = _IMemberService.CreateMemberWithIdentity(username, email, name, type, true);

 if (member != null)
 {
     member.CreateDate = TimeZoneHelper.CurrentTimeZone(_AppSettings.UmbracoConnection);
 }
 return ADGUmbracoMember.Init(member, _IMemberService, _IMemberManager);

}

Here is the login code. We use PasswordSignInAsync. We also have to update LastLoginDate and perform a .Save because there is a bug in Umbraco that will not update the LastLoginDate when logging in.

public bool Login(IADGUmbracoMember member, string password)
{
if (member != null)
{
var signInStatus = _IMemberSignInManager.PasswordSignInAsync(member.Username, password, isPersistent: true, lockoutOnFailure: true).Result;
if (signInStatus.Succeeded)
{
SetLastActive();

         member.LastLoginDate = TimeZoneHelper.CurrentTimeZone(_AppSettings.UmbracoConnection);
         member.Save();
         return true;
     }
     else
     {
         return false;
     }
 }
 else
 {
     return false;
 }

}

6

May not solve the problem, but I’d suggest using async patterns throughout the call chain. If you’re calling this on an API endpoint you can start the async chain there and work up.

7

There are some performance issues with member logins: Slow performance and deadlocks in SignInAsync during high load (v.10–13–?) · Issue #13768 · umbraco/Umbraco-CMS

8

Does anyone know if this issue (SignIn deadlock) is solved with Umbraco 15? Or should we expect the same issue after a potential upgrade from 13 to 15?

9

Just an update. We updated to make it more barebones and made it asynchronous as well.

Here is the create member code now. It hit this immediately and creates the member/logs them in. This is causing massive deadlocks on the database still when attempting with even 100 concurrent logins.

private async Task<IdentityResult> CreateMemberAsync(string netId, string firstName, string lastName, string email, string integratedPassword, IEnumerable<FeatureSettingModel> memberProperties, IEnumerable<FeatureSettingModel> memberGroups, MemberTypes memberType)
{
    using ICoreScope scope = _coreScopeProvider.CreateCoreScope(autoComplete: true);

    var identityUser = MemberIdentityUser.CreateNew(netId, email, memberType.ToString(), true, firstName);
    IdentityResult identityResult = await _memberManager.CreateAsync(identityUser,integratedPassword);

    if (identityResult.Succeeded)
    {
        IMember? member = _memberService.GetByKey(identityUser.Key);
        if (member == null)
        {
            throw new InvalidOperationException($"Could not find a member with key: {member?.Key}.");
        }

        _memberService.Save(member);
        await _memberSignInManager.SignInAsync(identityUser, false);
    }

    return identityResult;
}
10

I have applied the 13.9 updates and this did not fix the issue. It’s still deadlocking constantly when a large amount of accounts are attempted to be created at once. The site is unusable when these accounts are being created, it will not load. Below is a small glimpse at what our logs look like when I sent 100 requests to make accounts around the same time. This new update is actually bringing (on local) new unhandled errors, before it was not showing these errors in debug mode.

image
image1920×502 269 KB