Website crash when traffic is little high

[Menachem35 (Menachem Glik)]

I’m hosting my Umbraco 15 website on Azure. My plan is Standard S1.

When I’m sending a newsletter, or just traffic increase (because of social media posts, etc.), Instance count jump to 8, and yet, the website is crashing! Is it make sense?? It cost $73 / month and can’t handle some traffic???
(In a shared host I’d get a better performance …)

So now when I’m sending newsletter I change the plan to Premium v3 P0V3, then it stable. Problem is that the Premium v3 P0V3 cost $127 / month …

And here are my questions:
Is it Umbraco / my website or that Azure is just a bad service?
Like, is there something I can do to make the website more efficient ?

Like, it’s a simple magazine website, the code is not that complicated and I don’t have plugins / packages, etc. A standard code

Any ideas?

Thanks

[skttl (Søren Kottal)]

It’s probably your site, sorry

Read through Common Pitfalls & Anti-Patterns | Umbraco CMS it covers a lot of things you can do wrong.

[sotirisf (Sotiris Filippidis)]

It doesn’t make sense. Where I’d start would be the Umbraco logs. I would search for any exceptions or out of memory errors that could be an indication.

What Soren also suggested makes sense. Maybe, unwillingly, you’re doing one of the things mentioned there that contribute to your website being overloaded. It could be something as simple as getting a huge collection in memory every time you load a list without using caching, or making a content query that just returns too much. These things tend to create issues only on higher traffic scenarios.

[Menachem35 (Menachem Glik)]

Hi, thanks.
Which logs? At Umbraco back office or Azure kudu ?

Also, what should I look for in the logs?

How can I know what causes the issue ?

My code is pretty simple - at home page I take the latest news items from each category, then click on each of them takes to the article, which I get the document’s information, etc. Pretty simple. I looked at the link which Soren put and didn’t find something from there in my code

This for example a typical section at the home page (in the link it’s still the V11 code because haven’t pushed yet to my github the V15, but very similar - my current code just with the changes to 15):

Partial%20View%20Macro%20Files/HP_Photonics_Lasers.cshtml

Is anything wrong in this code?

And this for example the code to display posts for each category:

Partial%20View%20Macro%20Files/Archive_Magazine.cshtml

Thanks

GitHub - Menachem35/Opli-Umbraco-Magazine-Template at opli-Umbraco-11

[cheeseytoastie (Steve Morgan)]

I’ve had a very quick look at your repo. Nothing immediately jumps out but the following things will help:

1. Review your use of the ‘@’ - this is just to put Razor variables into markup - within c# blocks it’s possibly causing some weirdness.
   Lines like this below

var photonics_pagesToList = @photonics_node.Children.Where("Visible");

: the @photonics is unneeded - just photonics_node.Children

2. You’re mixing the use of partials and macros (in how you call these partials from your page layout master template and on the pages. I think you can just use partial views based on what I can see. Then you can cache too… (see next point)
   Partial Views | Umbraco CMS

3. Cache - for every page load your making this site traverse all these nodes - things like navs and side bars with links are not changing so use caching. You can control if the cache is the same site wide or different for each page (e.g. the main nav might be the same site wide if you don’t have any current page context) and the side bar related links is likely per page.

e.g. replace

@await Umbraco.RenderMacroAsync("ArticleImgHandler")

with

@await Html.CachedPartialAsync("ArticleImgHandler", Model, new TimeSpan(0, 1, 0, 0), true, false)

// the first true is cache by page - for a site wide you can have false.

But definitely look at the logs - check in the Settings section in Umbraco and look at what happens at the time of the crashes. My guess is in the use of macros when you don’t need them and your accessing of Umbraco media.

[Menachem35 (Menachem Glik)]

Thanks.
I refactored some of the code but still happening.

For example, this what the Azure logs look like when instance count jumps to 8 and website either very slow or not coming up

Is it make sense?

[cheeseytoastie (Steve Morgan)]

What do the logs say?

When you say instance count are you scaling the instance? If so have you followed the documentation on how to scale and load balance Umbraco on Azure?

What’s your traffic levels? A single instance on Azure that’s correctly sized won’t need multiple instances unless you’re handling big load.

[Saeve]

Hi @Menachem35 and welcome

I think there’s a lot of potential issues with the provided examples.
If you code follow the same pattern as the one you’ve linked to, then you’re most certainly falling into two anti-patterns:

1. Querying with Descendants
2. Over querying

There are far too many, too expensive queries that doesn’t really scale well with the amount of content.

Furthermore checking the website on the GitHub page with Google Page speed yields a pretty bad score:
https://pagespeed.web.dev/analysis/https-www-opli-net-lasers/xnwfk44nin?form_factor=mobile
Now, this doesn’t necessarily impact the server, but it does impact your visitors.

As Steve Morgan said, when you write:

@Menachem35

instance count jumps to 8

What does that mean? If you’re scaling the Umbraco instance, then you have to be super careful and strictly follow the loadbalancing guidelines.
Umbraco cannot work correctly if more then 1 publishing instance is serving content.
So, IF, you’re scaling your Umbraco instance, are you then making sure, only to scale the frontend render servers?

While looking through the thread I saw you reference that database tier you’re on is S1.
Please consider scaling to at least S2 for your SQL server as Umbraco has defined:
https://docs.umbraco.com/umbraco-cms/fundamentals/setup/server-setup/azure-web-apps

“The minimum recommended Azure SQL Tier is “S2”, however noticeable performance improvements are seen in higher Tiers”

This is a key part to optimal website performance, especially if you’re doing expensive querying.

To answer this question.

@Menachem35

Is it Umbraco / my website or that Azure is just a bad service?

No. Azure is just fine, but managing infrastructure is a complicated task.
There are certain requirements and configuration that MUST be done correctly or it will severely impact a sites performance and reliability.
Your website might have a few potential performance issues in code, but I think it’s largely a SQL DB issue.
What is your average DTU usage when the site crashes, my guess is close to 100%.
Scaling Umbraco incorrectly will also cause you many problem, I guarantee, so consider not scaling unless you fully understand the requirements and scenarios where this might be applicable.

Hope it helps

[Menachem35 (Menachem Glik)]

Hi, thanks.

I already refactored the code, still issues. I didn’t pushed yet the refactored code to my github, so put it here:

@inherits Umbraco.Cms.Web.Common.Views.UmbracoViewPage
@using Umbraco.Cms.Core
@using Umbraco.Cms.Core.Models.PublishedContent
@using Umbraco.Cms.Core.Routing
@using Umbraco.Extensions

@{
    var get_feature_story = Umbraco.Content(1561); //Get properties of the Features articles page
	var feature_story_id = get_feature_story?.Value<IPublishedContent>("mainNewsItem")?.Id;  //Get the content picker with the main story at home page
}

@{

	var photonics_node = Umbraco.Content(1061);
	var photonics_pagesToList = photonics_node?.DescendantsOrSelf().Where(n => n.ContentType.Alias == "NewsPage").OrderBy(x => x.CreateDate);

	var feature_story_HP = feature_story_id.HasValue ? Umbraco.Content(feature_story_id.ToString())
	                       : null; //Get the content picker with the main story at home page

	/* *** Get first items of Imaging & Solar *** */
	var Imaging_node = Umbraco.Content(1064);
	var Imaging_pagesToList = Imaging_node?.DescendantsOrSelf().Where(n => n.ContentType.Alias == "NewsPage" && n.IsVisible())
	                                       .OrderBy(x => x.CreateDate);

	var Solar_node = Umbraco.Content(1306);
	var solarNewsItem = Solar_node?.DescendantsOrSelf().Where(n => n.ContentType.Alias == "NewsPage" && n.IsVisible())
	                                                       .OrderByDescending(x => x.CreateDate)
														   .FirstOrDefault();

	<!-- ****************************************** -->
		
	<!-- *** Headlines *** -->
	<div class="row">
		<div class="col-xl-6 col-lg-6 col-md-12 MainNewsBox" id="MainNewsItem">
					
			<figure>
			   @if (feature_story_HP.HasValue("articleImg"))
				{
					var image = feature_story_HP.Value<IPublishedContent>("articleImg");
				    <img src="@image.Url()" alt="@image.Value("photoAlt")" class="img-fluid news-img" />
				}
				<figcaption>
					<a href="@feature_story_HP.Url()">
						<h3 class="MainNewsTitle">@feature_story_HP.Value("title")</h3>
					</a>
				</figcaption>
			</figure>
							
			<!-- Old version -->
			@*<p>
            	@feature_story_HP.Value("metaDescription")
                <br />
				<a href="@feature_story_HP.Url()" class="news_link">Continue >></a>
			</p>*@
		</div>
				
		<div class="col-xl-6 col-lg-6">
			<div class="row">
				<div class="col-xl-12 col-lg-12 col-md-4 MainNewsBox">
					<figure>
						@if (Imaging_pagesToList.Last().HasValue("articleImg")) 	
						{
							var image = Imaging_pagesToList.Last().Value<IPublishedContent>("articleImg");
							<img src="@image.Url()" class="img-fluid wide-news-img" alt="@Imaging_pagesToList.Last().Value("photoAlt")" />
							@* <img src="@Umbraco.Media(Imaging_pagesToList.Last().Value<IPublishedContent>("articleImg").ToString()).Url()" class="img-fluid wide-news-img" alt="@Imaging_pagesToList.Last().Value("photoAlt")" />*@
						}		
						<figcaption>							
							<a href="@Imaging_pagesToList.Last().Url()">
								<h3 class="MainNewsTitle-1">
									 @Imaging_pagesToList.Last().Value("title")  
								</h3>
							</a>
						</figcaption>
					</figure>
				</div>					
															
				<div class="col-xl-6 col-lg-6 col-md-4 MainNewsBox">
					<figure>
						@if (photonics_pagesToList.Last().HasValue("articleImg")) 	
						{
							var image = photonics_pagesToList.Last().Value<IPublishedContent>("articleImg");  
							<img src="@image.Url()" class="img-fluid news-img" alt="@photonics_pagesToList.Last().Value("photoAlt")" />
							@*<img src="@Umbraco.Media(photonics_pagesToList.Last().Value<IPublishedContent>("articleImg").ToString()).Url()" class="img-fluid news-img" alt="@photonics_pagesToList.Last().Value("photoAlt")" />*@
						}
						<figcaption>															
							<a href="@photonics_pagesToList.Last().Url()">
								<h3>@photonics_pagesToList.Last().Value("title")</h3>
							</a>
						</figcaption>	
					</figure>							
				</div>
						
				<div class="col-xl-6 col-lg-6 col-md-4 MainNewsBox">
					<figure>
						@if (solarNewsItem.HasValue("articleImg"))
						{
							var image = solarNewsItem.Value<IPublishedContent>("articleImg");
							<img src="@image.Url()" class="img-fluid news-img" alt="solarNewsItem.Value("photoAlt")" />
							@*<img src="@Umbraco.Media(Solar_pagesToList.Last().Value<IPublishedContent>("articleImg").ToString()).Url()" class="img-fluid news-img" alt="@Solar_pagesToList.Last().Value("photoAlt")" />*@
						}
						<figcaption>															
							<a href="@solarNewsItem.Url()">
								<h3>@solarNewsItem.Value("title")</h3>
							</a>
					</figcaption>	
					</figure>
				</div>
			</div>
					
		</div>
	</div>
	<!-- *** End Headlines *** -->
							
	<!-- Advertising -->
	<div class="row">
		<div class="col-xl-12 col-lg-12 col-md-12 col-sm-12">
		 	
			
			
			
						
		</div>
	</div>
	<!-- *********** -->
			
	<!-- *** More news at photonics and physics *** -->						
	<div class="row">
		<!-- *** Photonics and physics - left box *** -->
		<div class="col-xl-6 col-lg-6 col-md-6 col-sm-12 col-xs-12">
				
			<div class="category-title-container">				
				<h2 class="category-title-item">
					<a href="/photonics">Photonics and Physics</a>
				</h2>			
				<div class="stripe-line"></div>
					 
			</div>
			
			@foreach (var photonics_newsItem in photonics_pagesToList.OrderByDescending(x => x.CreateDate).Skip(1).Take(5))
			{
				<div class="img-text-container">
					<div class="news-img-left">
						@if (photonics_newsItem.HasValue("articleImg")) 	
						{
							var image = photonics_newsItem.Value<IPublishedContent>("articleImg");
							<a href="@photonics_newsItem.Url()">
								<img src="@image.Url()" alt="@photonics_newsItem.Value("photoAlt")" />
								@*<img src="@Umbraco.Media(photonics_newsItem.Value<IPublishedContent>("articleImg").ToString()).Url()" alt="@photonics_newsItem.Value("photoAlt")" />*@
							</a>
						}
					</div>
					<div class="news-text-right">
						<h3><a href="@photonics_newsItem.Url()">@photonics_newsItem.Value("title")</a></h3>
						<p>						
							@Html.Truncate(@photonics_newsItem.Value("metaDescription").ToString(), 120, true)
						</p>
					</div>	
				</div>
			} 
																				
		</div>
				
		<!-- *** More in Photonics and physics - right box *** -->
		<div class="col-xl-3 col-lg-6 col-md-6 col-sm-12 col-xs-12">
					
			<div class="category-title-container">
				<h2 class="category-title-item">More in Photonics</h2>	
				<div class="stripe-line"></div>						
			</div>
			
			 @foreach (var photonics_newsItem in photonics_pagesToList.OrderByDescending(x => x.CreateDate).Skip(6).Take(14))
			{
				<h4 class="physics-news">
					<a href="@photonics_newsItem.Url()">
						@photonics_newsItem.Value("title")
					</a>
				</h4>				
			}	
		</div>
		<!-- *** End More in Photonics and physics - right box *** -->
				
		<!-- Up coming events -->
		<div class="col-xl-3 hidden-lg-down Events-box">
					
			<div class="category-title-container">
				<h2 class="category-title-item">UPCOMING EVENTS</h2>
											 
			</div>
			<aside>		
				@await Html.PartialAsync("Show Events Sidebar")
			</aside>
		</div>
		<!-- End Upcoming events -->
				
	</div>
	<!-- *** End More news at photonics and physics *** -->
			
	<!-- Upcoming Events - lg-down -->
	<div class="row">
		<div class="hidden-xl-up col-lg-12 col-md-12 col-sm-12" style="background-color: #F7F8FF; min-height: 200px;">
			<h2 class="category-title-item">UPCOMING EVENTS</h2>

			@await Html.PartialAsync("Show Events Sidebar")
		</div>
	</div>
	<!-- End Upcoming Events - lg-down -->
		
}

Now here is the thing, in the link you put it says: Querying with Descendants using DescendantsOrSelf

but here is my problem, here’s my tree:

so I think I must use DescendantsOrSelf() ?

can I do it in more efficient way?

The code above renders part of the home page

Thanks

[mattou07 (Matthew Hart)]

Hey!

Based on what I read you are unintentionally trying to load balance Umbraco. (Single App Service Plan multiple instances)

Try reducing the instances to one and see if the issue goes away.

If you want to have more than one instance you need to configure the site for Load balancing as described here .

Load Balancing Azure Web Apps | Umbraco CMS

This needs two separate app service plans one that scales out (Public front end) and one that is single instance (Umbraco back office). The Umbraco back office cannot be scaled out currently but the front end can. Also insure your database is at least S2 Tier as mentioned by others.

Double check you also have the correct configuration setup for Azure web apps here: Running Umbraco On Azure Web Apps | Umbraco CMS

[Menachem35 (Menachem Glik)]

Ideally the app runs on one instance. I just set the Scale Out to Rules Based so that it adds instances based on metrics

My database is Standard S0: 10 DTUs

[LuukPeters (Luuk Peters (Proud Nerds))]

We had some issues with SQL connection pools that got exhausted. The Azure Web App would just be doing virtually nothing and the database would hardly be pushed, but the site would still be very unresponsive.

The issue in the end was that MultipleActiveResultSets (MARS) in the connection string was set to true and that didn’t go very well with our application. I doubt it’s your issue because connection pooling issues would probably show up in the log (they did in Application Insights), but I wanted to share, just in case.

[Saeve]

I still stand by mine, and the others, statements.

You cannot just scale the number of Umbraco instances.

So leave it to a single instance until you made your code ready for loadbalancing.

Still, as I wrote initially, the minimum SQL tier that Umbraco recommends is S2.

You should try to scale that from S0 to S2 and tell us if it resolves your issues.

Please also post some screenshots or information on the DTU usage for your database.
Preferably statistics over some time, like 7 days or so.

[cheeseytoastie (Steve Morgan)]

^^ what Saeve said! Just in case it’s confusing - just because Azure lets you scale the number of web app instances doesn’t mean this will #JustWork. I think you’ve turned this on to help but you’ve actually caused the issue.

Umbraco is a CMS - it basically needs a dedicated publishing instance that will control the load balanced instances (e.g. tell them when to update cache and content updates on publish etc) - or things get very messy, very fast. What you’ve got are multiple instances starting up and each thinking they are the master instance and probably locking database tables and trying to overwrite / update caches. This usually results in lots of cache errors and eventually a crash.

Read this -

Umbraco in Load Balanced Environments | Umbraco CMS

But the simplest step for now is to remove your scaling - ensure it’s only a single instance and scale UP not OUT (e.g., as Saeve says scale the resources of the single instance from S0 → S2 ).

Scale up features and capacities - Azure App Service

If you’re finding a single instance still isn’t handling your peak traffic (which will be an awful lot based on what I think your site is doing) then I would suggest the easiest step is to add Cloudflare in front of your site - this handles an enormous amount of load and shelters your webapp from the effects of lots of requests of the same pages. It’s not always suitable (for example if you have built the site to have server side rendering of carts or member login details within the markup rather than Ajaxing in) but for most brochure ware sites that serve the same pages up on each request it’s a no-brainer. Much cheaper than Azure resources too. If you DM me your site url I can quickly tell you if this would work for you or not.

[Menachem35 (Menachem Glik)]

Hi Steve, thanks. And also thanks @Saeve , @mattou07 and @LuukPeters for your help!

It seems that this indeed was the issue!

Today, after sending the newsletter the site crashed again. Now on the (expensive) Premium v3 P0V3 plan.

Ok, this is doesn’t make sense! Crash under Premium v3 P0V3??

Thankfully your answers here saved me this morning (as well as saved some money) because I noticed Instance count jumped to 8. So I changed the Scale out method to manual and set to 1 - and like a magic the website was back!

So now it’s set to 1. I also downgraded again to Standard S1 with Scale out method set to manual and set to 1 and it seems to work.

Now, about the Databse:

scaling to cost over $100 ! so it becomes very expensive. the Standard S1 which is $73 + over $100 database ? too expensive. Staying on tier 1 for the Database makes more sense in terms of budget

Thanks again.

[Menachem35 (Menachem Glik)]

This my database usage:

[Saeve]

Awesome, glad you got to a stable state!

Thanks for the images of the DTU usage.
I doesn’t look like this was causing you much trouble, so I think you can leave your SQL server at S1.

If you start experiencing issues with Umbraco being slow or the Web App not able to boot, then check the DTU statistics.
In case the DTU is consistently high, then you might have to consider up-scaling.

[cheeseytoastie (Steve Morgan)]

Good to hear you’re stable!

The DB usage graph in Azure is a bit misleading on the 7 day view. It will show the peaks - I would hope / guess that if you change that data to “1 hour” it will be pretty flat.

Umbraco really hits the DB on startup (and if the website restarts which it can do as a .net app) - unless you’re making lots of publishing changes regularly or have lots of custom tables (e.g. running your ecommerce) then a standard brochure site won’t use the DB much.

Unless you see that DTU usage hitting 100% for more than a minute at times then I would say you’re in a good place.

Again though - if it is a brochure site consider Cloudflare rather than scaling - much cheaper that giving Azure more £$£€!

[mattou07 (Matthew Hart)]

Congrats that’s great to hear things are becoming more stable!

There are a few ways to try and squeeze more cost efficiency out of Azure in terms of database highly dependant on your circumstances though!

If possible you may be able to utilize elastic pools to reduce cost: Manage Multiple Databases With Elastic Pools - Azure SQL Database | Microsoft Learn

If you have multiple databases on the same Azure SQL Server you can pool them into an elastic pool to save on ££ instead of scaling them to tiers individually. This can cost more depending how many databases you have on a single Azure SQL Server but if pooled together you can end up spending much less. This is only viable if you have a mixture of high and low utilized databases. If they are all highly utilized it won’t work and you will starve your databases of the compute resources they need.

For our clients various dev environments we have 55 databases in a single elastic pool, scaling those all to individual S2 Tiers will cost a fortune!

The other approach is caching either using more caching techniques to reduce the amount of calls to the database in the codebase and/or setting up something like Cloudflare to reduce the amount of traffic to your origin.