Bulk content update due to schema change is very slow

Questions
1

I have a couple of folders with several thousand product items in each. All the products are using the same data type.
Recently I had to add a new field to the type and attempted to deploy this to my DEV cloud environment. This failed multiple times which appeared to be a problem of db timeouts.
I ended up deleting all the products and clearing the Trash, and then the update succeeded. (Then I had to repopulate all the products. This was ok since there is a batch job that does it.)

When I deployed the DEV up to PROD I made sure to delete all products prior to beginning the deployment. That deletion process took over an hour all on its own. Clearing the Trash took another hour. A deployment process that normally takes about 20mins ended up taking nearly 4 hours.

So is there a better way to do this - updating the schema of a data type which is used by thousands of items?

The problem is mostly that it just takes a really really long time with no indication of progress up front. Deleting a folder of items when there are over 1000 entries also takes a really looooong time. And then I had to also delete those same items a second time from the Trash, else the schema change still took a loooong time.

And this was just an addition of a new field, not even a modification. (In SQL adding a field is usually quick, though modifying an existing one requires creating temp tables and migrating data, so I understand that that takes longer.) There are not 10s of thousands of records, just a few.

Does anyone have advice on a better strategy for mods to a schema when there are lots of records?

Cheers,
AJ

2

Hi @ajatnow

I assume you mean Umbraco Cloud. If it’s a different cloud, please let us know.

It may be quicker to do this locally and re-upload the database, but that depends on your site and whether you can enforce a content/change freeze. Umbraco Cloud lets you download a copy of your database as a .bacpac, restore it locally, then upload it again once you’re finished.

There are also some Umbraco Deploy timeouts you can adjust. The backoffice transfer timeouts default to 20 minutes, but schema updates from .uda files have a 5 minute timeout. That’s probably the one you’re hitting, so I’d try raising it first (on both environments):

Justin

3

Doing a db dump with local modification sounds like the least painful method for me. Though just creating the backup file takes about 15mins or so. Still better than hours of waiting.

I have a lot of log entries and old orders that could be cleaned out to reduce the db size. Just have to work out how to do that bit of maintenance work.

Thanks for the suggestion.
AJ