Time consuming first boot migration after migrating from v13 to v17

Questions
,
1

Hi,
we have a pretty big website with cca 4500 nodes. We recently migrated from v13 to v17 and we observe a much much longer first boot migration of uSync content.
Previously the import took cca 10 minutes, but now we are at cca 1+ hour. Previously each node took less than a second to be published, but now sometimes it takes a couple of seconds for it to be published.

We also see an increase of time for sites being published from the backoffice UI - often editors navigate away before the success message appears, which then effectively cancels the publish and only results in saving the changes.

Here is a log of the first boot import with timestamps:

[14:58:47 INF] Starting 'uSync_FirstBoot'...
[14:58:47 INF] Execute FirstBootMigration
....
[16:08:54 INF] uSync: Startup Complete 3ms

Iā€™m not even sure where to start debug? Is is Umbraco or uSync issue? Have anyone also see a pretty huge speed impact after upgrade?

Iā€™ve seen this post, but I checked our custom URL provider implementation and it doesnā€™t seem to have much impact on it event though it gets executed sometimes.

System:

  • Localhost development environment
  • Kestrel,
  • MSSQL server running locally in docker container
2

Hi @gregor-tusar-sowa

uSync uses the Umbraco APIs so I doubt uSync itself would have much of an overhead on itā€™s own. Do you have any custom notifications for saving/publishing content that could be running that are affecting performance?

If you do have any custom notifications, are you able to add any logging to see where they are slow? If it takes 1 second to publish each node that would give you about 1 hour processing time. You could try disabling them during migration if they are not required.

Justin

3

Hi @gregor-tusar-sowa

Are you running with a clean DB each install ?

the Execute FirstBoot-Migration log entry should only ever appear once for a database, (it happens the very first time the site is setup but not every time).

also even then the actual first boot will only run if you have the config setup:

  "uSync": {
    "Settings": {
      "ImportOnFirstBoot": true
    }
  }

without this - that bit of code just falls through to nothing.

if first boot is configured you see something like this:

[15:57:00 INF] Execute FirstBootMigration
[15:57:00 INF] Import on First-boot Set - will import All handler groups
[15:57:00 INF] uSync Import: 13 handlers, processed 47 items, 0 changes in 150ms
[15:57:00 INF] uSync First boot complete 0 changes in (173ms)
[15:57:00 INF] At FirstBoot-Migration

and this would be uSync doing work at startup.

the second line

[15:57:00 INF] uSync: Startup Complete 0ms

is uSyncā€˜s startup process (which does happen each time)

this doesnā€™t do anything at startup unless ImportAtStartup is set to something.

  "uSync": {
    "Settings": {
      "ImportAtStartup": "all"
    }
  }

but if it says 3ms (so 3 milliseconds) then i donā€™t think its doing anything on your site.

ā€“

I have noticed the very first publish (of the first bit of content) can be a little slow with v17, i think its priming the caches etc, but that shouldnā€™t be anything more than a few seconds.

on a very big site then there can be a lot of cache building etc, going on when the site and content gets setup, that might have an impact, but i havenā€™t seen anything go from minutes to hours. *(doesnā€™t mean there isnā€™t something to fix)

5

@justin-nevitech
Iā€™ll check for notification handlers and debug a bit.

@KevinJump
yeah, it is a clean database, we do it pretty often in development so that the content is shared between the developers and also that the ā€œtrashā€ that you make during development is gone, we do a cleanup and first boot quite ofter.

So this is our config:

...
"uSync": {
  "Settings": {
    "ImportAtStartup": "None",
    "ImportOnFirstBoot": true,
    "CacheFolderKeys": false
  },
...
}
...

Here is a screenshot from my logging:

image
image1088Ɨ114 21.4 KB

Thank you both for the tips, Iā€™ll try with notifications first.

6

Maybe unrelated, but Iā€™m just curious. You mention that trash that you make during development is gone, but not everything is trash, right? How do you get geniune good content into uSync while discarding the trash? Iā€™m going to work on a uSync strategy for development soon, so I could use some input :slight_smile:

7

We only commit the content that we want to share with other developers.
So when we develop and try things out there is many content nodes and uSync files that we actually donā€™t want, so we take care what we push to git.

Usually when we are done with developing a feature we clean the database completely, pull changes of the other devs and run the first migration import, so that we get the clear state with the newest content from the others.

I hope this helps.

8

I was planning to auto-delete the database on branch switch to make sure that you donā€™t get features from other branches into your branch and also always start with the latest content. So thatā€™s kind of what youā€™re saying, nice :slight_smile:

9

We did something a while back,. as an experiment.

deletes dbs on branch changes, does a couple of other things around how it names the db (assuming working with a SQLite db locally).

but really donā€™t have enough info on how people would want this to actually work,

if this looks even close to what you might want for cross branch stuff, we can work on it a bit more.

1 Like