Here’s what I have in my deployment scripts. Essentially anything that is in the deployment package apart from the media folder should be pushed - but don’t delete anything that is in the destination but not the package.
That can cause issues if you remove a dependency along the way, but since that’s a super rare thing for me, I haven’t looked into a tweak.
-skip:absolutePath=.*wwwroot\\media
-enableRule:DoNotDeleteRule