It is likely not the image itself that is temperamental but rather either some model casting gone wrong or the Umbraco context not being available in the instances it cannot load.
If you can narrow down when exactly it happens and add some checks for model types in your code you could narrow down if that is the cause or not.
If the Umbraco context can’t be created it is likely something with custom routing setups.