Sounds like you are on an older version, perhaps?
There should be a countdown modal informing you that your token is about to expire when there is 1 minute left as of Adds background worker to check timeout state by iOvergaard · Pull Request #19702 · umbraco/Umbraco-CMS · GitHub which is available in 16.2.0.
This background worker keeps checking the token for expiration even when you are not performing actions, however, it cannot know if the server thinks the token is invalid, fx if you cleared the tokens on the server or deleted the user. In that case, they will likely not know until they perform an action against the server.
If that should prove too bad of an experience, we might consider re-implementing a server-side check if the token is still expected to be valid. The old Backoffice had a call with something akin to “GetRemainingSecondsUntilLogout” that it would call constantly creating unavoidable network traffic. However, it might give the user a better experience.
In 16.3.0, which is out next week, in addition to the above, you now also have the option to stay logged in by setting an appsetting to do so (keepUserLoggedIn).
Keep in mind, we still need to refresh the token underneath, so it requires a connection to the server to stay logged in meaning, that you could technically still be logged out if your computer is left on standby and the refresh token expires. That is, for better or worse, how OAuth2 works.