A very good question and it reveals that you and I are assuming slightly different settings for the team. You are, I think, referring to a dedicated team in a sprint on a single client, usually with a “locked” or “promised” backlog. In that case, I think you are correct, that no manager would be allowed to adjust the team’s priorities since they technically “own” the sprint.
But this is a very narrow, though traditional, application of how Agile handles the need for Authority versus Control. (see here for my terminology) The usual model is that the Authority function (priority is one aspect of that) is used in project backlog development and grooming, as well as in sprint planning, during the selection of the Sprint Backlog items. Aside from that, and during the sprint, the Managers would not be allowed to do anything that would affect the team’s delivery.
We teach a more flexible Agile, though, one that supports multi-project teams (usually 20+ people) and it also allows for re-prioritization during the sprint, based on changing business needs. That is the case that I stated, which is that the manager should inform the team of the new priorities, but not instruct the team on how to implement those priorities, which is the delineation between Authority (factual) and Control (work process.)
Thanks for your great question!