Absolute silence is THE TRUTH, and the ego is not.
The ego starts with a false belief of āIā (it is merely a thought), and then the whole cocoon of the ego builds around it (I-me-mine).
Staying in absolute silence is staying in the truth, and anything that is not the truth becomes intolerable, just as an impartial judge does not tolerate any liars in his courtroom.