Will grant you on the fact that there were more than just five good emperors. Teutoburg Forest in my memory happened under Augustus, who went sort of crazed wanting his legions back.

Tiberius realized there are limits. Having read Robert Graves, Suetonius, and Tacitus, the later two in translation, after Augustus, no one was above suspicion, when it came to being a possible usurper.

Staying away can be a valuable tool, it keeps all the crazy day to day stuff out of one's head, and lets the important stuff seep through. It can also be potentially deadly, but in that position, Emperor, what could one do that wasn't?