Former Victorian Premier Dan Andrews' long reign at the top of the political and business worlds could be over, insiders fear.