Offhand remarks can’t disqualify PM: SC judge