PM Gordon Brown did call a female voter "bigoted" on a hot mic (1) - with some justification actually, as she did say some unpleasant anti-immigrant rhetoric. Mr Brown lost an election shortly after, but this incident was not in itself what brought him down.

1) https://www.standard.co.uk/hp/front/full-conversation-betwee...