It just crossed my head. What if we use a flag count (times a comment or submission was flagged and flagging was agreed on by a moderator) to "counter vote". If someone who get flagged often upvotes something, that upvote acts as a flag and could, maybe, be turned into a downvote...

I'll edit it and make it clearer in the next hour, but I wanted to leave it here while I have lunch (and think over it) so others can demolish (it's always nice to be corrected by people smarter than me) my idea.