The fact that EV certificates exist in the first place is an indication in my mind just how badly CAs messed up the certs originally. ("We need to sell more! Get rid of the checks")
It also drives me nuts that browsers still class self-signed certs below normal (non-ev) certs when they basically offer the same level of guarantees (in terms of "this person is who they claim to be")
You are wrong. Attacks against sites with self-signed certificates are trivial to execute (you just need to download the tools and learn how to run them) and can be fully automated. Obtaining fraudulent certificates is occasionally possible (getting more difficult every day), but it generally needs to be done one site at a time, and requires a _lot_ of resources.
That said, there are many ways in which browsers could improve the handling of self-signed certificates. For example, having a Convergence-like system to fall back to seems useful. Another possibility would be to use opportunistic encryption, where all access is encrypted even without a certificate. (This would defend only against passive attackers, but it's better than no encryption.)
It also drives me nuts that browsers still class self-signed certs below normal (non-ev) certs when they basically offer the same level of guarantees (in terms of "this person is who they claim to be")