As I add to almost every thread where this comes up, Privoxy is best used (IMO) with a well-maintained ABP list. Converters such as https://github.com/skroll/privoxy-adblock work pretty well for this task (put it on a cron and/or use LaunchControl to wrangle the update mechanism: http://www.soma-zone.com/LaunchControl/ )
I wonder if Privoxy would compile / install on the Ubiquiti EdgeRouter Lite (it should, since it just runs an ARM build of Debian, but firmware updates would possibly wipe away all your set-up). I'm also not sure if the little ERLite has enough horsepower to efficiently handle running a filtering proxy with tons of rules.
I was using it for a while in place of AdBlock because of the memory issues associated with the Chrome AdBlock extension (Gmail would regularly and quickly balloon to 1-1.5GB of memory usage).
I recently moved over to µBlock to give it a shot, and pages seem to load faster than through Privoxy (plus I can use QUIC on Google services since there's no proxy), but I haven't had an opportunity yet to test the memory implications.
I can highly recommend µBlock as a alternative on ABP. The developer has written a whole page on the github wiki regarding performance implications of ABP [1]. The more advanced extension µMatrix is also very much worth checking out. It is kind of a alternative of Noscript/RequestPolicy in one on chrome based browsers[2].The developer is very serious with the development and responds quickly on issues.
The best is that it does not make you depend on some plugin. Whatever experimentation with web browsers (dwb, luakit, ...), and the odd mail client that needs to access the web: The ad blocking just works. It is a solution implemented at the right layer.
Privoxy is amazing. I have tried Squid, TinyProxy and other proxy software for a high traffic project. TinyProxy and Squid managed to scale quite well but at a certain point it strains and I encounter an array of problems. Privoxy is the only proxy which scales well under a heavy volume of traffic whilst being very straight forward to setup. Privoxy is to Squid what Nginx is to Apache.
Looks like it doesn't support HTTPS, which means many sites can't be filtered by it, and that number will only increase.
I use Proxomitron ( http://en.wikipedia.org/wiki/Proxomitron ), which is quite similar in basic operation but also allows filtering HTTPS using OpenSSL. You do need to create and install your own certificates, which fortunately isn't all that difficult. I suppose you could call it a "benevolent MITM". The author has unfortunately long passed away, and it's not open-source, but there's still a small and active community working on patches to improve its functionality.
I haven't thought about privoxy in years. It's a great way to filter ads/malware garbage for an entire LAN without having to put ad blocking plugins in every browser. I'd consider adding it to my own firewall and pushing all port 80 traffic through it but with the trend for everyone to move to HTTPS this tactic will not work for long...
I haven't used Privoxy for ad blocking but that's a novel idea.
Just wanted to comment that setting up your own CA is the solution I chose to go with to continue filtering HTTPS sites. Certificate pinning can prevent this, but apps or devices that employ that can simply be uninstalled or resold.
The move to HTTPS is intended to benefit consumers, it doesn't have to be an obstacle to viewing sites the way you want to.
I believe in being able to verify that a device or app isn't leaking sensitive information, and I enforce that using HTTPS interception. As an app developer if you attempt to lock me out of the communications leaving my network, I the choice of potentially compromising my security and privacy or the choice of blocking the traffic, and I choose to take a hard line when it comes to security.
The frustrating thing is that HTTPS is typically seen as good guys (server operator) vs bad guys (anything who isn't the client browser). But there's a lot of gray areas.
Take any network that wants to scan HTTPS traffic for incoming viruses at the perimeter for example, which is a lot of corporate networks. Any use of certificate pinning restricts the network owner's ability to virus-scan or apply Data Leakage Prevention rules to that traffic.
We probably both agree that virus scanners are unlikely to catch emerging threats and that DLP rules are easily bypassed, but they are also layers of a much larger security onion.
Edit: HSTS headers can be stripped in transit, but certificate pinning requires significantly more effort to defeat and IMO isn't worth the effort, that's why I talked about cert pinning.
I don't think you're going to catch a lot of what you think you're going to catch with HTTP inspection. Actually getting data out of a compromised system could happen with all manor of seemingly innocent information that would pass through even a fine tooth comb of every packet. What if there's malware leaking data by appending whitespace on the end of URLs, messing with the timing of DNS requests, adding pixels to images on the fly? I don't think it's humanly possible to validate the amount of data which floods out of systems on a daily basis.
Being able to look at data inside an SSL tunnel won't tell you if it's communicating your stuff out or not. Only the most obvious leaks are going to be caught that way.
Do you realize that you disagree with your first sentence in your second sentence?
We understand that only the most obvious leaks will be caught this way. The existence of more sophisticated attackers shouldn't discourage you from trying to catch the less sophisticated ones.
I hypothesize the most reliably useful information is the metadata concerning each flow itself: what device is on the local end, what IP's on the other end, with what public key and cert chain in the KEX, at what time of day, and how much data is being transferred.
If you really want to see the data transferred by a specific app, inspect/modify your app's source code. If you don't have the app's source code, and you are worried about what data it transmits, what are you doing using it?
It's not a contradiction, though it could have been worded better. You're not ever going to be able to tell with certainty that data is not being leaked. As a person with some sensitive files, does it matter how sophisticated the attacker was? The end result is the same either way if they get out.
I disagree because the only people that will be impacted by me MITM-ing SSL connections at my home is me or people who have intruded into my network, and on a corporate network the only exposure is to employees who have agreed to corporate policies (some sites like banks are not accessible through the corporate filter for employee privacy reasons).
If I was an ISP or network provider I would have to agree with you.
Will someone please sell a well-maintained (and regularly blocklist-updated) Privoxy-as-a-service? I'd like to block ads on my phone, and haven't had time to set it all up myself.
I used to use this, back in the day with my Mac Os X setup, but then it just became a bitch to setup manually and did not support all the features I needed, like some HTTPS crap or something not working, can't remember exactly what.
Is there an easy way to install this on Mac Os X currently so I can test it out ?
Glimmerblocker is another easy-to-use OS X proxy. Its interface is a Preference pane.
I have always thought ad blocking in the browser is crazy since so many programs these days have browsers built in (mail, RSS readers etc) not to mention trying to sync the configs for multiple browsers.
Do you experience ads/malware/icky stuff inside the embedded browsers? I ask because I only use web mail and web based rss readers, and thus not a lot of experience with "native" apps and their embedded browsers.
I wonder if Privoxy would compile / install on the Ubiquiti EdgeRouter Lite (it should, since it just runs an ARM build of Debian, but firmware updates would possibly wipe away all your set-up). I'm also not sure if the little ERLite has enough horsepower to efficiently handle running a filtering proxy with tons of rules.