Whenever I see the "You don't need Mongo DB, use an SQL database" and then in the flaming back and forth, I never see my key problem mentioned:
MongoDB makes it easy to scale out (replica sets and sharding), where is the "easy to setup replicated and sharded open source SQL database?"
I mean, I know that Postgres has replication (via Slony? honestly, it's been awhile since I looked at their solutions) but I don't recall it being as dead simple to set up.
For me, setting up replication needs to be easy because we redistribute the store as part of our product and we need scalability (both replication for redundancy and sharding for scaling).
So I'm honestly asking here, where is the easy to use sharded and replicated open source SQL store that I've been missing?
That "easy to scale out" is a misnomer. Replica sets and sharding work in the technical sense, but the implementation isn't anywhere near what I would qualify as production ready.
For example, today my entire production MongoDB database was running 3x slower because a single replica in one shard was down, and their buggy PHP driver kept trying to talk to it despite it being marked down. I really enjoyed waking up at 2am to deal with that.
It relates back to the "easy to use" nature of their marketing. It really is super easy to use and develop on, but the minute you need to do anything important or serious, it breaks down.
You aren't doing yourself any favors going with it except as a proof-of-concept.
But Mongo DB being buggy isn't a reason to need to use an SQL database vs. a NoSQL store. An SQL database could be buggy as well (I still use Postgres and comparing anything to that quality-wise is just going to bring sorrow for the thing you compare it to ;) ).
FWIW, it's been spotless for us so far. Our needs aren't web scale, but they're big enough to need scaling features.
The main issue with MongoDB is that it's so easy to use and seems like it scales, but soon you're invested in it to the point of refactoring being a serious engineering effort, and you're stuck with something that doesn't actually offer real scaling features.
So, it's less "nosql vs. sql" and more just "don't use mongodb".
Postgres has hot standby built in nowadays, and it works well.
Sharding certainly isn't as easy - the technical compromises that mongo makes make it pretty trivial to implement, whereas it's relatively hard to make it work in an RDBMS while maintaining all the expected capabilities. It generally requires some application-level work on open source dbs.
With that said, I really think many people grossly underestimate the effectiveness of scale-up. It's worth remembering that Stack Overflow (for example) is still running on a single pair of master/hot standby database machines.
Standby is a pretty poor solution compared to replica set let alone what Cassandra has to offer. Sharding is trivial on MongoDB/Cassandra and it is open source. So let's be accurate here. It is a problem inherent with the SQL databases.
And I think you underestimate the benefits of scaling out. If I want to ensure close to 100% uptime or have a server closer to my users than Cassandra or even MongoDB would be infinitely easier to setup and manage than Postgres. These are "very nice to haves" for even the tiniest startup.
Sharding is, of course, trivial on those systems - after all, they're extremely feature-poor, and have given up those features specifically to support trivial sharding.
You can replicate much of this behaviour using open source RDBMSs, but yeah, it's not what they're designed for and it's harder. If you want quality replication/clustering you're currently looking at paid-for DBs.
Being able to scale out is absolutely a nice-to-have. I'm not sure it's a nice-to-have on the scale of giving up all of the features an RDBMS provides for most people's use-cases. Further, you might find that the relative lack of data headaches you get with an RDBMS more than makes up for a little extra time setting up hot standby.
Finally, if 100% uptime is that important you're probably not relying on a relatively niche NoSQL database. If uptime on the level of Stack Overflow is good enough (which for most people it probably is, let's face it), then you'll probably find replicated postgres good enough.
Thanks, that's very much what I'm talking about. Cassandra would be my ideal store, I absolutely love it except for the ability to index across nodes. My understanding, when last I looked at it, was that indexes were only local and didn't span Cassandra nodes. Does Cassandra now have properly distributed indexes?
When I was looking at implementing Cassandra instead of Mongo DB, it seemed like we had to create reverse column family (IIRC, been away from Cassandra for a bit now). Is that still the case?
How do you use a local index when your data is distributed across numerous nodes? Maybe I'm missing something fundamental, so I'd definitely like to understand.
It's hidden from me behind client library API(astyanax in my case), I shouldn't know anything about index locality. I just send request(give me records for this index value), and get response and don't care if it's local index or distributed index. Astyanax takes care about everything, it queries all nodes.
Just refreshing my memory here, but this blog post [1] is what kept me away from secondary indexes and explains why I need something more from Cassandra. Especially the section on "The Good: Secondary Indexes", I actually have some data that is stored by timestamp, that was, as I recall, the biggest turn-off on them.
Has the state of affairs advanced since a year ago? Would love to hear it has!
I will check that out, thanks! I was looking at Hector and Pelops at the time and I don't believe they provided anything like that. But now that you mention it, I bet Hive could do what you describe as well. Need to dig a bit deeper now!
MongoDB makes it easy to scale out (replica sets and sharding), where is the "easy to setup replicated and sharded open source SQL database?"
I mean, I know that Postgres has replication (via Slony? honestly, it's been awhile since I looked at their solutions) but I don't recall it being as dead simple to set up.
For me, setting up replication needs to be easy because we redistribute the store as part of our product and we need scalability (both replication for redundancy and sharding for scaling).
So I'm honestly asking here, where is the easy to use sharded and replicated open source SQL store that I've been missing?