We’ve been using EloqKV to replace one of our largest Redis node (we didn’t want to run Redis Cluster, just a single big node). One pain point we had with Redis was the RDB fork causing latency jitter during persistence. EloqKV handles this much better — the fork-related stalls are gone, and so far it’s been a smooth drop-in replacement for our workload.
Ursa published a blog post saying their leaderless, stateless, object storage–based Kafka replacement can reduce costs by up to 95%.
Has anyone here tried Ursa in production? How much cost reduction have you actually seen compared to Kafka or MSK in real workloads?
As near as I can tell, the claims of huge cost savings derive from the difficulty dynamically scaling Kafka and improved multitenancy. So if different pieces of your company each have overprovisioned kafka clusters, they could all move to Ursa and save all the overprovisioning.
I have not tried it, and full disclosure, I really like Kafka: it's one of the pieces of software that has been rock solid for me. I built a project where it quietly ingested low gb/s of data with year-long uptimes.
The bulk of the cost savings comes from the use of object storage rather than attached disks. This eliminates the inter-AZ networking costs associated with Kafka replication mechanism.
So basically Kafka, to provide availability guarantees, requires multi-AZ and the inter-AZ replication gets expensive. And Ursa avoids that by using object storage and probably then just talking inter-AZ?
And while I like Kafka, nobody would claim it likes being scaled up and down dynamically, so probably built-in tolerance for that as well? We ran Kafka on-prem so that wasn't an issue for us, and given the nature of the service, didn't have a lot of usage variance.
We also love Kafka as a protocol. However, the implementation can be evolved to adopt the current cloud infrastructure and and rethought based on the modern lakehouse paradigm. That was one of the reasons we created Ursa.
All your points are valid. I don't think most people "dislike" C. People have options and most choose non-C. From my perspective, when the software or the system itself is already extremely complex, using C just adds more complexity on top. Many people including me choose not to add more.
Talking about transactions in Redis, one area came on top of my head is metadata in file systems. I've seen colleagues/collaborators run large-scale training on a distributed file system w/ a billion files, which puts a lot of pressure on the metadata part. They tried a few options and Redis was one of them. It's fast and Lua is good enough to support metadata ops. But the thing is that it cannot scale (or Lua is gone) and may lose data from time to time, which is annoying. It looks like this durable-transactional combination may fit in. Will wait to see how this is unfolded.
Probably not. Data flow is declartive in data transformations and "differential" refers to be incremental. But what the link tries to model seems to relate to asynchronous transformations. Not on the same level.