Is the 2/3 for caching using part of each NVMe, or is there one dedicated for the rest? Mostly wondering about spreading writes more evenly, rather than peak throughout.
hey, this is Rami ... I lead HW team at Cloudflare.
Great Q! .. in our case, there was a sweet spot between the number of cores, the L3 cache per core, NUMA latencies, mem bw per core, cost & power. The 48c gave us the best req per second per $, and it likely was due to a combination of all these things.
We have two more blogs that will come out probably today, which will shed more light on why AMD worked better for us.
Hi, this is Drew from Netflix.. Just curious, are you running single socket rome in numa or non-numa mode? I've seen better performance in NPS=4 mode myself. We seem to see about a 7% increase when moving from NPS=1 to NPS=2 and another 7% going to NPS=4.
Can I ask the chassis and motherboard combination? I know previous generations have been based on QCT chassis and motherboard components. I am curious what the non-hotswap, shallow depth, 1S is as it probably saves a fair amount of money dropping the HPC/density and hot swap premiums.
We're still using ODM built servers that are available for the public.
Yes, moving off the shared infra chassis helps us improve reliability & cost thru simplicity, and enables us to scale our infrastructure using single nodes vs. X nodes as the smallest scaling unit.
Our machines are stateless as our globally interconnected infra is widely distributed and replicated. Hence we moved to no-hotplug.
If a server can take OCP mezz cards and the BMC can be managed in the same manner and with the same tools as a fully OCP-spec system, does it really matter that it's not in an OCP-spec chassis and rack?
Yes, the 3TB is 2/3 used for Caching, and 1/3 as a local storage for our SSL/DNS/Firewall/etc.