That's a bold claim. What are your sources? The two latest things I worked on, JPEG XL and quicksort, see a 1.4 and 1.6x speedup from AVX-512. That's on SKX and includes the much-maligned throttling. On a system level, I doubt moving those to GPU is helpful.

Also, moving things to the GPU may be good for throughput but bad for latency depending on the workload, since offloading to GPU has a cost and data exchanges too.

> offloading to GPU has a cost and data exchanges too.

This is bad with dGPUs over the PCIe bus, but not so much with GPUs that share a very fast memory bus with the CPU. In this case, the layout of the data may prove challenging to keep the same for when you use a CPU and a GPU.

Agree. Real-time signal processing with feedback loops is a case in point.

If you're processing a lot of data, you're better off moving it to the GPU. If you're processing only a little data, the speed-up doesn't matter. I wonder how wide is the Goldilocks Zone where AVX512 makes a practical difference?

Golly, another sweeping statement :) It seems to me a GPU might actually sort more slowly.

For 64-bit keys, we sort about 1 GB/s per (5 year old) Skylake core, and perhaps 5-6 parallel.

This (2018) reports 3.5 GB/s: https://benkarsin.files.wordpress.com/2018/10/dissertation.p... And a 6-year old GPU radix sort reports 2.1 GB/s: https://github.com/Bulat-Ziganshin/Compression-Research/tree...

BTW I've worked on a product that used GPUs. That typically requires everything to move to the GPU, which is not always desirable or feasible.

This (2020) reports "Despite the fact that we send the entire data array to the video card and back, sorting on GPU of 800 MB of data is performed about 25-fold faster than on the processor."

https://dev.to/tishden/computing-with-gpu-why-when-how-and-s...

This shows a approximately 20x speedup (2021, graph 1 vs 3): https://www.irjet.net/archives/V8/i7/IRJET-V8I7714.pdf

Thanks for the example! Sounds like 1.6 GB/s on an entire Tesla K80 (300W TDP). This is in fact several times slower than our results on Skylake (with half the TDP), but note that K80 is from 2014.

The "25-fold speedup", as is often the case for such reports, comes from not optimizing the CPU side.

Assuming AVX-512 actually works well (haven't had the opportunity to use it myself), it could be very useful in high-end gamedev. Data-oriented programming is popular there, which makes it much easier to optimize data transformations w/ SIMD. Good compilers can even do some auto-vectorization (it's a nice boost with no additional programmer effort but you can't rely on it much). GPUs tend to be already fully loaded by the intensive rendering workload, so there is a large incentive to squeeze as much as you can out of the CPU.

gpus generally aren't worth it for moderately cheap O(n) operations. avx 512 is really nice because if you match the memory speed, you beat GPU (since the cost of GPU will also be memory speed constrained).

'few workloads that benefit from AVX-512 would be better off on a GPU.'

Have you seen the state of GPU software development? GPUs are very expensive in cloud, are poorly supported in containers and virtual machines If you want to use GPU compute, some stuff is Nvidia-only, some stuff is glitching and crashy, it's probably not avaliable in your language of choice, etc.

It is literally impossible for me to add GPU compute to any of our corporate workloads, but I can tap into AVX easilly in my language of choice.

Pretty much all cloud CPUs are planned to continue supporting AVX-512. It's only the client side where they are axing it.

One would think that, but the reality is that there are some workloads which seem to fare better on AVX-512 currently. Usually where you can't get the benefit of the massive GPU parallelism/thread rotation into play, and you need low-latency between operations for a large-enough data set, while its still small enough for benefit from the cache on the CPU.

I'm pretty sure GPUs closes that gap over time though.