You do know that spinning up a new VM only takes a few seconds? With projects like https://firecracker-microvm.github.io/, the difference between launching a new Docker container or a new VM is negligible.
This works great if you own or rent the hardware, but most cloud providers don't allow nested virtualization.
The cost is not spinning up the vm, it’s maintaining the images. Docker composability reduces the combinatorics problem to a dull roar, and democratizes some of the maintenance effort. You want an image with the bug fix from the latest point release of python? And you need it by noon? Knock yourself out.
Although there are tools to convert docker images to vm images. I expect if I were running community CI infrastructure, getting really familiar with those would be high on my priority list.
This works great if you own or rent the hardware, but most cloud providers don't allow nested virtualization.