Why is it so slow? I use VMware player and swapping files is fine. WSL2 seems like it's just using a lightweight VM over HyperV... is Hyper V really that much worse than VMware (and VBox) in this?
WSL2 uses 9P (of the venerable Plan 9 pedigree, I assume because Erick Smith is a nerd!) to remote the filesystems. I don't think that's part of HyperV (though it could be a custom VSP/VSC, and that channel is ridiculously slow.)
Personally I haven't run into slowness other than Windows sucking at copying lots of small files, which is NT's fault for allowing FS filter drivers and their ridiculous locking scheme.