> based on FP4 FLOPs (yes it's disadvantageous to A100, that's the point)

The A100 doesn't have hardware FP4, and you'd be running a quantized model with some accuracy loss but unless this was natively trained on FP4*

* to add another layer, they own the model and could apply tons of post-training techniques to reduce that accuracy loss and probably already do