The justification is absolute rubbish. I The argument is that it reduces attack surface, but compared to what, the alternative to including the yubikey code is not using a yubikey, which reduces security. The alternative to using the browser integration (or the ssh agent) is to use the clipboard, which is likely more code and definitely less vetted code.
By the same argument we should remove encryption as well, because it increases the amount of code and thus the attack surface.
You can add different kinds of authentication to your password safe. I use a password and a key file. You can use just the Yubikey just like you can also have no encryption applied to the password safe.
By the same argument we should remove encryption as well, because it increases the amount of code and thus the attack surface.