Sure, but the problem of storing messages indefinitely isn't from the protocol, it's from the client. The client may need to verify if it's actually seen a message before and keep a local index of them; it may need to double check the message still is what it expects before it tries to delete it; it needs to present you with options on how to handle message deletion; etc.

POP3 is actually less reliable at keeping messages indefinitely than IMAPv4 is, due to limitations of the protocol. Some clients have defaults which simply delete the remote copy after download (or reading, which is different), but again that's not the protocol's fault.