Yeah, I've seen this behaviour with Signal. It's UI is somewhat confusing thou, it took me a while to understand that I needed to re-generate keys so what I could "fix" my conversation. There was no redelivery of the messages that couldn't be decrypted thou.