If your work around is to use ip4 to send to us, that's fine. The
percentage of spam mail on ip6 is higher than on ip4. Our own outgoing
code makes choices on whether to use ip6 or ip4 based on expected
authentication level, and it is ugly. I don't like ugly, but I don't like
spam either.
At the moment, my setup is identical for IPv4 and IPv6, with of course a
preference for IPv6.
If gmail would first return a 4xy then I would never notice, because
my mail server would retry over IPv4.
Unfortunately, every now and then gmail returns a 5xy for no reason on
my side. Then I resend the same e-mail from my MUA and it gets accepted.
I don't know why gmail seems to turn transient errors into permanent failures
at the SMTP level. As far as I can tell, that is unique to gmail.