I was checking my ISP's Out of Office reply system and I found that although they sent a message to my server, because my server replied with a 4xx error code, the server did not attempt to re-deliver the message.
I did a further check on one of their Outbound relays and found that the same thing was happening when they tried sending my server a DSN - the DSN also wasn't retried.
Is there anything I'm missing in the email standards that says DSN delivery should be treated any differently to normal mail delivery?