Prevent Calendar/Appointment Actions from triggering Auto Reply

Question

We have an email account that we only use to send out notices and appointments from, we don't want people sending messages to that account so we have an out of office auto reply set up on the account. The only problem is when people accept, decline, tentative or forward a calendar appointment, if this is the first time they've done so, they get the OoO bounce back. Is there a way using rules to prevent that from happening? Obviously if they get it once they likely won't get it again, unless the OoO is adjusted or turned off and back on, but is there a way to set rules so they don't EVER get it for calendar/appointment actions? Using O365 and outlook 2013