Is there a general solution to delivering bytes to a destination with reliability and buffering configuration?

Question

I have some application events.

I want these events to be delivered to a few places: a periodically rotated file (in proprietary TSV format in my case), a message bus (Kafka in my case), for archival in a networked resource (Amazon Glacier in my case).

I have what I think should be a very fundamental problem though. I want my applications to just send the bytes they want sent to a few sinks, and to have another process whose job it is to make sure that:

• the messages reach their destination
• if the destination is unreachable, they buffer in memory for some period of time
• if memory is full, they buffer on disk for some period of time
• if we're getting stuck, the messages start being dropped but when the whole network isn't down we have the ability to notice that we're starting to drop messages.

I'd like to be able to configure the behavior on the above with some simple parameters for how much memory to allocate, how important it is for a particular message to reach its destination (maybe number of retries say, etc.).

To me this sounds like a fairly "basic" general problem (a hard one no doubt but I would have imagined something existing for it). I want to write bytes to a Unix socket or something from my apps, and have the agent be responsible for buffering and retrying to deliver them to my destinations.

Does such a thing exist? Is that what people use (r)syslog or journald for?