If your threat model includes "malicious processes running as the same user as the legitimate process that needs the key", you need to go back to the drawing board and redesign. It is not possible to protect against that. Processes within a single user session are not, and should not be expected to be, secure from one another.
The concern you express about pipes, for example, extends to any other kind of IPC mechanism. TLS sockets with mutual auth? Any process that can connect to the socket can also be impersonated by another process in the same user context, reading the same key data, and spoofing the legit process. Even if you could securely establish the IPC, that's no guarantee that the data would be secure. For example, some operating systems (Windows notably, but I think also some Unix-likes) allow any process running under a given use to debug any other process under the same user, so the malicious process could just read the secret right out of the legit process' address space.
OK, let's talk about scenarios where you don't assume that another process in the same user context as you is malicious. Maybe that's because you (re)designed the whole thing so that it runs under a special user context used only for the specific purpose of your software. Maybe that's because you're working on an embedded device where you control all the code that runs. Maybe it's just because you accepted that trying to defend against malicious code with the same privileges as you is hopeless and pointless (if the bad guys are executing code in your user context, it's already game over, man).
You didn't specify the OS that your software needs to run on. However, in general, there are a few decent choices here for secret that you don't ever want to commit to persistent storage.
On Unix-like systems:
- Unix domain (local) sockets. Fast and fairly straightforward. Support bidirectional communication, if that matters. To be secure, you just have to create them in a location that is neither readable nor writable by any other user (something under the user profile is a popular choice). On the other hand, you said you can't host a listening socket on the Master (why not?) so this may not work.
- A named pipe, also called a FIFO. Fast and straightforward, but unidirectional only (if you want bidirectional, you need to create a pair of pipes). As with the local socket, make sure the FIFO is created where no other user can access.
On Windows:
- A named pipe. Unlike Unix named pipes, Windows pipes don't go in the standard file system; there's a special "named pipe file system" for them. This means you need to ensure you create the pipe with the right access control list (ACL). Once created, though, using the pipe is fast and simple.
- A named memory mapping. The
CreateFileMapping API can be used to create a memory mapping that is not backed by any normal file, but instead by the system paging file. While the pagefile is backed by persistent data and therefore could lead to the secret being stored on disk, that is a risk with any process; you can't guarantee that the secret won't ever be paged out of working memory unless your platform offers (and you use) some kind of kernel-secured buffer. Meanwhile, an attacker wouldn't know where to look for the secret in the pagefile, so you're probably safe. Named memory mappings, like named pipes, must be secure using an ACL that prevents processes running under other users from opening the mapping. Once both parties have opened the mapping, though, it's very fast.
- Direct memory access (read, most likely) between processes. The client process simply reads the secret out of the parent's virtual address space. This requires a way to pass the relevant address to the other process, but it's OK if that address is exposed; other users can't access the process' memory even if they know where to look, because processes always have an ACL that prevents other users from reading their memory.
