Does my second GPU need to be the same as my first?

Question

I have a single Titan X Pascal card in my deep learning machine. I'm looking to get a second GPU and I'm trying to figure out what my options are. It appears if I want to do SLI, then I have to install an identical card and the SLI bridge.

But if I don't setup SLI, then can I install a Titan Xp or a GTX 1080i? If I do, what kind of performance gains can I expect without customizing my code? Will a single parallelizable process run twice as fast? Or is one process going to be dedicated to once GPU such that I would need a second process to use the other GPU?

I do deep learning, but I am not tied to a specific framework (e.g., Theanos, Keras, TensorFlow). I believe that's relevant because I read Keras now has native support for multiple GPUs .

So really what I'm asking is shoudl I get another Titan X Pascal and SLI bridge, another Titan X Pascal, or a Titan Xp? Which would give the biggest performance boost?

Answer 1

This explains the requirements. Identical cards aren't necessary but there are limitations.