ROP gadgets are based on the process' executable pages, which in practice usually means based on the process binary and the binaries of any libraries linked to it at runtime (there are exceptions with processes that use JIT compilers, like browsers, but mostly that's not the relevant situation). This does mean that building a ROP chain requires knowing the process being targeted pretty precisely, including which architecture it's compiled for.
So, how to do that? Well, for a lot of things, it's pretty easy: most software doesn't change very often, so you get target the current version, and there aren't that many architectures in use so you can usually target x86 (if Windows desktop), AMD64 (if Linux or MacOS desktop, if one of the programs where the default Windows version is AMD64 now, or any server), or ARM/ARM64 if targeting a specific mobile device. Most malware isn't trying to target a specific computer, it's just trying to spread as wide as possible, because the impetus behind most malware is money (ransomware and cryptominers, but also old-school spyware and even adware).
So, you just target the most common kind of system that runs the software you have an exploit for, and the most common version of that software (at least among vulnerable ones). Remember that ROP is a payload; if the correctness of your ROP chain matters at all, you already exploited the process and gained control of the instruction pointer! Typically, that means you know pretty precisely the relevant software version.
Some info gathering is also an option, especially for more targeted attacks. Software such as nmap will try to guess operating system identity and software version based on either explicit version strings or heuristics about behavior. Some software just outright tells you all the relevant details, such as browser user-agent strings. You can also look for data through other sources, like if you're targeting a company, have they published anything about their infrastructure?
Another option is a multi-payload target. You build a collection of payloads, and then you hide them behind a bootstrapper that extracts some information about the system after initial exploitation, and loads the appropriate payload. This is, practically speaking, often mandatory anyhow: ASLR is near-universal now, and it breaks most situations where you would use ROP unless you can extract some object from memory and compare it to what you expected in order to determine the ASLR mask. Since you then must apply that mask to your ROP chain (unless you found a library without ASLR to target), you're already sending a custom ROP chain anyhow.
Finally, you can just try 'em all! If you're attacking a server, odds are that it's on a cluster of machines all running identical hardware and software. So you exploit one (whichever the load balancer gives you first), and if that doesn't work you just try again. The first server will probably crash if the initial exploit succeeds at all, but that's OK; the load balancer will probably stop targeting it until the watchdog brings it online again. This kind of brute-force approach is very "noisy", of course - tons of servers crashing unexpectedly tends to set off alarms - but firing off network requests is fast and most exploits are even faster, so you might still be able to get in before any human response can be mustered. What you do then is hard to say - the defenders will be trying to kick you out ASAP - but it's totally possible to try something like grabbing some secrets (keys, credentials, private files, whatever) or just doing quick recon for a future attack once they stop watching so closely.
