To protect your privacy and avoid tracking, nothing beats cash.
There are various services that let you purchase credits in cash at a brick and mortar store, which you can then later use to purchase goods and services online.
One example is paysafecard (I haven't tried it, but you should also be able to buy bitcoin with cash). There are a number of VPN providers which accept these payments.
Alternatively, you could simply purchase your VPN access directly offline at a store.
There are also domain registrars which accept these payment methods, but most will ask for identifying information (name, address, etc) when registering a domain. So if you want to conform with registrars TOS, registering a domain anonymously wouldn't be possible. You can hide your information from third parties by requesting that your registrar doesn't disclose the information, violate the TOS by providing false information (not recommended), or find a registrar or third party service that does not request this information.