Lets say I have 4 database servers. I want to add them to the "shard" cluster.
That's it. No replica set, no nothing. Just 4 servers.
If I turn "journaling" on all of them, is it OK?
Nope
What happens if a server goes down with hardware issues.. you just lost the whole setup till that one server comes back online.
It will work (technically), but if it's a production environment, it's a horrible idea. (If it's a dev/test/staging environment, it's only a "bad" idea because you're not practicing like you're going to perform.)
It depends on whether availability is important for your application. If you shard without replica sets, you will have multiole failure points. With journalling your data can be safe, but you would experience an outage in the case of a single node failure. For some apps (eg, log file analysis) this might be ok, but for many it won't. It really depends on your requirements.