There are various ways of setting this up, including using a different authentication method than passwords to start with.
One method might be using TLS/SSL certificates for connection. But normally, keeping the password in a read-only file would suffice - as Kondybas says, if the root account is compromised, then everything is compromised.... You might also want to have the startscript do a chmod 400 before starting the service, and chmod 000 once it's started.
So instead of only focusing on keeping the actual password secure, you'd want to do some work on the target server. For a database, I'd start by limiting what hosts can connect with the username used by the Linux server. I'd also limit what the user is allowed to do with the database - for instance, you may want the linux daemon to be able to add content to the database, but maybe not to do DROP TABLE.
You should also generally make the server as secure as possible. The daemon shouldn't be run by root unless absolutely necessary; the SSH port shouldn't allow access from any host on the Internet, and certainly not with root login allowed.
Also make sure that password is not reused anywhere else, otherwise a compromise on that server could lead to an escalation of the breach on your network.
You may also find a search at security useful.
