FIPS140-2 deals only with the proper way by which a cryptographic module must operate and be protected from attacks. One can be just compliant with his modules, or one can be compliant AND validated with her modules. There are 4 different levels of compliance, 1 to 4, with a higher level more protective than a lower level. Getting validated is an expensive undertaking, usually done by external security labs.
FIPS140-2 does not talk about what data need to be encrypted by an application. Various standards bodies, unique to each industry segment of interest, establish the requirements, procedures and processes of data protection and encryption specific to that industry. For example, PCI DSS by the credit-card industry, HIPAA by the health care industry.
These bodies may or may not demand that an encryption module would be FIPS140-2 compliant, but certainly being such is a prudent choice - at least from a liability point of view.
Some relevant comments from infosecisland.com:
"I have done FIPS 140-2 compliance tasks firsthand with:
OpenSSL, OpenSSH, Sun Java 6, Apache Tomcat 5.5 and 6.0, Mozilla NSS.
I can tell you that none of these modules are operating in a FIPS 140-2 compliant mode by default, getting them into a FIPS 140-2 compliant mode is not trivial, and operating in a FIPS 140-2 compliant mode is not always a good idea for compatibility reasons."
"Allow me to clarify the business cases that someone will want to invest in the time and expenses necessary to achieve FIPS 140-2 validation, not just compliance:
When you want to sell anything that is cryptographically relevant to the DoD or any government working group that looks to the DoD for security specifications. This is not going to be your typical enterprise customers, but specific government entities that require the assurance of cryptographic integrity that FIPS 140-2 provides."
So, if you don't have a crytographic module in your application, FIPS140-2 is not relavant to you. If you do encrypt some parts of your program, like data or a comm protocol - either because it is the proper thing to do or being required by a standards body or by customer requirements or by regulations - and you need to be FIPS140-2 compliant, that encrytion routine/module needs to either just adhere to the FIPS specs or even be fully validated.