The answer is "it depends"
Some 32-bit 'applications' will not work in 64-bit mode, for example graphics drivers and shell extensions (such as Tortoise SVN). In such a case you need a 64-bit version.
For other normal applications, then it depends on what they do.
If you happen to have, for example, 16GB of memory and are running SQL server with an extremely large database, then having 64-bit version of SQL is very important.
If, however, you have 4Gb of memory (with almost 1gb 'stolen' by the video card), then you may be running low of memory and using 64-bit applications (which may require more memory) may actually slow your system down.
Even ignoring memory considerations, you can not say for certain that a 32-bit application running in 64-bit mode will be faster or slower than the 64-bit equivalent, nor whether the application will be faster or slower than when running on a 32-bit operating system.
Personally, if I'm looking for a utility or small application for my machine, I will check that it works with 64-bit operating system, but whether the application is 32-bit or 64-bit is not normally high on my priority list.
According to Process Explorer, I currently have 25 64-bit processes and 28 32-bit processes running, ignoring Chrome)
1Just wait until you start wanting to play your favourite old games with 16 bit installers. Backwards compatibility for those are provided in XP64/32, Vista32,and 7 32, but not Vista64 or 7 64. – Alain – 2011-04-15T16:06:42.687
1@Alain : I was never able to get old games running on my 32-bit OS anyways, so it's probably not a very big loss. – apoorv020 – 2011-04-15T16:27:33.567
1It is if you love Master of Orion II and Populous the Beginning as much as I did ;) Anyways, Windows Virtual PC + an old OS installation CD does the trick and saves you from dual booting. – Alain – 2011-04-15T16:38:17.747
@Alain, because the various VM software have varying levels of audio/video driver support for old OSes, even that isn’t always going to work. I had a heck of a time getting Jewels of the Oracle II to run. – Synetech – 2013-08-11T17:46:08.873
The largest improvement I noticed from a user perspective is the ability to load larger files than 2GB-ish. Especially working with large images in Adobe software. – Svardskampe – 2013-08-11T17:33:05.843