First off, Fail-Over and Load Balancing are two different things.
Fail-over is having two devices or servers, one active and one passive.
When the active fails, the passive detects it, for example using CARP or VRRP, and takes over the active role.
This is used mainly with firewalls, routers and other networking devices.
Load Balancing is having many devices that are all active and are receiving their share of a load.
That being said, software or hardware load balancer will work pretty much the same. They both have pros and both have cons.
Having hardware load balancer means adding devices to your network, that means taking time to maintain these devices, implement the devices, having spaces and electricity for the devices, so on and so forth. If you only add one load balancer, then you just added a single point of failure on your network.
But, having two robust hardware load balancer can offload your servers (with SSL acceleration for example)
Both will usually use VIPs to present the service on one IP and then distribute the load between members.
If one of the members fails, it will be temporarily taken out of the "VIP network" and stop receiving it's share.
This can be done by CPU load, RAM load, network load, network latency or simply in a round-robin mode.
But it's not limited to that.
Hope this helps.