As I understand it, there are numerous problems with using client certificates in practice. Here are some of the issues I've heard:
Getting client certs. It is a pain for users to initially get a client cert in the first place. Obviously, sites don't want to create unnecessary hurdles that users have to jump through before they can even begin to use their site. (Even having the site generate the client cert and provide it to the client is not a familiar operation for a user -- not to mention that it has security problems, since ideally, only the client should know the client's private key.) Modern browsers provide ways to create or import private keys and client certs, but the user experience is horrible.
I've heard that some servers apparently don't support client certs well (e.g., http://osdir.com/ml/encryption.cryptlib/2005-09/msg00000.html
).
Mobility is a problem. Despite all their problems, passwords are very portable; client certs are not. Even if the client manages to generate a private key and get a client cert for it, if they want to start using a second computer, they'll have to do a complicated and confusing dance to transfer the private key and cert over to the second computer. If their first computer dies (or suffers a hard disk failure) and they have to re-install their OS or buy a new computer, and if the client hasn't backed up their private key, they will have no way to put their private key and cert onto their new computer; they're pretty screwed. A site can create a backup method of authentication so users who lost their private key can generate and submit a new private key, but this backup method of authentication can easily become the weakest link in the chain. It doesn't matter how strong your primary method of authentication is, if your backup method is easily defeated.
It's not clear whether there is sufficient support from browsers for client certs. I don't know if all the various browsers out there (including mobile browsers) adequately support client certs.
The PKI model is pretty well broken anyway. The assumption was that the CA would verify the identity of the user (e.g., their real name) before giving them a client cert, and the protocol was designed around this assumption. However today's CAs don't really verify identity. This creates a mismatch between the design assumptions and the reality.
Client certs create privacy risks for users. In some browsers, they enable tracking of users across domains. (All browsers ensure that cookies carefully defend against this threat: only the domain that set the cookie can read it. However, browsers don't incorporate similar protections for SSL client certs.) I don't know if modern browsers have effectively addressed this problem, but it's an area where client certs are relatively immature, as far as I know.
There are annoying technical problems, if the client generates and signs its own client cert. According to the TLS protocol, the server is supposed to first ask for a list of CAs that it's willing to accept; the client subsequently responds with a client cert issued by that CA, if it has one. (Note that this happens before the client has sent a HTTP request, any HTTP cookies, a username, or any other identifying information.) This means that, if you want to use client-issued (self-signed) client certs, then the server has to know the name of the CA that the client is using before the client even authenticates or identifies itself. That's tricky.
Finally, the cherry on the pie: client certs don't solve the phishing problem. They prevent theft of the client's private key, but not other personal information. Bad guys can still set up a fake bank site (which discards any client certs sent to it). The phishing site won't learn the user's private key, but if the user thinks they've connected to the real bank site, they may type in their bank account number, SSN, PIN, etc., revealing that information to the attacker.
All of these translate to poor usability and increased support costs (e.g., helpdesk calls). In short, you can use client certs, at least in principle; it's just that, in practice, doing so is inconvenient for everyone and usability sucks.
More fundamentally, there is a chicken-and-egg problem: until many sites start to use client certs, browser manufacturers will not place a high priority on making client certs usable; and until browsers make client certs usable, sites won't adopt them. To put it another way, if we want to use client certs to solve the general authentication problem, we have to both convince all browser manufacturers to make major changes to browsers, and convince sites to adopt new technology. No benefits accrue to anyone until all these parties make coordinated changes. That's a huge hurdle to adoption.
There are better solutions for secure authentication on the web, involving HTTPS, secure persistent cookies, and email-based recovery. However that's getting beyond the scope of your question.
Last comment: The fundamental challenge in secure web authentication is usability, usability, usability. Making systems that work well for users -- and are secure, when used as ordinary users are likely to use them -- is very challenging, and is all that matters. Crypto is only a tiny piece of this problem.