When Chrome first came out, it was the fastest browser, but yes, over the years it has come to be known as a memory hog.
Chrome has both a 32 and 64-bit version, as far as CPU performance is related, which many people don't realize (when it was first released, it was 32-bit ONLY).
Also, the more extensions you are running, the more memory is being consumed by Chrome. By default, Chrome runs a new process for each tab that you have open, and in the past, it has not been very quick to cleanup after itself, which I have heard has been improved in some of the latest or upcoming releases.
There are also many options that you can enable or disable in the chrome://flags
URL that will affect the way Chrome behaves. It tends to always run a process in the background, which I believe will quick-launch any Chrome apps you have installed or something. Someone may want to clarify this answer, but these are the basics.
In a nutshell, Chrome is not what it once was.
From my personal experience, 32-bit Chrome seems to be less of a memory hog (and obviously less of a CPU hog) than the heftier 64-bit version, and there is also the open source Chromium Browser, as well as Google's own bleeding-edge Chrome Canary, which doesn't boast stability by any means, but will always have the latest and greatest experimental features of upcoming Chrome releases.
I would also like to see some recent CPU/memory benchmarks of the latest browsers.
If you're going to downvote my answer, please post an answer of your own or add a comment as to why it's not relevant or a viable answer. – rubynorails – 2015-11-14T18:26:04.670