I'm considering implementing a configuration management tool such as Puppet or Chef. Is this reasonable, or will the overhead of learning the tool outweigh the benefits?
It is reasonable depending on how much time and money you have to burn, and whether or not it's your money that you are burning.
A configuration management tool (any of them) is becoming a valuable skill in todays market.
Spending the time to learn and implement a CM tool may not be the most efficient thing to do from the perspective of your business or environment, but from your skillset it may be worthwhile.
Where is the tipping point between manageability & implementation cost?
Most configuration management tools are available for free with the caveat that they are harder to install and get going.
This question is a little hard to answer since it really depends on what you are doing on a day to day basis to manage these servers. If you aren't required to do much at all, then a configuration management tool may be total overkill.
If you only really need to enforce your infrastructure into a predictable and basic state, then it may not do much harm to pick up the very basics of something like SaltStack or Ansible.
In my personal experience, Salt is very easy to get going and bootstrap onto servers, and can be used for very basic remote execution and reporting, which may come in handy if you don't have it already implemented in your environment.
Keep in mind, I am biased. You should evaluate each CM tool by yourself.