15

We have around 300 RHEL servers that are currently connecting to a Puppetmaster server. However, we have noticed some performance bottlenecks and it is the point of failure in our system. I am fairly new to puppet in general and I am considering creating a decentralized puppet architecture instead of having Puppet clients connect to the Puppetmaster server. Aside from what I would suspect to happen such as performance gain and lack of signing and exchanging SSL certs for new machines, what are other pros and cons to setting up a decentralized architecture?

JMeterX
  • 3,387
  • 15
  • 31
  • 3
    Is there a reason why it has to be one way or the other? Have you considered options somewhere in between? With that many servers, and a concern about a single point of failure, then why haven't you setup additional masters? Setting up multiple load balenced puppet servers is covered in the 'Pro Puppet' book. There is a lot of flexibility, it should even be possible to setup a hierarchy of puppet servers if that would make sense. – Zoredache Jul 16 '12 at 20:26
  • @Zoredache There really isn't a reason that it has to be one way or the other, I was looking for more information on a decentralized architecture in general to help facilitate the decision. I have considered additional masters but the core of the idea, which I apologize for not mentioning, is to reduce server count since it directly impacts our budget. I agree, load balancing the puppet servers makes sense, but if I can get rid of a server all together that would be the best solution. – JMeterX Jul 17 '12 at 12:25

3 Answers3

7

Go decentralized.

Instead of signing certificates, make ssh keys. Don't give the keys to non-admins

You can use Git as your transport instead of subversion, and then you can branch for different machines/roles, and then version your changes, as well as allowing for... but you must know the DVCS spiel at this point.

It's faster, and less finicky to set up. Add some commit hooks for sanity checking.

Now, at this point, you've replaced the puppetmaster, with its client-server model, with ssh and git, both of which scale better than the puppetmaster.

Now, there might be a need in your organization for hierarchy. No problem, just store the git repo containing the definitive branch somewhere safe.

Bonus:

git blame

will allow you to see who made a change.

http://bitfieldconsulting.com/scaling-puppet-with-distributed-version-control

https://www.braintreepayments.com/braintrust/decentralize-your-devops-with-masterless-puppet-and-supply-drop?

chiggsy
  • 1,576
  • 1
  • 15
  • 20
3

Are you running puppet in passenger? stored configs? You really shouldn't have scalability problems at all with 300 nodes as long as you handle basic setup issues.

czervik
  • 241
  • 1
  • 3
  • 1
    We are using Apache+Passenger configuration. We are also using subversion to push the changes to the Puppetmaster – JMeterX Jul 17 '12 at 12:32
1

Decentralized is the best way to go cause each client compiles its own manifest from a local copy of the manifest source. This is updated each time you push updates from the say git server. Much more efficient use of bandwidth as clients don't ahve to contact the puppetmaster on every run. Also eliminates single points of failure as clients can be updated from anywhere.

What'sTheStoryWishBone
  • 357
  • 5
  • 17