1

On the face of it, it looks like I need to be able to install some software daemons on the target nodes for these things to work. If I can't do that (e.g. an Amazon RDS instance), what are my options?

Similarly, Ansible seems to be based all around SSH. Can this be used to configure RDS?

Thanks for the feedback -- the discussion has led me to a more targeted question here: https://serverfault.com/questions/588237/whats-a-good-cm-tool-for-managing-a-cloud-cluster-composed-diverse-resources-ac

Community
  • 1
user14645
  • 1,530
  • 2
  • 12
  • 17

4 Answers4

7

You CAN administer RDS using Ansible with rds, rds_param_group and rds_subnet_group modules. They do not rely on installing any remote management agents(ew!) and they don't ssh directly into the node. These 3 modules communicate with AWS using officially provided API calls(with the help of boto library).

Granted they might not be able to do everything that you can on your own EC2 instance running MySQL, but these modules still give you some degree of management.

Mxx
  • 2,312
  • 2
  • 26
  • 40
  • Thanks -- looking into Ansible. Have been trying to look into Ansible. Looks like their whitepaper is behind an email firewall and they're not responding... :-( – user14645 Apr 11 '14 at 23:49
6

Expanding on the good answer and comments from Nathan, I can offer some further explanation as to why configuration management tools won't work well (if at all) with SaaS solutions like Amazon RDS.

RDS is essentially "hosted MySQL". You don't get any access to the underlying operating system, so you can't install packages, poke around the configuration or tail log files. Your only interface with RDS from a configuration perspective is either via Amazon's Web Console or API, in exactly the same way as Elasticache is "hosted Memcache" and ELB can be thought of as "hosted HAproxy" - they are SaaS (Software as a Service) offerings.

As a result, traditional configuration management tools like Puppet or Chef will not work, as they need full OS-level access.

A tool that could be of use however is Amazon's own CloudFormation utility. Not "configuration management" per se, but it can be used as a replacement. Check out Amazon's documentation for more details.

If you really want to use Puppet or another configuration management tool on Amazon, you will need to run your own MySQL installation on top of an EC2 instance. This has drawbacks as well as advantages however, as you gain control at the expense of easy management, which is exactly what RDS provides - Amazon handles your backups, maintenance, high-availability etc and you just get a MySQL endpoint to talk to.

As ceejayoz pointed out in his comment, the above isn't to say that configuration management tools are worthless - I personally happen to be a massive Puppet fan - you just need to pick the right tool for your environment. Puppet is awesome at doing OS-level management. CloudFormation is awesome at managing your Amazon services. Put both technologies together and you have a comprehensive set of tools that can manage your entire infrastructure.

Craig Watson
  • 9,370
  • 3
  • 30
  • 46
  • Should people just go back to straight colo and dedicated servers? Sometimes, it seems like wrangling cloud resources is more complex than running your own gear! – ewwhite Apr 10 '14 at 22:56
  • Maybe some combination (Cloud Formation + Puppet) will be nice. But I suspect it will be a hacky nightmare. I've started a more targetted question here http://serverfault.com/questions/588237/whats-a-good-cm-tool-for-managing-a-cloud-cluster-composed-diverse-resources-ac – user14645 Apr 10 '14 at 23:08
  • Its seems there's a split between "pull" based systems (chef, puppet, salt?) that need stuff installed on the managed resource, and "pull" based systems. Ansible is in the later category. Some useful discussion here: http://serverfault.com/questions/568187/configuration-management-push-versus-pull-based-topology – user14645 Apr 12 '14 at 00:13
3

You don't have any options with RDS. All the management is handled for you. If you need more power over the database, use EC2 and install your own.

Nathan C
  • 14,901
  • 4
  • 42
  • 62
  • Well ... that's not strictly true that the management is all handled for me (though it may be a limitation of the configuration tools that they ignore the RDS use). Amazon provide some very nice APIs for configuring and managing RDS instances. In fact I already have a set of makefiles perform common setup and configure tasks on RDS and SQL-EC2 systems. – user14645 Apr 10 '14 at 20:23
  • 1
    True. The APIs let you do *some* things, but you can't manage the packages and software on RDS unfortunately. – Nathan C Apr 10 '14 at 20:24
  • Sure OK -- so the model I should have in mind with these configuration tools is that they're only good for use on the subset of systems that that the tool builders have thought to support (seems a bit limiting IMHO). In other words -- if it doesn't behave like a linux box I shouldn't expect my CM tool to be able to configure it? What if I want to configure a load balancer or network router (which may have its own web APIs) ... am I SOL here also? – user14645 Apr 10 '14 at 20:28
  • 1
    That's an entirely different question and not related to RDS in any shape or form... – Nathan C Apr 10 '14 at 20:29
  • Its been my experience that managing the packages is the easy part of this problem (a shell script works just fine - especially with the availability of on-demand instances where we can toss out an instance, create and new one an run a configure script from start to finish). Having a language to connecting all these machine, keeping track of who is the master, who is the slave, the upgrade ballet of disconnecting instances from a load balancer during upgrades, etc... I was hoping that CM tools would help me out here. Sound like they're the wrong tool. – user14645 Apr 10 '14 at 20:33
  • Sorry -- SX has reordered the discussion. What's not related to RDS? – user14645 Apr 10 '14 at 20:35
  • 1
    "the model I should have in mind with these configuration tools is that they're only good for use on the subset of systems that that the tool builders have thought to support" The problem here isn't the tools, it's that AWS RDS is a managed solution for which Amazon restricts your access. No tool is going to be able to work around that limitation. – ceejayoz Apr 10 '14 at 21:10
  • 1
    "Having a language to connecting all these machine, keeping track of who is the master, who is the slave, the upgrade ballet of disconnecting instances from a load balancer during upgrades, etc." RDS handles master/slave stuff, upgrades, multi-AZ replication, etc. If you want to do all that stuff, it's **entirely** possible with a configuration management tool... **but you can't use RDS.** Again, this is a RDS limit, not a tooling limit. – ceejayoz Apr 10 '14 at 21:11
  • @ceejayoz *Clearly* a specific tool limitation, not an RDS limitation. Facts: 1) RDS provides much flexibility for C&M, 2) I already have C&M tools to C&M RDS: make and shell scripts. 3) I have *lots* of resources to C&M that (for good reasons) prevent the installation of 3rd party software onto the resource. Examples: load balancers, routers, DNS tables, virtual machine servers, virtual network configurations, ... the list is endless. Tool problem. Its entirely *possible* to configure and manage all these things. – user14645 Apr 10 '14 at 21:36
  • Look, it's not Ansible's fault you can't even search for "RDS Ansible" and wind up on http://docs.ansible.com/rds_module.html. Ansible manages RDS just fine, and much more easily and reliably than a shell script. No tool - Ansible, bash, or manual input - is going to be able to configure RDS master/slave stuff because **RDS doesn't allow it**. – ceejayoz Apr 10 '14 at 21:39
  • ...unless you want to defining the scope of resources managed by CM software as "only those resources I can install my custom software onto.". Is that the case? The popular definition of CM tools only applies to this narrow scope? What I'm really after is a tool that's good for managing all these things ... load balancers, routers, DNS tables, virtual machine servers, virtual network configurations, managed databases systems,.... etc... etc... etc... It is absolutely *possible*. – user14645 Apr 10 '14 at 21:39
  • No, that's not the scope of CM. Load balancer? http://docs.ansible.com/ec2_elb_module.html DNS table? http://docs.ansible.com/route53_module.html VMs? http://docs.ansible.com/virt_module.html Your inability to Google is not a limitation of Ansible. – ceejayoz Apr 10 '14 at 21:41
  • I'm not saying its Ansible's fault. (Well it kind of is -- most of the docs for these tools fail to simply describe the limits of their scope to nOObs so that its clear what possible and what's not). Mostly I'm looking for a practical solution to managing clusters in the cloud. Unfortnately, they're all composed of resources accessible by diverse APIs. Sadly, I'm left with make as my tool of choice. – user14645 Apr 10 '14 at 21:43
  • 1
    Anything you can do with a shell script can be done in Ansible. If it's not handled by default, you can make a module - hell, you can make one with shell scripts. If make fits your needs and you like using it, we don't really care - go for it. – ceejayoz Apr 10 '14 at 21:45
  • If that's true -- that sounds great. – user14645 Apr 10 '14 at 21:56
1

With Salt you can use the boto_rds module, but, as others pointed out, don't expect the same level of control that you would have over a self-managed MySQL database, see https://docs.saltstack.com/en/latest/ref/modules/all/salt.modules.boto_rds.html

mastazi
  • 109
  • 6