5

I noticed that, after a new Amazon AWS EC2 spot instance is auto-launched (I have that option turned on), the state of filesystem is not current (represent the last state before previous spot instance is auto-shutdown due to current price going over my set limit). I guess the initial state of newly launched spot instance's filesystem is based on the AMI that I use to auto-launch the spot instance. I have the following questions:

Is it possible, and, if yes, what can be done, to automate saving the state of filesystem before the auto-shutdown and restoring it during the auto-launch of a new spot instance? If it's not possible, at least, is it possible to automate downloading and running shell scripts that would bring the system to a desired state (auto-update the system via apt-get, get the latest source code via git, etc.)?

Thank you!

Aleksandr Blekh
  • 308
  • 1
  • 3
  • 12

2 Answers2

7

You can launch your spot instance with a persistent EBS root volume (or attach a separate EBS volume post-boot). The EBS volume will remain after your spot instance terminates.

See http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts-spot-instances-applications-ebs.html

jstell
  • 536
  • 2
  • 4
  • Thank you! I don't have enough reputation on ServerFault yet to upvote/accept the answer, but when I'll do, I'll make sure to return to all answered questions and give the appropriate credit. In regard to your advice, I actually have done (or should I say "have tried") that when configuring my spot instances (via Web interface): I've set Delete-On-Termination option unchecked. It doesn't seem to work as expected, unless I miss something. Any ideas? – Aleksandr Blekh May 21 '14 at 21:24
  • 1
    Unchecking The Delete-On-Termination will just ensure that the root volume created for that spot "session" won't be automatically destroyed. Subsequent "sessions" will still create their own root volumes. I suppose you really want your data in a separate EBS volume that is auto-attached at spot startup. See http://serverfault.com/questions/448043/auto-attach-ebs-volume-to-a-new-spot-instance – jstell May 23 '14 at 14:41
  • I should also mention, in response to your question about automatically running apt-get / git, you can pass a shell script in user data at launch that will take care of all that. – jstell May 23 '14 at 14:43
  • Appreciate your today's comments as well! So far I only have some experience with AWS' GUI configuration, but not with the CLI stuff. I hope to try your recommendations in the future, when/if I have more time. – Aleksandr Blekh May 23 '14 at 15:13
1

Spot instances (and really all of EC2) are optimized for use cases where you don't need to maintain state. When you configured your spot instance request, you provided an AMI id which each instance uses to boot. As you've found, changes do not get committed back to that AMI.

If you truly need a shared filesystem, configure a long-running (non-spot-instance) server in EC2 and export an NFS filesystem from there to your spot instances.

If all you're really needing to do, though, is run a few commands on boot to get things updated and to deploy your code, that can be done through the user data text that you can configure in your spot instance request. Just write a shell script to do what you want, then supply that as your user data in your spot request (or any other EC2 instance launch configuration). That script will get executed as root when the instance boots.

EEAA
  • 108,414
  • 18
  • 172
  • 242
  • Thank you! I don't have enough reputation on ServerFault yet to upvote/accept the answer, but when I'll do, I'll make sure to return to all answered questions and give the appropriate credit. In regard to your advice on NFS filesystem, do I need to have an on-demand instance to be running all the time I want to use exported NFS, or I can keep it stopped all the time (inactive) and just use it as a filesystem image for my spot server? I actually one of my on-demand instances as a base for the spot one, but I didn't export filesystem as NFS. – Aleksandr Blekh May 21 '14 at 21:18
  • NFS would require that the server be powered on. – EEAA May 21 '14 at 21:19
  • Thanks! I'll keep this in mind. However, it defeats the reason I'm using spot instances for, which is to minimize the expenses (accepting in return the risk for my instances to be shut down at any time). – Aleksandr Blekh May 21 '14 at 21:27