3

I have come across so many answers to this question that I am completely lost! I am moving our 2 sites to a load balanced ec2 system with scalr as our cloud manager. Now the question is coming up about persistent storage for the user's uploaded content and other files. Could someone please give me a suggestion and possible a link to a tutorial for the following setup and goals. 2 websites (1 Forum, 1 ecommerce). 1 LB 1 App server (to scale out to as many as needed) 1 DB server (to scale out to as many as needed)

Our sites will need to autoscale and according to what I am learning about scalr, that means as new instances load up, I need to run a script to set the basics up on that server (git,php mods, pull site from git, move keys, etc)

What I don't understand is how should I handle user uploaded content like profile pictures, avatars, product images, themes, etc...

Do I mount an EBS or s3fs folder to hold the websites (maybe /var/www/websitefolder) or do I do something like mount the avatar folders /var/www/websitefolder/images/avatars)

I am not sure where to go with this. Could someone give me some detailed help? -John

john h.
  • 135
  • 1
  • 5
  • Would avoid s3fs and work directly with S3 APIs. Once things get busy it will be much better. Traditionally, file assets like pictures, avatars, product images are stored on S3. But for things like database storage, it's a different question. – SilentSteel Jul 05 '16 at 15:49

1 Answers1

2

This is a common issue when moving to EC2. Your options are one of the following:

  1. S3

    • Use s3fs as mentioned but you may suffer performance issues
    • Rewrite the application to store user content on S3 rather than disk.
      • You application could proxy the request to S3 or
      • You could get your client to upload directly to S3 using pre-signed URLS.
      • User could then be directed to retrieve content directly from S3.

  2. Use a clustered filesystem such a Gluster.

  3. Use 2 x dedicated NFS servers in Active/Passive using a DRBD backend.

EDIT 20160705

AWS now offers Elastic File System (EFS) in a number of regions. EFS is a hosted NFS service, effectively giving you a NAS as a Service.

Alastair McCormack
  • 2,184
  • 13
  • 22
  • Thanks, couple of followup questions: 1. If I go with the s3 route and allow uploads to the bound s3 folder but user/client downloads are done via cloudfront. Should that be decent for performance. Can all the instances be bound to the same s3 bucket at the same time? 2. I am not familier with Gluster, do i need an extra instance with it to run off of? Can I attach it to S3? 3. Is there any real options for an EBS use and if so how? My main goal is that new instances load up identical to the first. – john h. Oct 21 '12 at 19:33
  • @johnh. 1) a) The performance should be fine and will no doubt be faster than you could serve through your app. b) AFAIK, yes 2) Gluster works like RAID 1 disks mirrors where you can increase the number of mirrors as more app nodes join the cluster (http://gluster.org/community/documentation/index.php/Gluster_3.1:_Expanding_Volumes) or you could use two dedicated gluster servers. 3) EBS only allows you to attach one VM to one EBS at any one time. You can not use an EBS without an EC2 VM attached. – Alastair McCormack Oct 21 '12 at 19:50