3

My problem is to process images through ec2 instances.

Based this similar question: Separate server for Resque workers

And in this article: http://highscalability.com/blog/2009/11/6/product-resque-githubs-distrubuted-job-queue.html

I'm planning to have distributed workers around ec2 instances, lets say 3. But I havent figured out the right way to do this.

My doubts are:

  1. I have to install resque on each amazon ec2 instance?
  2. If yes, they have to share a single redis server or each one can have their own?
  3. Is that right to use SQS and each intance will listen this global queue and process my incoming files?

Thanks a lot if you guys can help!

Community
  • 1
Luccas
  • 101
  • 5

1 Answers1

0

You will need to install resque to each instance, but they can use a single Redis server to pull work from. The Resque pool-master on each instance will track the list of queues it's configured to poll, and will spin up workers based on what it pulls.

sysadmin1138
  • 131,083
  • 18
  • 173
  • 296
  • But is that a way to access a url of resque and it gives me all the view of workers per instances? With one redis server and accessing a ec2 instance, with 'http:amazon...ec2-.../overview', is already possible to see all worker through instances? – Luccas Jul 18 '12 at 09:12
  • @Luccas That would be resque-web, and it does indeed give you that list of who is doing what with how many you seem to be looking for. We put it on the Redis server, which also has a resque install. – sysadmin1138 Jul 18 '12 at 12:01
  • Thanks! My doubt now is how to maintain all this instances. Because I can launch one instance and then create 10 "launch more like this" with resque installed. But suppose that I changed my code, I have to remove all 10 instances and launch again? – Luccas Jul 18 '12 at 21:31