4

I am operating a system with heavy image editing processed by imagemagick.

Now for a new server I was wondering which system requirements are most important to boost performance of imagemagick and image editing such as converting and composing gifs, jpgs and pngs.

The resolution of the images is usually between 800x400 and 1980x1080 px. Multiple images may be processed at the same time.

I was thinking that a lot of (fast) RAM would be most important but then even full hd in 32bit only takes around 8MB, so maybe CPU is more important?

Any insights are highly appriciated!

Update:
htop shows that CPU is the most probably the bottleneck.
Since imagemagick needs a lot of resources I could imagine that I would hit CPU limits with one server pretty quickly. Any ideas how I could balance the load dynamically (there could be huge peaks)? Something like Amazon Web Services but quicker since images are rendered on the fly?

horen
  • 411
  • 2
  • 7
  • 22
  • 1
    What is your current bottleneck? Benchmark, see what limits you and then improve this. – Sven Jun 28 '13 at 13:03
  • @SvW From htop it looks like the CPU would be the bottleneck. It goes up to 100% work load meanwhile RAM is under 1% on a 4 CPU, 4 GB RAM server. – horen Jul 01 '13 at 11:21
  • 1
    I think you've answered your own question. – Michael Hampton Jul 05 '13 at 07:39
  • Well, other variables also leads to CPU bottleneck. Just saying what htop reports is not the full truth. CPU could be waiting on I/O, then it's your block device subsystem that is holding CPU back. More CPU will allow you to context switch faster, to instantiate more parallel process, but it won't help anymore if your HDDs are 5400 rpm, for instance. Your bottleneck appears to be CPU but CPU waits for disk I/O, synchronously, you should dig deeper. – Marcel Jul 10 '13 at 17:43

4 Answers4

7

Processing such small images (and 1920x1080 is small) doesn't require much memory, CPU is dominant.

You need lots of CPU power, if you have many pictures to convert at a time, running the conversions in parallel is a good idea. Also, because it's easy to run it in parallel having CPU with more cores vs higher frequency will give you higher performance. You just need to make sure to run many jobs at the same time.

In other words, look for machines with minimal amount of memory (4-8GB) and many cores or even many CPUs. If you're using Amazon, then High-CPU Instances is what you're looking for. For very large load, it may be good idea to look at cluster instances.

Depending on the specific number of conversion requests, you may have better luck scaling horizontally - getting more machines and spreading your load over them. But that depends on your exact use case, collocation costs or VM costs, etc.

Hubert Kario
  • 6,351
  • 6
  • 33
  • 65
4

From htop it looks like the CPU would be the bottleneck. It goes up to 100% work load ...

If your CPU is pegging and everything else is at nominal values then you need to add More CPU.

Nijeesh Joshy
  • 61
  • 5
user9517
  • 114,104
  • 20
  • 206
  • 289
  • @lain: since imagemagick needs a lot of resources I could imagine that I would hit CPU limits with one server pretty quickly. Any ideas how I could balance the load dynamically (there could be huge peaks)? Something like Amazon Web Services but quicker? – horen Jul 05 '13 at 08:19
  • @horen, you would need to create/utilize a queueing system where workers will poll some place to look for more work to do. That way you can have exact and controlled number of imagemagik workers to utilize all the available CPUs/cores without overwhelming them. Read up on Message Queues, available products and services. – Mxx Jul 09 '13 at 20:26
4

I've done photobooth project, based on opensource software, we use ImageMagick for image processing. We've experimented with both x86 and ARM v6/v7 platforms. Different image sizes, different effects, combining of images etc. We've tested with 256 RAM for both old x86 PC and ARM (Raspberry Pi). CPU was the bottleneck and we were practically unable to use 1 core for instant processing - it was taking about 30 secs to complete on ARMv6, 10 secs on P3 and 1 sec for 1 GHz newest Core. It's a CPU. If you are running ImageMagick apps from command line, then you should check /tmp, which is usually tmpfs/shared memory. For us it was taking about 64MB per one large print image, combined from four jpegs from Canon cameras. If you are running several processes at once - just add more cores.

How to balance the load? Depends on your configuration, how your system works. Do you use web services, running ImageMagick with php or have your own programs etc... Please clarify...

GioMac
  • 4,444
  • 3
  • 24
  • 41
0

HAProxy will let you 'load balance' between multiple servers from the networking side... And provide fail over.

Daniel Widrick
  • 3,418
  • 2
  • 12
  • 26