0

I expect to host a website with many images, and I mean many - around 300-400k.

I am wondering if the number of items on my server affects access time? In other words, will GET requests be slower because I host so many images? Is saving images in plain folders even the way to go? Should I use some other method of organising images, such as databases?

The main concern here is how fast the client can get images from server.

Thanks for help

sanjihan
  • 113
  • 5

1 Answers1

2

Yes. Although, there are many variables to performance.

There are file system specific limits on the number of files per file system. Usually a few hundred thousand is tiny in comparison, but know the limit.

One challenge is the enormous directories that contain hundreds of thousands of files. Not the total size of the files, but the directory entry itself. This can drastically slow down listing the directory.

Files can be spread across a directory structure, perhaps based on some kind of hash from them. So the file with SHA1 da39a3ee5e6b4b0d3255bfef95601890afd80709 goes in d/da/da3/ as deep as you want.

Or, send them off to object storage and let that system deal with files in its own way. This could be distributed storage you build or a cloud that does it for you.

In either case you likely can put it behind a content delivery network. CDNs have POPs closer to your users and can cache things.

John Mahowald
  • 30,009
  • 1
  • 17
  • 32
  • We made the mistake of storing more images than 300k-400k in single directories, we have quite a few of these directories too. I say a mistake, it doesn't affect website performance, but if you try to do manual tasks in the directory it is cumbersome. I would suggest that you create different sizes of the images so you are only serving up the size needed and not resisting big images in browser just to save on disk space. – captain_G Apr 09 '17 at 00:57