0

I’m working in releasing a video hosting website (files are from webcams) Each file size will be less than 1 mb. I would like to be able to easily extend my storage capacity and have high availability. I will use dedicated servers from OVH with 2 NIC (1 for WAN and 1 for private network) I will need to transcode video files, store user information in mysql and use apache for front website. I will start with 12 TB storage; and will need 100 TB in next 6 months. It even would be a good opportunity to aggregate bandwidth of each server. I’m thinking about virtualization with proxmox or use glusterfs for storage and use 2 servers in HA for mysql + apache. What would be the best choice?

Nathann
  • 17
  • 1
  • possible duplicate of [Can you help me with my capacity planning?](http://serverfault.com/questions/384686/can-you-help-me-with-my-capacity-planning) – Paul Apr 04 '15 at 14:29

1 Answers1

1

Virtualization doesnt really gain you much here - that is for making one big machine act like several small ones. You want a bunch of big machines. Splitting a host's physical storage across a bunch of virtual machines will just cause you headaches. It also would make it harder to use redundancy in clustered filesystems since you need to make sure they dont't end up on the same physical disks.

Your best setup, depending on the traffic would be something like:

  • Main webserver that serves the pages. This might be one machine or several behind a load balancer.
  • Database server(s), which depending on load might be the same machines as the webserver or a separate machine or cluster.
  • Storage servers. Run a lightweight web server like nginx on each and serve the files directly from there. Each would have its own URL to serve the files from.
  • Transcoding server(s) - which will take the source files, decide which storage server to store the output on, and record it in the database. Splitting this onto a separate server means it won't slow the website to a crawl while video is being encoded.

When you start these could all be on the same couple machines - as you grow split them off into separate ones.

Clustered filesystems may sound easier, but with that all the traffic goes through the main server which limits you.

I suspect you also underestimate what a huge pain it is to administer clustered filesystems if you can avoid it. Clustered filesystems are great until they break. They do have advantages in adding more redundancy. And are the only option if you need to present a large set of files as a single filesystem - but I don't think you need the added complexity for this use.

Separate storage servers are easy to find the video (simply record the url or what server it is on in the database). Deciding where to store new files could be as simple as each server running a little script that records their free disk space to a database table. a single query will tell you which has the most space and you nove it there via rsync or scp. Adding new servers is easy too - once it records itself in that table videos start going to it. If you need redundancy, pick the first two servers, copy to both, and record the primary and alternative servers for each video.

Grant
  • 17,671
  • 14
  • 69
  • 101
  • Thanks for your reply, I'm ok with that, It would just be easier for me to have 1 storage for all files that's why I was thinking about glusterfs, because it can be hard to manage all files locations even harder to know best place to store a file. For example one user send 1 file, how can I know where to place the video file? With glusterfs or similar I would just have to store it in the mounted partition... – Nathann Apr 04 '15 at 14:30
  • @nathann added section about clustered filesystems to the answer. – Grant Apr 04 '15 at 14:45
  • Thanks for your message. I think you're right. Just try to find maybe php script to manage multi servers like ispconfig multiserver environment, and use nagos to monitore all servers. – Nathann Apr 04 '15 at 19:40