Syncing files between two load-balanced servers?

3

In PHP we're creating various files on our servers. The problem is we have two servers, which are load balanced. What would be the best way to keep the created files in sync between the two? Is there some common practice I've just never heard of?

loneboat

Posted 2011-05-04T21:58:22.970

Reputation: 489

Answers

3

#1: You can let the OS do the work. Assuming you are using Microsoft Windows environment, a DFS would be the simplest solution (in terms of administrative work). You have to set up a Domain Controller, which needs an R2 version of the Windows Server (be it 2003 or 2008).

#2: Don't create files on load balanced servers' own HDD. You should separate file-handling from the webserver and use an external storage.

karatedog

Posted 2011-05-04T21:58:22.970

Reputation: 809

What do you think is in Win2k8 R2 that's not in Win2k8? – uSlackr – 2011-05-05T01:14:34.000

ie. replication groups. – karatedog – 2011-05-16T20:57:33.807

1

Shared storage is the way to go. We use NAS to make the files available to all servers.

uSlackr

Posted 2011-05-04T21:58:22.970

Reputation: 8 755

0

Well, a one way sync would be easy with rsync. What about bi-directional? I did a quick google and it seems it's possible.

http://forums11.itrc.hp.com/service/forums/questionanswer.do?admit=109447626+1304547785383+28353475&threadId=1278777

At the bottom of that link it gives you a method.

Matt H

Posted 2011-05-04T21:58:22.970

Reputation: 3 823

Be aware that there will be a delay. So if you create files on one server, there will be a delay before they end up on the other one. That may have a negative impact on your system, or it may be an acceptable tradeoff. – Matt H – 2011-05-04T22:27:43.577

0

Your load balancer should allow you to configure it so that all requests from one particular client will always go to the same load-balanced server. With this turned on, any time a file is created for a client, it's there on subsequent requests from that same client. Doesn't work so well, though, if e.g. Client A is triggering the creation of a file for Client B.

You may want to consider a database server. For example, if your created files are storing session information, store that data in a database instead, which both web servers point to (you could add replication here to keep load distributed if you'd like/need). If it's not session data, consider storing the data necessary to generate the files in this database, and only generate the actual file on-demand when requested by pulling this data from the database; might not be applicable in your situation, though.

Finally, you could consider shared and/or replicated storage. DFS in a Windows environment is a good replication solution; NFS or SMB might be applicable, or a SAN; basically everything @karatedog mentioned.

One more option to consider is a file-centric database solution, such as MongoDB, which basically takes the form of a database serving as a centralized file store. Put your files there, and either server can pull them back out again later.

Kromey

Posted 2011-05-04T21:58:22.970

Reputation: 4 377