I am doing some distributed work with RackSpace cloud servers and I am using bittorrent to distribute my files. It works surprisingly well. However, distributing the torrent files themselves are not so nice. How would you go around doing that? Right now I just scp the torrent files to the servers, and of course I could write a script that copies it to sqrt(n)
servers instructing each to again copy to sqrt(n)
but that's a pita to work it.
![](../../users/profiles/64874.webp)
- 1,665
- 1
- 16
- 25
4 Answers
Not knowing what exactly your problem is, I can recommend pscp
from parallel-ssh as a tool to upload small files to multiple servers.
You prepare a list of servers to upload to and let it know what to take locally and where to put it remotely. For example:
$ pscp -h list-of-servers file.torrent /tmp/
[1] 02:11:22 [SUCCESS] 10.0.0.21
[2] 02:11:22 [SUCCESS] 10.0.0.20
[3] 02:11:22 [SUCCESS] 10.0.0.45
[4] 02:11:22 [SUCCESS] 10.0.0.19
[5] 02:11:22 [SUCCESS] 10.0.0.2
[6] 02:11:22 [SUCCESS] 10.0.0.5
[7] 02:11:25 [FAILURE] 10.0.0.3 Exited with error code 1
![](../../users/profiles/114782.webp)
- 7,569
- 1
- 28
- 57
-
1There's also http://code.google.com/p/pdsh/ – chx Oct 31 '12 at 20:03
Given they're very small files typically couldn't you just have a 'dropbox' using HTTP with a cron'ed script doing a curl/wget with wildcard?
![](../../users/profiles/1435.webp)
- 100,240
- 9
- 106
- 238
-
2This is the method I would use, no need to push a file out to each server and maintain a server list, have them periodically check for new .torrent file – jwbensley Oct 31 '12 at 17:41
-
2
You could have the torrent clients check an rss feed for new torrent files or switch to magnet links instead of torrent files.
![](../../users/profiles/135087.webp)
- 401
- 4
- 8
-
RSS may not be an option if he/she is using a basic command line torrent client. – Hengjie Oct 31 '12 at 22:44
-
He could you [Flexget](http://flexget.com/) with rtorrent without gui – chewbakka Nov 01 '12 at 11:07
-
-
You might want to look into murder
Murder is a method of using Bittorrent to distribute files to a large amount of servers within a production environment. This allows for scaleable and fast deploys in environments of hundreds to tens of thousands of servers where centralized distribution systems wouldn't otherwise function
.
Murder was/is developed by the folks at twitter, and they're using it daily to distribute files.
![](../../users/profiles/134587.webp)
- 866
- 4
- 10