Need a "virtual FedEx" -- some way to transfer massive files cross-country

My company has offices in NY and LA, and we spend a lot of money FedExing hard drives back and forth.

Is there some service that would allow us to save money by uploading the files at either end over a fast connection? Like 100 Mbps or more?

We don't need a 'dedicated' connection; we'd like to pay by the hour or GB.

Most of the files are 100-200 gigabytes.

FIOS would do the job; unfortunately, it's not available at either location.

Thanks!

UPDATE: Seems like I'm not making the problem clear -- we're not looking for a software solution; the bottleneck is our physical internet connections which max out at around 5mbps upstream.

We need some facility that will allow us to 'rent/lease' a high-speed (50 Mbps+) internet connection.

Sorry, maybe this is the wrong place to ask.

Dan

Posted 2012-02-22T23:22:24.517

Reputation: 1 716

2Hire interns with access to major university computer labs at each end... – Spiff – 2012-02-22T23:29:14.713

2Frankly, I suspect it's cheaper to just post the drives. This is sort of the cost of doing business on opposite coasts (and is a reason small companies usually DON'T do business on opposite coasts) – Shinrai – 2012-02-22T23:41:07.830

5we spend a lot of money FedExing hard drives back and forth. Reminds me of the old truism, "Don't underestimate the bandwidth of a UPS Truck." It often is the easiest way of moving huge amounts of traffic. Big pipes = Cubic Dollars. Renting/Leasing off site high speed still means you've got to get it there. – Fiasco Labs – 2012-02-22T23:42:24.710

how would you get the data to your facility? – Journeyman Geek – 2012-02-23T00:38:18.293

Is there any repetion of what you are transferring, it is it a new unique 100-200 Gb every time? Why is the peak rate so important that you want max transfer for only a little time each day? Some telcos may be able to provide a burst mode agreement – barrymac – 2012-02-23T01:07:48.387

It's big old ProRes Quicktime files, uncompressible, unique. Maybe 2 or 3 times a week. – Dan – 2012-02-23T01:16:51.177

maybe this is the wrong place to ask. You are probably right. I think this fits in the shopping recommendation category of being off-topic on pretty much any stackexchange site. If you are in LA and NY, maybe you could put host some equipment in a COLO somewhere. Then just go plug a drive into the server at the co-location facility. that isn't going to be cheap though. – Zoredache – 2012-02-23T02:06:02.403

Answers

I can recommend bittorent for this. If you upload your files to a tracker you can adjust download speed by adding more uploaders.

Gigamegs

Posted 2012-02-22T23:22:24.517

Reputation: 1 784

Not a solution ... the bottleneck is our upload connection speed. – Dan – 2012-02-22T23:36:04.440

@Dan: You can compress it before you share it with bittorrent. – Gigamegs – 2012-02-22T23:59:53.480

These are video files, already compressed. – Dan – 2012-02-23T00:53:22.423

To get a 100Mbps connection speed you need to talk to your ISP. I have AT&T and I currently have a 50Mbps line burstable to 100Mbps. So we can scale UP when we need to. This didn't come free, and we certainly had to talk to our AT&T rep to get this prepared ahead of time. The time it took AT&T to implement this was about 2-3weeks. Otherwise you can do what google does. Overnight Fedex external hard drive.

cohortq

Posted 2012-02-22T23:22:24.517

Reputation: 183

100-200MB shouldn't take very long to upload. I'd say at the very most, it wouldn't take more than a few hours, and that's much much faster than waiting for a physical thing to be shipped.

You could use a free service like http://www.dropbox.com/ or http://skydrive.live.com/

JacobTheDev

Posted 2012-02-22T23:22:24.517

Reputation: 608

100-200 GIGAbytes, not MB. Problem is not software, it's hardware. – Dan – 2012-02-22T23:36:30.460

Ah, makes a hell of a lot more sense lol. – JacobTheDev – 2012-02-22T23:39:06.850

-1

I would install rsync and run it almost continuously. 100GB takes 10000 seconds on 100Mbit i.e 3 hours - faster and cheaper than fedex

ZaB

Posted 2012-02-22T23:22:24.517

Reputation: 2 365

2We do NOT HAVE 100mbps; more like 5! That's the point, where can I go to 'rent' a 100mbps connection? – Dan – 2012-02-22T23:35:31.227

Go buy it? Why you said you can have it? At one fedex shipping you can have T1 for a month or so... – ZaB – 2012-02-22T23:47:35.403