2

I have a slow connection between two machines, one in Germany and another in China. Connection sometimes breaks apart, and the packet loss is about 5%, at times even reaching 20%. Chinese machine has an outgoing bandwidth of ~256 kbit. Transfer of machines to another location is impossible. There is a sort of replication process between them, coming from China.

So far I had good experience with tunneling specific traffic through zebedee. However it all stops on network split (which is often), and I have to manually restart it on client side.

I'm basically looking for config or tool that can buffer all traffic on client side, and push it through tunnel only when the connection is available. If not, it must be accumulated in file, and be able to survive system restart.

I'm thinking of writing a simple proxy daemon doing just that, but I feel there are some ready solutions available.

Thanks!

Michael Sayapin
  • 21
  • 1
  • A tool that can buffer all traffic over a single TCP connection? Or all traffic over a network link? (None of these things sound like good ideas.) – David Schwartz Aug 21 '12 at 09:29
  • Single TCP connection. Sort of buffering TCP proxy with automatic retry. – Michael Sayapin Aug 21 '12 at 09:35
  • I don't think you'll be able to make this work. If the system restarts, a new TCP connection will be needed. I don't see how the other side will be able to make sense out of the data starting at an arbitrary byte position. The protocol you are using on top of TCP would have to have automatic resynchronization capability, and almost no protocols have that. The other end would just be baffled by the data gap. – David Schwartz Aug 21 '12 at 09:37
  • What is the "sort of replication process"? Wouldn't it make more sense to look at how that handles interruptions and retries rather than going further down the network stack. – dunxd Aug 21 '12 at 09:44
  • Let me elaborate. I'm using ElasticSearch replication, when network disconnects, it puts all "cluster" into "yellow" state, means writes are postponed. I want to use "localhost" as replica nodes' host setting in config, both in China and in Germany. Under this there will be daemons acting as tcp proxy buffer and handling disconnections nicely. That's the outline of the idea, however maybe it would better to just leave it to ElasticSearch's protocol and handle yellow state in my app. – Michael Sayapin Aug 21 '12 at 10:50

1 Answers1

0

and the packet loss is about 5%, at times even reaching 20%

5% loss is appalling, never mind more.

when the connection is available

Unless you can see a specific pattern of packet loss, then there's no way to know how available the connection is.

I'd be looking at a network solution to a network problem - not sure what penetration providers are available in China, but you might want to have a look at Amazon (you'd need to roll your own replication or move the replication onto a AWS hosted server) and Akmai (provides alternate routing)

symcbean
  • 19,931
  • 1
  • 29
  • 49
  • Both AWS and Akamai are unavailable in China. There is a Great Firewall however, and all connections are routed through it. I solve this problem by using an SSL VPN tunnel to my servers in Germany, it's slower, but much more stable. Alas, it breaks on disconnect. – Michael Sayapin Aug 21 '12 at 18:02
  • Trying to run any sort of TCP connection across this connection is going to be really tricky. A UDP based solution might be more appropriate. A quick google turned up this: http://www.filecatalyst.com/company/faq/, although TFTP might be adequate. – symcbean Aug 22 '12 at 11:09
  • Thanks, if I could intervent into ElasticSearch replication protocol, I would use something like that or P2P like BitTorrent proto. However I think my best chance here is to emulate the second node at localhost. – Michael Sayapin Aug 22 '12 at 12:17