5

I am starting to experiment with websockets.

Does anyone know of a websockets server (open source or paid) that provides a durable store of the websocket "channel"? All of the examples that I have found do not address durability -- if a websockets server goes down, all "channel" data is lost.

Services such as Pusher do not really discuss whether they address the durability issue (and I have not received a response from tech support yet).

Happy to roll my own, but would rather not reinvent the wheel.

EDIT:

I'm not looking for websockets 101 information. That is readily available and understood.

I'm looking for a server (open source or paid) that supports websockets and has a durable store for the websocket data so that, in the event that a server fails, a new server can take over where the original one left off.

Two main purposes:

  1. support failover scenarios contemplated by the websockets Network Working Group https://datatracker.ietf.org/doc/html/draft-ibc-websocket-dns-srv-02#section-5.1 (most importantly so that missed messages are sent when a client connects to a failover server)
  2. support scenarios where new subscribers must receive all past messages that were published.

Of course this can be handled at the application layer...but that is not what I am looking for.

EDIT

So, after some research the following installed options seem to be the most robust:

Hosted services that seem "real"

  • Pusher (great API but no history feature yet)
  • PubNub (has history)

All of the above services have graceful fallback to other communication methods if websockets are not available.

I was not able to find any open source that provided "out of the box" clustering, fail-over, and a durable message store to play back history. There are some projects that may serve as good starting points, but not exactly what I am looking for.

Community
  • 1
smitchell360
  • 61
  • 1
  • 6

5 Answers5

3

Hornetq is a general message broker that supports JMS, STOMP and Websockets+Stomp. It provides durability of messages along with a host of other features.

Craig Skinfill
  • 31
  • 2
1

What do you mean by channel data?

It should be the duty of an application that runs within the server to take care of the data (think LAMP).

Just think of how much data is saved when you make an HTML file with a form that posts to itself and just put that in a plain apache. All "form-data" is lost, after every single request.

EDIT:

You are on the wrong track.

Websockets don't have any relation to persistence or durability, neither does SMTP, HTTP or IMAP. It is just a transport description. (Heck, not even syslog is talking about persistence in the RFC)

I don't know what you're looking for but I'm pretty sure that it is not durability of the bytes sent or received by a websocket, rather it is durability of the data constructed after the bytes have been sent.

This problem has been solved a couple of times, I'll just reference the standard RDBMS frameworks:

  • Hibernate (java)
  • SQLAlchemy (python)
  • ActiveRecord (ruby)

If you only send JSON you can just as well use some non-relational store like Riak, Redis, MongoDB, CouchDB.

Of course it's up to you to parse the data and create someting from it that will work for your setup. I hate to say it but saving what's sent over a websocket without knowing the meaning of it is just about the same as taking a tcpdump and then reading it with ed.

May I suggest to migrate this question over to stackoverflow I don't think that serverfault is the right place for framework architecture and development.

Martin M.
  • 6,428
  • 2
  • 24
  • 42
  • Indeed. That is my question. Does anyone know of an open source or paid framework that has already solved the durability issue? It will not be difficult to solve -- I just want to avoid reinventing the wheel. Sounds like you do not know of an open source or paid framework ... which is fine. – smitchell360 Jun 19 '11 at 19:55
  • No I don't know of a framework. I wouldn't even now how to write one without a proper use case. Are you streaming stock data, base64 encoded Jpegs or uuencoded dvd images? maybe you are a site where movies are streamed -- please note I didn't make any indiciation of the direction of the stream, maybe you are just creating yet another chat server. Your question just doesn't make any sense to me sorry :) – Martin M. Jun 19 '11 at 20:26
  • Thanks for the info. I'm looking for or will build a more general purpose framework that could be used in all of those scenarios (chat, stocks). The design is fairly straight forward (though the devil is always in the details). The websocket server persists the messages published and, if a websocket server fails, a new one can spin up and reconstitute. Websocket clients can use the failover strategies specified in the current websocket drafts. We are currently doing this with a CQRS / Event Sourcing project. Not sure why the question didn't make sense to you :) – smitchell360 Jun 19 '11 at 20:55
  • Ahhhhh. I think I understand the issue. I am not asking about websockets as a protocol. I am asking about a "websockets server." Stated differently (better) a "server and related framework that implements websockets and has a durable store for the information published, etc." I have found a few projects in the past 2 hours. I will move over to stackoverflow if there are no responses to this by the end of tomorrow. – smitchell360 Jun 19 '11 at 21:31
  • Oh...and yes, I will comment / reply with links to these projects after a bit of vetting. – smitchell360 Jun 19 '11 at 21:31
  • Heh, confusion all over the place :) – Martin M. Jun 19 '11 at 21:53
  • Hopefully not everyone is / was confused :) I've found some good solutions. Still vetting them a bit. – smitchell360 Jun 20 '11 at 01:08
1

So, after some research the following installed options seem to be the most robust:

Hosted services that seem "real"

  • Pusher (great API but no history feature yet)
  • PubNub (has history)

All of the above services have graceful fallback to other communication methods if websockets are not available.

I was not able to find any open source that provided "out of the box" clustering, fail-over, and a durable message store to play back history. There are some projects that may serve as good starting points, but not exactly what I am looking for.

smitchell360
  • 61
  • 1
  • 6
1

To address the part of the question which asks if Pusher is durable I can confirm that if a user that was connected to the Pusher service and the WebSocket server that they were connected to died that they would be automatically reconnected to another WebSocket server. Channel data only exists within the WebSocket server as transient data and we have an entirely separate system which allows our WebSocket servers to distribute and consume information from connected clients. For the moment we do not persist information within Pusher. The data is simply distributed between connected components such as our REST API and connected clients.

At the moment if messages were sent during a client reconnect that user would miss those messages. However, we are implementing channel history which would mean that we will persist data within channels. This would mean that any messages that a user has missed can then be retrieved. Since our WebSocket servers would not deal with the persistence of data if we lose a WebSocket server we would not lose the channel data.

leggetter
  • 111
  • 3
1

PubNub - http://www.pubnub.com/ - provides channel durability automatically. Automatic means: When 3G/4G/WiFi drops, upon re-connect missed messages are delivered; automatically. This is valuable for mobile clients on the go with spotty network connections. Additionally PubNub provides an API to check the history for past messages sent.

PubNub is free and you don't need to signup for accounts to use it. There are optional for-pay features available.

Check out the latest PubNub Powered mobile app launching soon: http://www.projectmos.com/

Stephen Blum
  • 11
  • 1