0

I have 2 nodes, one gateway and one worker.

Data must be passed from gateway to worker (without direct connection).

I would like to store a Key into a database identified with an ID and a Time To Live of 30 seconds.

The protocol is very simple : 

- Gateway stores (Id, Key) to the database with a 30s TTL
- Worker will fetch (if needed) the key with the Id it has in it's possession and remove it directly
- If The key is not needed within 30 sec. it is deleted by the database (90% of the keys should be retrieved)

The insert, retrieve and delete must be really fast, and handle something like 30.000 per seconds easily. (With a quit descent configuration of course)

Data does not have to be permanent, if something goes wrong I don't care about data.

Would be nice if the TTL actually delete the row since it might become huge if not.

Id and Key are simple strings.

Which system would you recommend ? Knowing that I'm using Erlang for both gateway and worker

I'm not looking for message passing system like RabbitMQ or anything, since it seems impossible to get a specific key, without retrieving others.

TheSquad
  • 395
  • 1
  • 3
  • 10
  • 1
    Seems like a job for NoSQL honestly, as you won't be using much of the ACID functionality and overhead of SQL. – NickW Aug 12 '13 at 11:46
  • Yeah but which NoSQL solution ? there is plenty, and each have their advantages. For example I'm pretty sure this is not a fit for Cassandra... – TheSquad Aug 12 '13 at 11:51
  • @TheSquad looks like the task for Redis or Memcache, both have TTL capabilities. If you're using Erlang the first DB to look at would probably be Riak. – synapse Aug 12 '13 at 11:59
  • @synapse Thanks, I though memcached was a good fit, just wanted more feedback, I'll give it a try. – TheSquad Aug 12 '13 at 12:02
  • Questions about software design and architecture can usually be asked at our sister site [programmers.se]. – Michael Hampton Aug 12 '13 at 13:48
  • @MichaelHampton Thank you, I'll keep that in mind ! – TheSquad Aug 12 '13 at 15:36

1 Answers1

0

May I suggest using Memcached. What's good with it it's so dumb that you don't care about configuring it.

DukeLion
  • 3,239
  • 1
  • 17
  • 19