0

I am using Twitter streaming APIs to fetch latest tweets from Twitter and respond to them if they match a certain criteria (a particular Hashtag) If deployed on one instance, it works very well as their is only one app responding to the tweets.

If I want to scale this to multiple instances, there will be many apps that would process the same tweet. A probable solution I thought was to maintain a common db where every time I pickup a tweet, before responding to them I enter it into db and other instances check if the tweet is responded or not. But it does not seems to be right as it would slow down the process and might act weird is both instances pick same tweet at same time.

Sorry if posted in wrong forum.

kanadenipun
  • 3
  • 2

1 Answers1

0

What you propose is how things should be done. Have one server or pool of servers that consumes the API and adds each new tweet to a database, and then another server or pool of "workers" that perform actions on the database entries.

You may also consider a work queue as well to pass messages on new tweets to your workers.

If implemented well, the additional complexity will only add a handful of milliseconds to your response, which likely won't matter for this use case.

EEAA
  • 108,414
  • 18
  • 172
  • 242