I am running an application on AWS Elastic Beanstalk. If an instance responds too often with an HTTP status code in the 500 (server error) range AWS marks this instance as unhealthy and removes the instance from the load balancer.
I understand this and agree that this is actually a good behavior. But unfortunately, this leads to problems with my application.
My application needs to connect to several external APIs and aggregates their data. One of the external APIs – which is not under my control – is flaky and responds quite often with a 500 status code.
At the moment if an API raises an error my application just passes that error back to the user. Causing AWS thinking my application had an error and therefore terminating that instance and starting a new server. But actually, it is only one of the endpoint causing a constant rate of 500 errors, whereas all other endpoints are still fine.
On the one hand, it is correct that an external server error causes my application to just return that error. On the other hand, this kind of external server error is not in my application and I could catch it. But even if I catch the error I cannot return anything useful to the user and therefore still need to return with an error code.
How can handle this? Avoiding server error status codes to not trigger unhealthy instances, but at the same time not using a client error status code because there is nothing the user can do, they just need to retry?
What do you suggest? Or is there another option to fine-tune AWS Elastic Beanstalks behavior?