0

I have been banging my head for months over this, no answer, no info, no "cure".

my AWS ELB listener "randomly" forgets its SSL settings and reverts to this on its own, at random times, and without any human intervention. Every time I fix it it will hold for 3, 5, 10 days and then boom -- back to TCP, taking the whole site offline. I am desperate to know what the hell is going on.

This is what it reverts to on its own:

enter image description here

this is what I fix it to again...and again...and again... only to see it revert randomly to this non-working state time and again.

enter image description here

This is NOT a security breach. We have verified that by going over logs, changing passwords and limiting to a single admin account assigned to one person. This seems like an AWS issue for which we have no answer. We're not going to hire a "night guard" to watch the ELB admin panel to see when AWS forgets the SSL cert on its own...... we're seriously considering ditching AWS for this showstopping issue.

JasonGenX
  • 522
  • 1
  • 5
  • 16

1 Answers1

2

  1. I'd bet that you've got some kind of automation messing with the settings. Some ansible, terraform, lambda or a custom script scheduled to run from time to time that reverts the ELB config to what (presumably) was the original state.

  2. Check CloudTrail logs at the time it happens - you will see which IAM User or IAM Role made the changes to the ELB.

  3. You can also enable AWS Config which essentially creates a catalogue of all your resources and tracks the changes. It can alert you whenever a resource changes (via SNS or email).

  4. To catch any issues with the ELB as soon as they happen and break the site you can also set up CloudWatch Alarm on the ELB traffic or connection errors or some other metric that changes when the issue re-appears. It's always good to have monitoring set up, however fixing the underlying issue with one of the above methods is indeed preferable.

Hope that helps :)

MLu
  • 23,798
  • 5
  • 54
  • 81
  • Absolutely this. AWS CloudTrail is the first place to look to see what is being changed and who's changing it. What's described is not typical AWS behaviour. AWS paid support is also excellent, and relatively reasonably priced if you don't have massive AWS bills, they can help with this sort of thing. – Tim Aug 09 '20 at 21:56