How do I change a third party service password without stopping my service that depends on that service?

Question

Here's my scenario: I have a service that has to run 24/7 and depends on a third party service that has a password.

I need to have that password changed. If I blindly change the password my service no longer works because it will continue using the old password until reconfigured. No matter how fast I reconfigure my service there's a period of time when the third party service has password changed and my service is not yet reconfigured.

How is this scenario typically handled?

score 7 · Answer 1 · answered Oct 01 '13 at 12:31

7

How is this scenario typically handled?

You schedule downtime and perform the maintenance required. Seriously. Expecting your service to be available 100%, 24x7x365 is not reasonable.

answered Oct 01 '13 at 12:31

EEAA

108,414
18
172
242

+1 Even with the implied February 29th maintenance window the expectations are unreasonable. High Availability generally means five 9s (EG: 99.999% uptime [less than an hour a year]... and this is the kind of uptime that people pay silly and obscene amounts of money for) – Daniel Widrick Oct 01 '13 at 12:49

score 4 · Answer 2 · answered Oct 01 '13 at 12:36

This situation should almost always be handled within a scheduled maintenance window. Dont have one? Schedule them (see the plurality there? maintenance windows are a good things).

Anything else is 'unexpected downtime' and your service needing to run 24/7 suggests that unexpected downtime is unacceptable. I do hope you have accounted for issues of outages (power/network/etc) and hardware failures.

How do I change a third party service password without stopping my service that depends on that service?

2 Answers2