52

The IT Manager may be leaving, and it's possible that the parting of ways may not be completely civil. I wouldn't really expect any malice but just in case, what do I check, change or lock down?

Examples:

  • Admin passwords
  • Wireless passwords
  • VPN access rules
  • Router / Firewall settings
Yaron
  • 221
  • 2
  • 3
  • 15
Marko Carter
  • 4,092
  • 1
  • 29
  • 38
  • 3
    See related [back doors from previous IT?](http://serverfault.com/questions/171893/how-do-you-search-for-back-doors-from-previous-it/171924#171924) – Zoredache Aug 25 '10 at 22:14

19 Answers19

40

Obviously the physical security needs to be addressed, but after that...

Assuming you don't have a documented procedure for when employees leave (environment generic as you don't mention which platforms you run):

  1. Start with perimeter security. Change all passwords on any perimeter equipment like routers, firewalls, vpn's, etc... Then lock out any accounts the IT manager had, as well as review all of the remaining accounts for any that are no longer used, and any that don't belong (in case he added a secondary).
  2. Email - remove his account or at least disable logins to it depending on your company policy.
  3. Then go through your host security. All machines and directory services should have his account disabled and/or removed. (Removed is preferred, but you might need to audit them in case he has anything running that is valid under them first). Again, also review for any accounts that are no longer used, as well as any that don't belong. Disable/remove those as well. If you use ssh keys you should change them on admin/root accounts.
  4. Shared accounts, if you have any, should all have their passwords changed. You should also look at removing shared accounts or disabling interactive login on them as a general practice.
  5. Application accounts... don't forget to change passwords, or disable/remove accounts from all applications he had access to as well, starting with admin access accounts.
  6. Logging... make sure you have good logging in place for account usage and monitor it closely to look for any suspicious activity.
  7. Backups... make sure your backups are current, and secure (preferably offsite). Make sure you've done the same as above with your backup systems as far as accounts.
  8. Documents... try as much as you can to identify, request from him if possible, and copy somewhere secure, all of his documentation.
  9. If you have any services outsourced (email, spam filtering, hosting of any type, etc..), make sure to do all of the above that are appropriate with those services as well.

As you do all of this, document it, so that you have a procedure in place for future terminations.

Also, if you use any colocation services, make sure to have his name removed from the access list and ticket submission list. It'd be wise to do the same for any other vendors where he was the primary person handling, so that he can't cancel or mess with services you get from those vendors, and also so that vendors know who to contact for renewals, problems, etc... which can save you some headaches when something the IT manager didn't document happens.

I'm sure there's more I missed, but that's off the top of my head.

skraggy
  • 1,723
  • 13
  • 10
20

Don't forget physical security - make sure he can't get into any building - it's great that you're all over the network kit but if he can get to the data centre it's pointless.

Chopper3
  • 100,240
  • 9
  • 106
  • 238
14

We suspected that a disgruntled employee who was still in their notice period may have installed some remote-access programs, so we limited his logon account to work hours only, so that he couldn't remote in after-hours when nobody was around to do things (during work hours we could see his screen clearly so if he got up to mischief we would have known).

Turned out to be valuable, he had installed LogMeIn and did in fact attempt after-hours access.

(this was a small company network, no ACLs or fancy firewalls)

Mark Henderson
  • 68,316
  • 31
  • 175
  • 255
  • 5
    Why would you keep a DISGRUNTLED employee on through their notice period. An employee who is not disgruntled, no problem, but a disgruntled employee? That's just asking for trouble. – Jason Tan Jun 22 '09 at 16:30
  • 1
    I have a hunch based on him saying it is a small company no ACLs or fancy firewalls, the company probably could not get rid of him. The IT guy had them by the short-hairs if you know what I mean. A bad spot to be in, but I could see it happening. – Matt Jun 22 '09 at 16:41
  • 2
    He was kept because the boss was a tight-wad. If you sack someone in Australia you either have to keep them on for 4 weeks at full pay, or pay them 4 weeks pay all at once and get rid of them. He didn't like the idea of paying someone for 4 weeks of pay and not getting anything returned from them. – Mark Henderson Jun 22 '09 at 21:15
  • 7
    I had not even considered this idea, that the employee may not be in the US. How self-centered my point of view can sometimes be. – Matt Jun 24 '09 at 17:35
11

Also be carefull not to lockdown too much. I remember a situation where someone left and a day later it became apparent that some business critical software was actually running under his personal user account.

Niels Basjes
  • 2,176
  • 3
  • 18
  • 26
  • 4
    Been there done that. If our sysadmin ever leaves we're going to be in real hot water cos a lot of services are set to run under his account. Bad practice, I know... – Mark Henderson Jun 22 '09 at 09:47
  • 2
    I you know that why don't you tell him/her to change these fatcs? – Martin M. Jun 22 '09 at 10:30
  • Use any breakage as a result of this as an opportunity to move all the services to dedicated service accounts. – tomfanning Jun 22 '09 at 11:46
6

Just to add - also make sure you've got auditing of failed and successful logins - bunch of failures for an account followed by success could equal hacking. You might also make everyone else change their passwords too if the IT Manager was involved in password settings. Don't forget database passwords too and you may want to scrub his/her email account for secure information. I'd also put access checks on any confidential information/databases, and disallow him/her to perform system/database backups.

Hope this helps.

Paul Randal
  • 7,184
  • 1
  • 35
  • 45
  • 1
    Yes but getting people to change their passwords would be a issue, although i guess you could set the all the accounts to require change on next login and tell everyone that a server update is forcing (no one likes being made to change passwords, espically end users) It's also a good time to perform a audit of all users accounts (both local machine and network) that exist within the business. – p858snake Jun 22 '09 at 12:40
6

Make sure too, before you let this individual go, to understand that things can and will go down, or be problematic until you replace that individual. I would hope that you won't blame them for everything that goes down just because you assume/know it wont be a good parting of ways, or think they are hacking you somehow because the toilet overflowed.

Hopefully that scenario sounds preposterous to you. But it is a true story from my last job that now the owner is trying to sue me for sabotage (basically because I quit and they aren't willing to actually pay anyone the market rate to replace me) and cyber crimes such as hacking and internet racketeering.

Bottom line is, evaluate the "why" for the reason of their dismissal. If it is anything other than economical needs, I suggest you refine your hiring procedures so that you can hire a more professional individual in which, by profession, needs to be reliable and trustworthy with business mission critical and usually confidential information and who can install proper security procedures that everyone must follow.

One way to know as you are interviewing is how well they are interviewing you and your business in return. Liability (As in what the company thinks the IT Manager can be held at fault for should something go wrong- usually would be in a contract) and overall network security is one of the 3 top things on any proper IT manager/CTO's mind when coming in to interview for a job.

Jason
  • 61
  • 1
5

Change all admin passwords (servers, routers, switches, remore access, firewalls) Remove all firewall rules for remote access for the IT manager. If you are using security tokens, disassociate the IT manager's token(s) from all access. Remove TACACS access (if you use this).

Make sure to do these changes with the IT manager in a conference room or otherwise under physical control, so s/he can't observe the process. While reading a poassword as it's being typed on a keyboard is non-trivial (not hard, just not trivial), if this needs to be repeated, there's a higher risk of teh password being gleaned.

If possible, change locks. If keys can be replicated (and in short, they can), this will stop the IT manager from gaining physical access afterwards. Disable any passcard you cannot account for (not only card(s) you know have been issued to the IT manager).

If you have multiple incoming phone lines, check ALL of them, to make sure no unknown devices are attached to them.

Vatine
  • 5,390
  • 23
  • 24
3

Check the firewall policies
Change the admin password and check for accounts that are no more in use.
Revoke his/her certificates
Backup his/her workstation and format it.
Use checksum controls for the important files on your servers and put an IDS to a span port in your rack for while.

Just my 2cts.

Maxwell
  • 5,026
  • 1
  • 25
  • 31
3

Check for extra accounts, too. He could easily add a new account once he knows he's leaving. Or even soon after he arrived.

pgs
  • 3,471
  • 18
  • 19
  • I've seen that. We removed admin access to the servers at this guy's location only to find a user named JBond logged in at the console. Poor Jame's account had its admin access removed as well. – Mitch Jun 22 '09 at 12:39
3

It depends how paranoid you are. Some people go to the extent - if its bad enough - of replacing all keys and locks. Another reason to be nice to sys admins ;)

All the aformentioned advice is good - another one is even possibly getting all users to change their passwords (and if Windows) enforce the complex password policy.

Also - if you've ever done remote support, or setup a remote office / client (ie. another site) - get them to change their passwords too.

3

Don't forget to blow out any extranet type accounts that he might have on behalf of your company. These are often overlooked and often the cause of much grief post-mortem.

Might (along the "I'm ultra-paranoid" track) want to also notify your sales reps for different vendors that you work with in case he tried to contact someone there .

squillman
  • 37,618
  • 10
  • 90
  • 145
2

If he had any control of your company web-hosting,

  • recheck all access paths through the web pages
  • get all code validated for possible back doors

Weaknesses in this area can impact based on the way your hosting is done,

  • Caged hosting with administrative control -- in the least, possibility of a defaced site
  • Local hosting from your premises -- access to internal network (unless you have a DMZ which is also locked-down)
nik
  • 7,040
  • 2
  • 24
  • 30
2

My company let a developer go not too long ago and it was a similar situation. He knew much of the system and it was of the utmost importance to ensure he was cut off the second he was informed of his dismissal. Asides from the advice given above I also used Spectre Pro to monitor all his work for the 2 weeks prior to his leaving: network activity (IO), chat windows, emails, screenshots every 2 minutes, etc. It was probably overkill and I never even looked at any of it because he left on good terms. It was good insurance though.

JohnyD
  • 1,614
  • 2
  • 20
  • 29
2

The two keys things to manage immediately are:

  1. Physical access - if you have an electronic system, revoke his card. If your locks are all physical, either ensure that any keys issued to him are returned, or if you are really concerned about mischief, change the locks to critical areas.

  2. Remote access - ensure that VPN/Citrix/other remote access account of this admin are disabled. Hopefully you are not allowing remote logins with shared accounts; if you are, change the passwords on all of them. Also be sure to disable his AD/NIS/LDAP account.

This just covers the obvious however; there is always the possibility for example that he has installed a couple of modems in the server rooms, with console cables into key network devices/servers. Once you have done the initial lock down, you probably want his replacement to do a full sweep of the infrastructure to A) make sure the documentation is up to date and B) highlight anything that looks odd.

Murali Suriar
  • 10,166
  • 8
  • 40
  • 62
2

At a previous job at a smaller company, the sysadmin being let go knew a lot of other employee's passwords. The morning he was let go, we set the "user must change password" property on anyone's Active Directory account that had remote access.

This may not be feasible everywhere, but may be prudent depending on the situation.

Charles
  • 91
  • 2
1

I would recommend following procedures:

  • disable all building security access cards
  • disable all known accounts (especially VPN and accounts that could be used from outside the company)
  • disable unknown accounts (!)
  • change all admin passwords
  • review firewall rules

This should cover most of the possible access options. Review all security relevant information in the following weeks, so that you can ensure no option was left "open".

Dieda
  • 41
  • 1
1

Make all staff aware that this employee is leaving so that they aernt as vulnerable to telephone social-hacking attempts.

He already knows how the system works and whats there. So he wouldnt need too much info in order to get back in if he desired.

If I left to day under less-than-desirable circumstances I believe that I could call up staff, which I have to do from time to time anyway, and find out enough info to get back into the system.

Maybe I would give an existing domain user admin privileges (before leaving). I could phone up this user and have him/her reveal her password to me.

cop1152
  • 2,626
  • 3
  • 21
  • 32
  • 2
    Hence the reason when auditing your network post-termination, you check the Domain Admins group and make sure there aren't any people that shouldn't be in there. "Steve from Marketing is an admin, wtf?" – phuzion Jun 23 '09 at 12:45
1
  • Disable their user account in Active Directory. Check for any other accounts that the IT manager might know the password of and change or disable them.
  • Disable any other accounts not part of Active Directory either because they're on a different machine or because they were written in-house. Get legitimate users to change their password. (I can still log in as admin to another employee's account to this day.)
  • If your company web site is hosted outside the building, change the passwords for that too.
  • It may also be fairly trivial for a disgruntled employee to get your Internet and/or telephone service cancelled. Not sure how to defend against that though.
  • Change the locks AND the alarm code. A break-in could go unnoticed long enough that they steal all your stuff.
Scott
  • 1,173
  • 3
  • 13
  • 25
-1

The only way to be totally safe in the case of servers, is the same way you ensure that a hacked box is clean: reinstall. Thanks to puppet (or some other configuration management system) reinstalling servers and getting them to a specific state can be quite quick and automated.

ptman
  • 27,124
  • 2
  • 26
  • 45