171

Yesterday, I was performing a bit of general maintenance on a VPS of mine, using the IPMI console my host provided.

Upon setting up SSH keys again via the IPMI console, I logged in via SSH and was shocked to see this:

Welcome to Ubuntu 14.04.2 LTS (GNU/Linux 2.6.32-042stab116.2 x86_64)
Documentation:  https://help.ubuntu.com/
Last login: Sat Sep 17 04:39:57 2016 from ic.fbi.gov

Immediately, I contacted my hosting company. They said that they didn't know why this might be, and that it's possible the hostname was spoofed.

I did a bit more digging, and resolved ic.fbi.gov to an IP address.

I then ran this on the system:

last -i

This returned my IP address, and then two other IP addresses which were unknown to me. I geoIP'd these two IP addresses. One of them was a VPN and the other was a server from a hosting company in the state of Washington.

Again, the IP that I resolved ic.fbi.gov to was not on the list.

Do you think I should be concerned/worried about the "FBI" obtaining access to my VPS? Or is it just a hacker that spoofed the hostname?

lol what is this
  • 1,551
  • 2
  • 9
  • 11
  • 219
    It's plainly someone's idea of a joke. What you really should worry about is what that someone did after he got in. Nuke the entire VPS from orbit. It's the only way to be sure. – Michael Hampton Sep 18 '16 at 17:12
  • 110
    I'm with @MichaelHampton, the wrong question is being asked here. You should be concerned that someone other than you logged in. – h4ckNinja Sep 18 '16 at 20:43
  • 43
    `Do you think I should be concerned/worried about the "FBI" obtaining access to my vps?` I would be. Lucky for you, it was *not* the FBI, because a federal investigation can really eff up your life, even if you don't get charged with anything. – HopelessN00b Sep 18 '16 at 23:36
  • 33
    Also, if you (later) get a popup, usually with spelling and grammar errors, that your files have been "siezed" because they are involved in "offenses" like child porn and terrorism but you can get them back by paying a "fine" within a few hours in BTC, that isn't really the FBI either, that's some criminal(s). – dave_thompson_085 Sep 19 '16 at 09:38
  • 11
    Relevant: https://serverfault.com/questions/218005/how-do-i-deal-with-a-compromised-server – Jonas Schäfer Sep 19 '16 at 09:50
  • Would you have noticed that the date was wrong if it just showed an IP address instead? Kind of ironic. – JMK Sep 19 '16 at 09:59
  • 3
    @MichaelHampton: "It's plainly someone's idea of a joke" - possibly the NSA's ;-) – Steve Jessop Sep 19 '16 at 19:58
  • 7
    They were obviously connecting from "FBI Surveillance Van #5" – Michael Sep 19 '16 at 19:59
  • 3
    The FBI isn't stupid enough to do that. I hope... – Mark Buffalo Sep 20 '16 at 03:43
  • You have a good opinion of government organizations. Their employees don't get paid as much as private companies' workers and they have other things to worry about. I'm sorry, but you are not *that* valuable as a potential suspect to spend taxpayers money on. – wintermute Sep 19 '16 at 22:40
  • It's probably a hacker that wanted to either prank you, or maybe to try to make you think that the FBI was at fault so that maybe you didn't investigate the issue... – Bakuriu Sep 20 '16 at 11:36
  • 1
    You may even want to report this to the *real* FBI. They do care about hackers. Of course, by now you've nuked your system and the evidence is gone... Let me re-phase that. They may not care about you being hacked, but they probably do care about people pretending to be them... –  Sep 20 '16 at 21:25
  • 4
    If you are sufficiently paranoid, you will realize that it is the FBI pretending to be a hacker pretending to be the FBI. Those clever devils! –  Sep 20 '16 at 21:41
  • While it wasn't mentioned in any answers it should be noted resolving to that domain it made a great smoke and mirrors style distraction from the real issue. – Bacon Brad Sep 21 '16 at 00:46
  • 1
    `...worried about the "FBI" ...` Do you have stuff serious enough to concern the FBI? Would you be more worried about (a) the FBI, or (b) some unknown someone who can both login to your system and also poison name resolution for you? – user2338816 Sep 21 '16 at 11:42
  • @jmk What's wrong about the date? – TylerH Sep 21 '16 at 15:22
  • @TylerH Apologies I meant the part of that string showing a domain name – JMK Sep 21 '16 at 17:57
  • Trawl your access log, hopefully you rotate the last 12 months, to see how long this has been going on... check the timestamp on /etc/hosts while you are at it. Who is providing your VPS ? Do they have at least maintenance backdoor access ? I suppose the password is not one that is common to leaky websites along with a host identifying email or username, or address (cross referenced to whois) ? auth.log for the timestamp to confirm what you can..... – mckenzm Sep 22 '16 at 03:09
  • 1
    “Or is it just some casual hacker that spoofed the hostname?” Exactly. And if this was the FBI, don’t you think they did a horrible job by leaving a trail like this? Considering how man “cloud” servers there are out there, I would be quite confident that a real hack by a government agency would be carried out via a proxy that is hosted on some neutral system somewhere. – Giacomo1968 Sep 24 '16 at 21:11
  • @dave_thompson_085 but if there aren't any spelling errors or they ask for DogeCoin instead, *then* it's the FBI. – pydsigner Sep 25 '16 at 07:30
  • You should worry about your **password**!!!!!!!!!!!!!!!!!!!!!! The real problem is that **someone** logged in! – usr-local-ΕΨΗΕΛΩΝ Feb 10 '17 at 12:48

10 Answers10

296

An IP address can be set up in DNS to resolve to any host name, by whoever is in control of that IP address.

For example, if I am in control of the netblock 203.0.113.128/28, then I can set up 203.0.113.130 to reverse-resolve to presidential-desktop.oval-office.whitehouse.gov. I don't need control of whitehouse.gov to do this, though it can help in some situations (particularly, with any software that checks to make sure reverse and forward resolution matches). That wouldn't mean that the president of the United States logged into your VPS.

If someone has access to your system, they can change the resolver configuration which will effectively enable them to resolve any name to any IP address, or any IP address to any name. (If they have that level of access, they can wreak all kinds of other havoc with your system as well.)

Unless and until you verify that the IP address that was used to log in actually is registered to the FBI, don't worry about the host name being one under fbi.gov. That name mapping may very well be faked. Worry instead that there has been a successful login to your account that you cannot explain, from an IP address that you don't recognize.

Chances are that if the FBI wanted the data on your VPS, they would use a somewhat less obvious approach to get it.

You should worry, but not about the fbi.gov hostname.

Go read How do I deal with a compromised server? on Server Fault, and How do you explain the necessity of “nuke it from orbit” to management and users? here on Information Security. Really, do it. Do it now; don't put it off.

user
  • 7,670
  • 2
  • 30
  • 54
  • 133
    Or a somewhat more obvious, lawyery one. – Undo Sep 18 '16 at 22:01
  • 82
    [Like a 5 dollar wrench.](https://xkcd.com/538/) – isanae Sep 19 '16 at 06:34
  • 9
    @undo Possibly. I was thinking something like a [NSL](https://en.wikipedia.org/wiki/National_security_letter) directed at the ISP. – user Sep 19 '16 at 07:20
  • Well, these FBI people use all sorts of techniques and disguises to gather information and/or evidence about anything or anyone. While this may seem very innocent and/or accidental, make no mistake that like the Michael Kjorling said, someone successfully logged into your VPS without your permission and knowing. – ThN Sep 19 '16 at 16:16
  • 2
    You really don't need control of the IP block to set the reverse DNS for it. You just need control of the DNS server/host file/other name-resolving mechanism of the computer which is resolving said IP address. You only actually need to own the IP block if you want to set the name in the official DNS servers. The hacker could have, for example, set up a DNS server (either on your box or elsewhere,) set your computer to use it for name resolution, logged in again, and then changed your name server setting back. – reirab Sep 19 '16 at 21:49
  • 5
    If you control the DNS server, you can make any forward or reverse lookup resolve to whatever you want. Several years ago, a coworker once found that google.com resolved to a webserver that redirected him to a certain YouTube video featuring Mr. Astley. :) – reirab Sep 19 '16 at 21:51
  • 1
    To paraphrase a certain probably-not-Russian, ["God. DNS server. What is difference?"](http://ars.userfriendly.org/cartoons/?id=19981111) – Wayne Werner Sep 20 '16 at 09:04
  • 2
    @reirab Yep. A few years ago while doing remote troubleshooting for a customer in New Zealand, I found that their "loopback" was resolving to some address geo-located in Hong Kong. You can't bet your life on a _hostname_. – user2338816 Sep 21 '16 at 07:26
39

I think you MUST be concerned if anyone has unauthorized access to your server. As others mentioned there isn't much work for faking reverse DNS host name. Maybe they want you to believe it's okay for a government agency to have access to your server so you won't investigate the incident anymore.

You should backup all your server logs for later analysis and preferably rebuild your server to eliminate any risks that a compromised server could cause. After that you (with help of an expert) should setup server with security best practices and precautions.

Koorosh Pasokhi
  • 1,107
  • 1
  • 9
  • 10
21

So should you be concerned if it was the FBI, or is it ok if it was just some casual hacker? From the logs, someone successfully logged onto a host you control. It should be assumed compromised regardless of who it was. Scrap it and rebuild.

Also keep in mind that a reverse DNS entry can be created by anyone who has control of a specific IP block. It doesn't need to resolve to something they control, ie, if I control an IP block I can create a reverse entry to whoever I choose. Reverse and forward entries don't have to match, and they are often maintained by different people.

s3c
  • 311
  • 1
  • 3
11

Kill it with fire. Like yesterday.

The FBI DITU or any other cyber unit from any Alphabet Soup to include Army CYBERCOM is NOT in the business of simply accessing your system from fbi.gov, someone is playing a joke on you - no serious investigator/TF is doing something that apparent.

What you need to worry about is how someone with a higher-than-the-average-skiddie knowledge got access to your VPS and did that.

Back to the first point: destroy it.

VaeInimicus
  • 240
  • 1
  • 5
3

Normally, a hacker will try to hide his/her identity. They typically don't use an IP address for their attacks which will resolve to their true identity. So fbi.gov is fake.

On the other hand, secret services have been known to "accidentally" leak the fact that they are observing someone, in order to get the person or organization to get into panic mode, make mistakes or simply flee from the country.

Take a look out of the window. is there a suspiciously harmless looking van parked outside your building? Any guys which look like Agent Smith from the Matrix?

Probably not.

Assume that your system is compromised. The hacker may not have anything special yet, but there is a market for servers which can be used for illegitimate purposes. It might host child porn in an hour or so (if you haven't nuked it yet).

Klaws
  • 149
  • 3
3

Take it from the human perspective.

It's not from the FBI. The FBI knows better than that to log in from fbi.gov.

BUT the main point is, anybody logging into your system unauthorized like that should be investigated. My recommendation would to move your system elsewhere and replace it with a system just for forensic analysis. Throw in a honeypot to distract the hacker so you can record their moves.

noɥʇʎԀʎzɐɹƆ
  • 416
  • 1
  • 3
  • 15
  • They know better now, but even they occasionally screw up. Back almost a decade ago, when IE6 was still king, someone in the Department of Justice provisioned IE on most of the DOJ's desktops and laptops in such a way that IE had a custom string added to the user-agent string. It was possible to search web server access logs, then, and determine if someone from the DOJ had visited a site from an official computer, even from a laptop at Starbucks. Current versions of IE no longer offer the functionality that made this possible. – Michael Hampton Sep 22 '16 at 22:53
  • @MichaelHampton just install user agent switcher on chrome/firefox – noɥʇʎԀʎzɐɹƆ Sep 22 '16 at 23:14
  • 1
    @MichaelHampton adding the custom string was most likely *not* a mistake. Generally, when accessing public information held by a presumed US entity, US Government agencies do not attempt to hide the fact they are getting that information; they may try to be as explicit about that access as possible. – Randall Sep 24 '16 at 17:19
2

You have two ways:

  1. It's a hacker that has access to your credentials for logging in to your VPS.
  2. FBI have access to all hosting servers and you need to get a response from your hosting company, but I don't think so.

Analyse your backups files, and check if your configuration is secure, Root user can connect or not, have you created a specific user for ssh access or not etc.

Change your SSH password, and check and track every week if a activity has suspect on your VPS.

Anders
  • 64,406
  • 24
  • 178
  • 215
Kate
  • 121
  • 1
  • 5
    [How do I deal with a compromised server?](https://serverfault.com/q/218005/58408) on [sf] has solid advice. – user Sep 20 '16 at 07:36
2

I would be more concerned about the integrity of your system than about FBI logging in; as for DNS spoofing or defining fbi.gov reverses, alas, it is far more easier for the attacker to rewrite his "real" address with a "fbi.gov" in the address fields of /var/log/wtmp and /var/log/lastlog. The structure of the fields of those files has been documented for decades.

I would be more concerned about you being hacked, as to play with your logs to plant fbi.gov and real FBI addresses, there is a need for root access for the aforementioned files.

It seems likely your VPS has been compromised seriously, dispose of it and reinstall everything again.

Rui F Ribeiro
  • 1,736
  • 8
  • 15
1

As folks have been saying... nuke it. This is also a good reason why you should regularly back up any data not placed on a system via a re-install to another location.

Secondly, after re-installing, make sure any account which has a password has a very secure password. Use something like keepass to generate a long random string for any passwords other than the one you use for your main user. I use 16+ random character passwords even on machines which are inaccessible from outside my firewall, and my bastion hosts are now 24+, since one should never login directly as root (this does not include using authorized ssh keys), nor should one use su. If you have to for anything other than a dire emergency, you are doing something wrong.

Finally, regarding SSH keys... any keys I use across the wild west of the internet itself, I never use less than 2048 bit keys, and am mostly using 4096 bits or higher.

All this will not protect you against someone gaining root access through the backdoor via some process such as sendmail (such as I did years ago when I was responsible for UN*X at CompuServe), then changing a password, to then come in the front door, nor will it protect you against some piece of malware which you ended up running on the machine to open things up, but it will go a great ways towards hardening your system.

Oh... and this whole deal about reverse DNS spoofing...that is one reason why the datafiles used by commands like last often include the IP address, and why all logs for services should also log the IP address. A last -i will display the IP address itself, and not do the reverse DNS. Other commands have similar flags.

Doug Needham
  • 11
  • 2
  • You say one should never use the root user or su. Occasionally, I've had to edit system files (in `/etc`), so how would you do it? – rubik Sep 21 '16 at 07:27
  • use root or su for those less frequent needs, but only those. – Skaperen Sep 21 '16 at 07:59
1

I would be concerned if the real FBI was that incompetent to make such a mistake. Real hackers would assume you will rebuild if they let their presence be known. It's kids or sloppy wannabes playing around.

Skaperen
  • 315
  • 2
  • 11