45

As titled, we discovered some unknown IP address is accessing our API Server.

We have set up an AWS EC2 instance as an API server. The API server URL is only used in our mobile app. However, our mobile app has not been released yet and the API server URL is not linked from any public website.

We can see the (multiple) attackers are randomly trying the URL path, 

i.e. 
/admin/i18n/readme.txt
/a2billing/admin/Public/index.php
/current_config/passwd
/recordings/
/.git/objects

How do they discover our server really?

psmears
  • 900
  • 7
  • 9
King Chan
  • 579
  • 1
  • 4
  • 7
  • 67
    Do you have any evidence that they're accessing your site through a domain name? Or would direct IP access be consistent with what you're seeing? The answer is different in both cases.. – Soron Mar 22 '17 at 10:54
  • 18
    http://devopsreactions.tumblr.com/post/44611270530/checking-the-logs-on-a-server-thats-directly – jpmc26 Mar 22 '17 at 16:40
  • 2
    @EthanKaminski Sorry, I thought it was accessed by domain name, but Black Magic made me realized it is just a IP scan :P I just edited the title as I realized might not be access via URL directly. – King Chan Mar 23 '17 at 01:08
  • 28
    Welcome to the Internet. – David Schwartz Mar 23 '17 at 07:54
  • 47
    How does a burglar find find your house if you haven't published your street address anywhere? – Randy Orrison Mar 23 '17 at 07:57
  • 1
    Similar question: _[Script Kiddies - how do they find my server IP?](https://security.stackexchange.com/q/113456/13154)_ – user2428118 Mar 25 '17 at 12:07

3 Answers3

117

The short answer is that many people are scanning everything most of the time.

Doing so was, some years back, considered impractical, but the combination of better networks, better tools better throughput, and more of the space being in use means that is no longer the case.

For example, Zmap claims on its front page:

ZMap is capable of performing a complete scan of the IPv4 address space in under 5 minutes, approaching the theoretical limit of ten gigabit Ethernet.

Botnets tend to distribute the same sort of scan across significant numbers of nodes, to achieve a similar result: any given machine on the internet is likely to be scanned at least once a day by a determined attacker/scanner.

Once a webserver is identified on a given IP, there are all kinds of tools to test well-known paths, and tools that will try and guess at your sitemap.

In short, welcome to the internet - where obscurity is not security.

Do consider this in the context of setting up your app/services/widget - in all likelihood, things you would probably prefer to be 'secret' will not be, and defending your assets and resources is necessary.

iwaseatenbyagrue
  • 3,631
  • 1
  • 12
  • 24
  • 20
    +1 but it's really more likely to be scanned hourly rather than daily. There's a lot of people running a lot of systems looking for a lot of insecure devices to add to their botnets. – phyrfox Mar 22 '17 at 16:13
  • 15
    You should add that this only applies if the server is available via an IPv4 address. If it only is available via IPv6, it can run for years without being scanned a single time. – UTF-8 Mar 22 '17 at 16:30
  • 1
    Jesus, I didn't know IPv4 address can be scanned that quick nowadays... Now I do not feel hosting anything my home network is safe anymore lol – King Chan Mar 23 '17 at 00:47
  • 7
    `many people are scanning everything most of the time` this was true way back in 1999, it's just that most servers like Apache and Nginx tend to reject known attacks without logging. I wrote my own web server in 1999 (for fun) and got hit within 2 minutes of first putting it online. I ran Apache and looked at the access logs and saw nothing. Then I pored over my own server's logs and discovered some of them were worms like Love Letter and Melissa (Melissa is an email virus, no idea why it would ping http) – slebetman Mar 23 '17 at 03:46
  • 12
    @UTF-8 unless you leak your IPv6 in some way. Shodan (a search engine for network-connected devices) hosts an IPv6 NTP server joined in the NTP pool that also records client's IPv6 addresses for future scanning. – André Borie Mar 23 '17 at 10:14
  • 6
    Side note: please make use the disastrous experiences of peers who ran zmap in their network and ended up saturating everything on the way to Internet. This tool (brilliant by the way) must really be used from the outermost perimeter (in practical terms, without a firewall in between), or in well controlled environments. – WoJ Mar 23 '17 at 10:33
  • 2
    @UTF-8: here's an interesting talk about discovering IPv6 hosts on the internet: https://media.ccc.de/v/33c3-8061-you_can_-j_reject_but_you_can_not_hide_global_scanning_of_the_ipv6_internet – oliver Mar 23 '17 at 12:57
  • @AndréBorie Didn’t they get [kicked off the pool after the NTP people found out](http://lists.ntp.org/pipermail/pool/2016-January/007758.html)? (But as they mention in that mailing list thread, people will find creative ways to harvest active IPv6 addresses no matter what.) – Jonas Schäfer Mar 24 '17 at 07:46
  • @JonasWielicki I recently saw an IPv6 on there so I would assume they're still on it unless they found an alternative solution. – André Borie Mar 24 '17 at 10:00
15

About every IP has been under continuous probing for years. You don't even need to have a domain name, a normal consumer DHCP IP access is enough. Running a dummy http service on your computer scores hits from automated /phpmyadmin and such probes. Should your service give http code 200 as an answer, some abuse attempts may follow.

The probers and attackers use botnets and therefore the IP used can be anything from a hacker's home address to a kitchen appliance's IP ... and more. Blocking IPs doesn't work anymore. If you want to have something in the Internet before release, you might want to whitelist the addresses that need to access the site.

Remember: never allow direct SSH access to your servers from the Internet. The SSH ports are under similar bombardment all the time. If you have an SSH server and username 'dave' with password 'letmein' open to internet, you'll probably get hacked in no time.

  • 3
    ...and if SSH access is needed (which it is in some professional scenarios), either use truly long and unguessable passwords (which a robot couldn't happen upon if it tried every dictionary in the world and MILLIONS of random combinations) - or SSH public key authentication (which means that anyone not in possession of a certain 2048 to 4096 bit keyfile can try until they turn blue in the face, leaving bugs in the sshd implementation as the only possible vulnerability). – rackandboneman Mar 22 '17 at 16:53
  • 14
    Advice about SSH is rather bogus; it should be simply **disable password authentication** (and everything except public key authentication). Even if you think your SSH access is "not direct", an attacker can probably find a way to brute-force it; conversely, no matter how direct the access you provide, they can never brute force pubkey auth with a proper key size. – R.. GitHub STOP HELPING ICE Mar 22 '17 at 17:38
  • 4
    Since the machine is running on EC2, a fairly standard practice would be to whitelist certain ips for ssh in the instance security group. If your ip changes, it is only a matter of logging into the AWS console and changing it to something else. – trognanders Mar 22 '17 at 17:42
14

This seems like it is just random scanners on the internet. They just roam around trying different IP adresses and looking for certain folders and files that might indicate vulnerabilities.

Black Magic
  • 1,212
  • 1
  • 10
  • 15
  • But the thing is, how? Usually Web crawler/ Web Scanner takes the known website pages, download the page content, identify the links exists in the page, then access the links, download the page content, etc repeatedly. But in our case, our API Server URL is not linked in any where of the public. – King Chan Mar 22 '17 at 07:48
  • 2
    Do you maybe have a robots.txt that contains these urls to prevent search engines to index it? Also, do you see it trying directories etc that are unrelated to your API? Cause in that case it might also just run something like Dirbuster with a list of known used directories. – Black Magic Mar 22 '17 at 07:52
  • No we do not have a robots.txt. It is just I have no idea why they figure out our domain name. I looked at each of their IP Address and I found other people reported them as well for doing web application attack or port scanning. – King Chan Mar 22 '17 at 08:04
  • 10
    Figuring out a domain name from a random ip is easy. That is just a simple Reverse DNS lookup. The fact that other people reported them means they are just trying random things and/or are just randomly scanning IP adresses. – Black Magic Mar 22 '17 at 08:06
  • 1
    Oooooh! So basically they are just keep on scanning the IP Address ranges until they got a hit... It is strange that their ISP not banning them for doing so.... Actually that's the answer I am looking for :) – King Chan Mar 22 '17 at 08:10
  • Yep it could be as simple as a sequential probing of the IPv4 address space. Also their ISP should not *necessarily* ban them - it depends on the terms of their contract, what country the ISP is in etc – niilzon Mar 22 '17 at 13:37
  • 11
    @niilzon more important, is the ISP *their* ISP, and not *someone else's* ISP? – Mindwin Mar 22 '17 at 14:16
  • 3
    @KingChan If you were an ISP would you fire customers who use lots of bandwidth and pay you for it? – user253751 Mar 22 '17 at 20:12
  • 2
    @immibis I might prefer the customers who *don't* use lots of bandwidth but still pay me for it – A C Mar 22 '17 at 21:30
  • @immibis Back in 15 years ago when I learns about networking stuff, I scanned IPs for not even an hour, then my ISP stopped my internet :P I haven't tried since then lol – King Chan Mar 23 '17 at 00:37
  • 3
    @KingChan 15 years ago ISPs were actually competent. Now their only priority is scamming customers and forcing them to buy into contracts ("our service is so crap that we need contracts to make sure you stay with us"). – André Borie Mar 23 '17 at 10:17
  • @AndréBorie and the contract also says that we can at any time stop internet service to your house and deny you access to particular websites at our own discretion and you have no rights to say otherwise. It also says that any purchase of an alternate ISP while in contract will result in an immediate $60,000 penalty and the seizure of all electronic devices in your home. Sarcasm you ask... I sure hope it is... – user64742 Mar 24 '17 at 18:45
  • @AndréBorie It's called sarcasm. – user64742 Mar 27 '17 at 15:17