How do I stop nginx from serving requests on my public IP?

Question

I am serving 2 public sites on a Pi from my home network using nginx on Ubuntu Server. My domains point to my no-ip account; both domains work perfectly. However, going to my IP address returns one of my sites, which I've only set to be resolved using it's domain name. How do I force my IP to either serve a Not Found Error or point to a different site?

The reason I want to stop my website displaying on my IP is that because my site is HTTPS only, with HTTP 301s, it displays insecure connection. I'd rather there be nothing there at all.

EDIT: ../sites-available/sitename

server {
    listen 443 ssl;

    root /var/www/****/www;
    index index.html index.php;

    server_name ****.ddns.net;
    server_name ****.**;
    server_name www.****.**;
    server_name ***.****.**;

    error_page 403 /403.html; 
    error_page 404 /404.html;
    error_page 500 502 503 504 /500.html;

    ssl_certificate /etc/letsencrypt/live/www.****.**/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/www.****.**/privkey.pem;

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
    ssl_prefer_server_ciphers on;
    ssl_ciphers 'EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH';

    location ~ \.php$ {
        try_files $uri =404;
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        fastcgi_pass unix:/var/run/php5-fpm.sock;
        fastcgi_index index.php;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        include fastcgi_params;
    }

    location / {
        try_files $uri $uri/ =404;
    }
}

server {
    listen 80;
    server_name www.****.**;
    server_name ****.**;
    server_name ***.****.**;
    return 301 https://$host$request_uri;
}

Answer 1

You can define a server with an empty server name that drops the requests:

server {
    listen      80;
    listen      443;
    server_name "";
    return      444;
}

Note that listen 443 is only needed if you also want to deny https access to unknown hosts.

(source)