Change wordpress port number in nginx

Question

I have a wordpress installation with nginx. It's currently use port 80. But I'd like to change it to something else. So I found /etc/nginx/sites-available/wordpress file and changed the following two lines:

    listen 80 default_server;
    listen [::]:80 default_server ipv6only=on;

But no matter how I change it, it only works with both port set to 80. I actually don't know what the second line is for. But removing it doesn't seem to make a difference anyway.

What am I missing?