Setting noindex some pages on NGINX

Question

I want to set noindex some URLs. If any URL contains ?lang= | (start with) /plugin | (start with) /account it should be noindex by adding HTTP header in NGINX configuration.

I tried below code before,

location ~ .*/(?:?lang|plugin|account)/.* {
    add_header X-Robots-Tag "noindex, follow" always;      
}

Other NGINX location directives that I use for my website: (These default directive for my script are working well.)

#Disable access to sensitive files
location ~* /(app|content|lib)/.*\.(po|php|lock|sql)$ {
    deny all;
}
#CORS headers
location ~* /.*\.(ttf|ttc|otf|eot|woff|woff2|font.css|css|js) {
    add_header Access-Control-Allow-Origin "*";
}
#Upload path for image content only and set 404 replacement
location ^~ /images/ {
    location ~* (jpe?g|png|gif) {
        log_not_found off;
        error_page 404 /content/images/system/default/404.gif;
    }
    return 403;
}
#Pretty URLs
location / {
    index index.php;
    try_files $uri $uri/ /index.php?$query_string;
}

Although no error messages when I reload the Nginx, the noindex directive doesn't appear.

score 1 · Accepted Answer · answered Aug 04 '19 at 10:07

location directives only match normalised URIs, which don't include query arguments. This is the reason why you cannot match ?lang.

To match that one, you can use a query argument variable to make the match.

if ($arg_lang) {
    add_header X-Robots-Tag "noindex, follow" always; 
}

This snippet should be included in the location / block.

For matching the other cases, I would do the following:

location /plugin {
    add_header X-Robots-Tag "noindex, follow" always; 
    try_files $uri $uri/ /index.php?$query_string;
}

location /account {
    add_header X-Robots-Tag "noindex, follow" always; 
    try_files $uri $uri/ /index.php?$query_string;
}

It is important to include the try_files statements in these, since only one location block is selected to be used.

Setting noindex for `?lang` query argument (inside `location /`) is causing 404 error. for these pages which include `?lang` query. But it doesn't matter for my website. I don't use these pages. Thank you! I really appreaciate it! — Serdar Koçak, Aug 04 '19 at 22:44

Setting noindex some pages on NGINX

1 Answers1