6

A customer of mine is trying to upload a file to our remote nginx webserver via a form POST using XHR2 form data (and a cross-domain request with CORS). During mid-upload, the webserver returns a 408 and the ajax error handler halts processing as a result. The files are in the 20-120MB range. The access logs for some of the file uploads are as follows (he has tried on Chrome 31 and IE11):

[24/Dec/2013:16:44:18 -0500] "OPTIONS / HTTP/1.1" 200 0 "http://www.example.com/files/upload" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63 Safari/537.36"
[24/Dec/2013:16:47:50 -0500] "POST / HTTP/1.1" 408 0 "http://www.example.com/files/upload" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.63 Safari/537.36"
...
[27/Dec/2013:01:23:51 -0500] "OPTIONS / HTTP/1.1" 200 0 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"
[27/Dec/2013:01:33:11 -0500] "POST / HTTP/1.1" 408 0 "http://www.example.com/files/upload" "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko"

Sometimes files will upload perfectly fine for him with Chrome instead of IE, and sometimes vice versa, but most of the time neither browser will work for him.

I've been reading the nginx wiki and the only two settings that correlate with 408 errors are client_body_timeout and client_header_timeout. I'm having a hard time understanding the meaning of these two directives. I increased both to 180 seconds but the issue persists. I asked him if he had a slow connection, and he said that he has 2.5 mbps up, which should be fast enough to fully receive the request header (but again, we are unsure of what these 2 directives mean according to the wiki, such as what a "readstep" is). We have successfully received 1GB uploads to our server before from other customers, which usually take around an hour to finish.

For the problematic files that we have eventually successfully received from our customer, we tried uploading the same file in various browsers and it worked perfectly.

I have read that using SSL could cause the timeouts, but the server doesn't have SSL enabled and we are only using http.

Our nginx.conf:

user www-data;
worker_processes auto;
pid /run/nginx.pid;

events {
    worker_connections 5000;
    # multi_accept on;
}

http {
    ##
    # Basic Settings
    ##

    sendfile on;
    tcp_nopush on;
    tcp_nodelay off;
    keepalive_timeout 25;

    # Increase client/head body_timeout to prevent 408's for slow Internet connections??
    client_header_timeout 180;
    client_body_timeout 180;


    types_hash_max_size 2048;

    server_tokens off;

    # server_names_hash_bucket_size 64;
    # server_name_in_redirect off;

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    ##
    # Logging Settings
    ##

    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

        log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                    '$status $body_bytes_sent "$http_referer" '
                    '"$http_user_agent" "$http_x_forwarded_for"';

    ##
    # Gzip Settings
    ##

    gzip on;
    gzip_disable "msie6";

    gzip_vary on;
    gzip_proxied any;
    gzip_comp_level 5;
        gzip_min_length 256;

        gzip_types
        application/atom+xml
        text/javascript
        application/javascript
        application/json
        application/rss+xml
        application/vnd.ms-fontobject
        application/x-font-ttf
        application/x-web-app-manifest+json
        application/xhtml+xml
        application/xml
        font/opentype
        image/svg+xml
        image/x-icon
        text/css
        text/plain
        text/x-component; 

        # Increase FastCGI buffers
        fastcgi_read_timeout 1500;
        fastcgi_buffers 8 16K;
        fastcgi_buffer_size 32K;

    ##
    # Virtual Host Configs
    ##
    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;


}

Our upload server config:

server {
    listen 80;
    server_name upload.example.com;

    root /var/www/releases/latest/_UPLOAD/public;

    # remove trailing slash, that throws ZF router
    if (!-d $request_filename) {
        rewrite ^/(.*)/$ /$1 permanent;
    }

    # 1.2G upload limit + 10M for post data (which is extremely liberal)
    client_max_body_size 1210M;
    client_body_buffer_size 4M;    

    proxy_max_temp_file_size 0;    

    location = /favicon.ico {
        access_log     off;
        log_not_found  off;
    }

    location = /apple-touch-icon.png {
        access_log off;
        log_not_found off;
    }

    location / {
        # This is for AJAX file uploads... need to set CORS headers
        if ($request_method = OPTIONS ) {
            add_header Access-Control-Allow-Origin '$scheme://www.example.com';
            add_header Access-Control-Allow-Methods 'POST, OPTIONS';
            add_header Access-Control-Allow-Headers 'X-Requested-With, Content-Type, Content-Range, Content-Disposition';
            return 200;
        }

        try_files $uri $uri/ /index.php?$args;
        index index.php index.html index.htm;
    }

    location ~ \.php$ {
        try_files $uri $uri/ /index.php?$args;
        index index.php index.html index.htm;

        fastcgi_pass   unix:/var/run/php5-fpm.sock;
        fastcgi_index  index.php;
        fastcgi_param  SCRIPT_FILENAME  /var/www/releases/latest/_UPLOAD/public$fastcgi_script_name;
        fastcgi_param  APPLICATION_ENV production;
        fastcgi_param  PATH /usr/bin:/bin:/usr/sbin:/sbin;
        fastcgi_intercept_errors on;
        include        fastcgi_params;
    }

    error_page 403 =404 /404.html;
    error_page 404 /404.html;
    location = /404.html {
        root /var/www/releases/latest/_UPLOAD/public;
        internal;
    }

    error_page 500 502 503 504 = /50x.html;
    location = /50x.html {
        root /var/www/releases/latest/_UPLOAD/public;
        internal;
    }
}

Is there anything in the config that might impede successful uploads and be causing these pesky 408 errors? Nothing is mentioned in the error logs about this issue.

NOTE: We have only used reload when we make changes to the nginx configuration. We are trying to avoid a restart but if that is required for our configuration changes to take full effect then we will do it.

danronmoon
  • 163
  • 1
  • 6
  • I am having the same problem for two days. Customers were able to upload 5MB image files without no problem, but in the last two days some of the uploads are stuck. Nginx logs show huge number of 408 responses Tried increasing `client_header_timeout` and `client_body_timeout` but nothing changes. Have you found a solution to problem? – laltin Jul 04 '14 at 15:01
  • @laltin I have not found a solution yet. – danronmoon Jul 04 '14 at 15:34
  • Are you still experiencing same problem? – laltin Jul 04 '14 at 20:45
  • I'm also facing the same problem, much worse I would say – so far it has happened only when uploading from IE and the file sizes are less than 1 MB. – Gaurav Aug 11 '14 at 12:40
  • Anyone found a solution for this yet? – Gaurav Sep 08 '14 at 11:29
  • Sorry to revive this very old question but I think I am facing the same issue. Was anybody able to successfully debug this? – joidegn Nov 13 '15 at 14:25

0 Answers0