As mentioned in http://nginx.org/en/docs/http/ngx_http_upstream_module.html#keepalive
It should be particularly noted that the keepalive directive does not limit the total number of connections to upstream servers that a Nginx worker process can open. The connections parameter should be set to a number small enough to let upstream servers process new incoming connections as well.
I want to understand if a new client comes, why can't they use existing keep-alive connections?
Does Nginx need to create a new connection with upstream everytime a new client comes?