IIS impersonate 401 Unauthorized (2x asp.net MVC)

Question

I have IIS website with windows auth (only) enabled (lets say site.domain.com). My site calls HTTP rest API on it's subsite (lets say site.domain.com/api). It is different application. It works OK, when I call this api as application pool identity. When I use Impersonation I get error 401 Unauthorized.

IIS server setup details:

IIS servers are loadbalanced by third party loadbalancer. Two sites in IIS manager look like image

Both sites have only windows auth enabled and Negotiate as the first provider. Both sites have different app pool. These app pools have the same app pool identity (lets say DOMAIN\apppool). Spn setup is (you can see I tried everything):

PS C:\WINDOWS> setspn.exe -l DOMAIN\apppool
Registered ServicePrincipalNames for SOMECN:
HTTP/site/api
HTTP/site.domain.com/api
HTTP/site
HTTP/site.domain.com

I can communicate with both sites using Kerberos. useAppPoolCredentials = True and useKernelMode = false is set up for both sites. In AD app pool account has marked option "Trust this user for delegation to any service (Kerberos only)"

Error:

IIS log for failed request using Impersonation:

2017-10-04 14:40:57 192.168.81.101 GET /api/Test/Impersonate - 443 - 192.168.81.21 - - 401 2 5 1433
2017-10-04 14:40:57 192.168.81.101 GET /api/Test/Impersonate - 443 - 192.168.81.21 - - 401 1 2148074254 0
2017-10-04 14:40:57 192.168.81.101 GET /api/Test/Impersonate - 443 - 192.168.81.21 - - 401 1 5 15

IIS log for successfull attempt using app pool credentials (without impersonation):

2017-10-04 13:52:28 192.168.81.101 GET /api/Test/Impersonate - 443 - 192.168.81.21 - - 401 2 5 0
2017-10-04 13:52:28 192.168.81.101 GET /api/Test/Impersonate - 443 DOMAIN\apppool 192.168.81.21 - - 200 0 0 15

Application details:

Application runs successfully on my development machine. Problem is only in production. On development machine I use my hostname to access the site.

Here is how I call the API site from my MVC controller:

[HttpGet]
public ViewResult ImpersonateTest()
{
    System.Security.Principal.WindowsImpersonationContext impersonationContext;
    impersonationContext =
        ((System.Security.Principal.WindowsIdentity)User.Identity).Impersonate();
    using (var client = new WebClient() { UseDefaultCredentials = true })
    {
        ViewBag.User = client.DownloadString(GlobalConfig.ApiPath + $"Test/Impersonate");
    }
    impersonationContext.Undo();
    return View();
}

If I remove first three lines, credentials of app pool are used and all works as expected. But I want to use impersonation ofc.