6

We swapped to ASA's over the weekend, and we replaced our VPN infrastructure which was previously based on openvpn and are now using IPSec between our ASA 5520's and our other sites that have linux (CentOS) routers.

The VPNs connect just fine, but after a period of time the connections fail. On the ASA, it shows no ipsec SA's for the peer, but it does show an isakmp sa still active. If I clear the SA's on both sides of the connection, the VPN will come back up again.

I'm assuming the problem is a rekeying issue, but it appears like all of the proposals have the same key lifetimes (shown below). Any thoughts on what could be the problem?

NOTE -- I've obfuscated the ip addresses from these captures; I have a suspicion something's wrong with my proposal so the IPs shouldn't be relevant. Assume all IPs are placeholders.


ASA show run crypto


crypto ipsec ikev1 transform-set ESP-3DES-SHA esp-3des esp-sha-hmac
crypto ipsec security-association lifetime seconds 86400
crypto dynamic-map OUTSIDE_DYN_MAP 10 set ikev1 transform-set ESP-3DES-SHA
crypto dynamic-map OUTSIDE_DYN_MAP 10 set security-association lifetime seconds 288000
crypto dynamic-map OUTSIDE_DYN_MAP 10 set reverse-route
crypto map vpnmap 10 match address colo1_to_hq_vpn
crypto map vpnmap 10 set pfs
crypto map vpnmap 10 set peer 1.1.1.1
crypto map vpnmap 10 set ikev1 transform-set ESP-3DES-SHA
crypto map vpnmap 20 match address colo1_to_colo2_vpn
crypto map vpnmap 20 set pfs
crypto map vpnmap 20 set peer 2.2.2.2
crypto map vpnmap 20 set ikev1 transform-set ESP-3DES-SHA
crypto map vpnmap 65500 ipsec-isakmp dynamic OUTSIDE_DYN_MAP
crypto map vpnmap interface OUTSIDE
crypto isakmp identity address
crypto isakmp nat-traversal 300
crypto ikev1 enable OUTSIDE
crypto ikev1 policy 1
 authentication pre-share
 encryption 3des
 hash sha
 group 2
 lifetime 86400

ASA show crypto isakmp sa detail


IKEv1 SAs:

   Active SA: 2
    Rekey SA: 0 (A tunnel will report 1 Active and 1 Rekey SA during rekey)
Total IKE SA: 2

1   IKE Peer: x.x.x.x
    Type    : L2L             Role    : responder
    Rekey   : no              State   : AM_ACTIVE
    Encrypt : 3des            Hash    : SHA
    Auth    : preshared       Lifetime: 86400
    Lifetime Remaining: 85905
2   IKE Peer: y.y.y.y
    Type    : L2L             Role    : initiator
    Rekey   : no              State   : MM_ACTIVE
    Encrypt : 3des            Hash    : SHA
    Auth    : preshared       Lifetime: 86400
    Lifetime Remaining: 85976

ASA show crypto ipsec sa


peer address: x.x.x.x
    Crypto map tag: vpnmap, seq num: 10, local addr: y.y.y.y

      access-list peer1_to_hq_vpn extended permit ip z.z.z.z 255.255.0.0 t.t.t.t 255.255.0.0
      local ident (addr/mask/prot/port): (9.9.0.0/255.255.0.0/0/0)
      remote ident (addr/mask/prot/port): (8.8.0.0/255.255.0.0/0/0)
      current_peer: 38.104.67.142

      #pkts encaps: 4714, #pkts encrypt: 4714, #pkts digest: 4714
      #pkts decaps: 4672, #pkts decrypt: 4672, #pkts verify: 4672
      #pkts compressed: 0, #pkts decompressed: 0
      #pkts not compressed: 4714, #pkts comp failed: 0, #pkts decomp failed: 0
      #pre-frag successes: 0, #pre-frag failures: 0, #fragments created: 0
      #PMTUs sent: 0, #PMTUs rcvd: 0, #decapsulated frgs needing reassembly: 0
      #send errors: 0, #recv errors: 0

      local crypto endpt.: 1.1.1.1/0, remote crypto endpt.: 2.2.2.2/0
      path mtu 1500, ipsec overhead 58, media mtu 1500
      current outbound spi: 06596006
      current inbound spi : 55EC97A1

    inbound esp sas:
      spi: 0x55EC97A1 (1441568673)
         transform: esp-3des esp-sha-hmac no compression
         in use settings ={L2L, Tunnel, PFS Group 2, }
         slot: 0, conn_id: 204800, crypto-map: vpnmap
         sa timing: remaining key lifetime (sec): 85731
         IV size: 8 bytes
         replay detection support: Y
         Anti replay bitmap:
          0xFFFFFFFF 0xBFFFFFFF
    outbound esp sas:
      spi: 0x06596006 (106520582)
         transform: esp-3des esp-sha-hmac no compression
         in use settings ={L2L, Tunnel, PFS Group 2, }
         slot: 0, conn_id: 204800, crypto-map: vpnmap
         sa timing: remaining key lifetime (sec): 85731
         IV size: 8 bytes
         replay detection support: Y
         Anti replay bitmap:
          0x00000000 0x00000001

CentOS IPSec config:


TYPE=IPSEC
ONBOOT=YES
IKE_METHOD=PSK
SRCGW=1.1.1.1
DSTGW=2.2.2.2
SRCNET=1.1.1.1/16
DSTNET=2.2.2.2/16
DST=64.34.119.71
AH_PROTO=none

racoon config:


sainfo anonymous
{
        pfs_group 2;
        lifetime time 24 hour;
        encryption_algorithm 3des, blowfish 448, rijndael;
        authentication_algorithm hmac_sha1, hmac_md5;
        compression_algorithm deflate;
}
remote 1.2.3.4
{
        exchange_mode aggressive, main;
        my_identifier address;
        proposal {
                encryption_algorithm 3des;
                hash_algorithm sha1;
                authentication_method pre_shared_key;
                dh_group 2;
        }
}

Relevant SAD/SPD entries:


64.34.119.71 38.104.67.142
        esp mode=tunnel spi=106520582(0x06596006) reqid=0(0x00000000)
        E: 3des-cbc  8973cb22 ce1ab25c c4a4427c aac0c857 06917359 9b88e01e
        A: hmac-sha1  3655fb9b e6882226 829f2214 0b22ec27 8155587b
        seq=0x00000000 replay=4 flags=0x00000000 state=mature
        created: Apr 16 11:30:43 2012   current: Apr 16 11:36:58 2012
        diff: 375(s)    hard: 86400(s)  soft: 69120(s)
        last: Apr 16 11:30:43 2012      hard: 0(s)      soft: 0(s)
        current: 898519(bytes)  hard: 0(bytes)  soft: 0(bytes)
        allocated: 2749 hard: 0 soft: 0
        sadb_seq=3 pid=12574 refcnt=0
38.104.67.142 64.34.119.71
        esp mode=tunnel spi=1441568673(0x55ec97a1) reqid=0(0x00000000)
        E: 3des-cbc  0f5bdfdc 23b140f8 4636326f f194fa0d 6a919f28 a6974b5f
        A: hmac-sha1  586e3bf7 794960e1 e9da8707 5863e94d e88e0a11
        seq=0x00000000 replay=4 flags=0x00000000 state=mature
        created: Apr 16 11:30:43 2012   current: Apr 16 11:36:58 2012
        diff: 375(s)    hard: 86400(s)  soft: 69120(s)
        last: Apr 16 11:30:43 2012      hard: 0(s)      soft: 0(s)
        current: 645624(bytes)  hard: 0(bytes)  soft: 0(bytes)
        allocated: 2764 hard: 0 soft: 0
        sadb_seq=0 pid=12574 refcnt=0

1.1.0.0/16[any] 2.2.0.0/16[any] any
        in prio def ipsec
        esp/tunnel/1.1.1.1-2.2.2.2/require
        created: Apr 16 11:30:12 2012  lastused:
        lifetime: 0(s) validtime: 0(s)
        spid=12784 seq=59 pid=12583
        refcnt=1

2.2.0.0/16[any] 1.1.0.0/16[any] any
        out prio def ipsec
        esp/tunnel/1.1.1.1-2.2.2.2/require
        created: Apr 16 11:30:12 2012  lastused: Apr 16 11:37:59 2012
        lifetime: 0(s) validtime: 0(s)
        spid=12777 seq=57 pid=12583
        refcnt=402

1.1.0.0/16[any] 2.2.0.0/16[any] any
        fwd prio def ipsec
        esp/tunnel/1.1.1.1-2.2.2.2/require
        created: Apr 16 11:30:12 2012  lastused: Apr 16 11:37:59 2012
        lifetime: 0(s) validtime: 0(s)
        spid=12794 seq=55 pid=12583
        refcnt=54
Peter Grace
  • 3,446
  • 1
  • 26
  • 42
  • Is there constant traffic on the tunnels, or is there a chance that the IPsec SAs sit idle for a time? Can you turn up the ASA's logging on VPN-related stuff to debug? I expect that the disconnect is correlating to some interesting event, such as a rekey of the SA. – Shane Madden Apr 16 '12 at 15:57
  • @ShaneMadden: There should be pretty much constant traffic – Kyle Brandt Apr 16 '12 at 16:01
  • What's the displayed state of the still-live ISAKMP SA (from `show crypto isakmp sa`)? Is it in a state where it's waiting on a rekey? Not sure how racoon handles having no ISAKMP lifetime set, it may be of value to try setting `lifetime time 24 hour;` in the `remote` section of the racoon config. – Shane Madden Apr 16 '12 at 16:10
  • @ShaneMadden, thanks for pointing this out -- I changed the config on my linux endpoints to explicitly define the ISAKMP lifetime, and as a result my ISAKMP SA's reflect the 86400 I expected to see. I don't think this would solve the problem since the default is 8 hours and the problem occurs within 1-2 hours, but, important to be complete about it. – Peter Grace Apr 16 '12 at 16:25
  • @PeterGrace Yup. See if you can crank up logging verbosity, and check the state on that ISAKMP SA when it happens again - I think you're dead on with the rekey issues. – Shane Madden Apr 16 '12 at 17:09

1 Answers1

3

The cause of the issue was that the version of racoon in CentOS (ipsec-tools-0.6.5) appeared to have a bug with regards to properly re-keying. I compiled the latest ipsec-tools from source and the problem has not recurred as a result.

TL;DR - Upgrade ipsec-tools first before banging your head repeatedly on the wall.

Peter Grace
  • 3,446
  • 1
  • 26
  • 42