IPSEC Review

This is a conversation I had a while back with security guru, Darren Spruell. He lead this overview of IPSEC which I'm sure he pulled straight off the top of his head.

Darren: so you wanna firm up some knowledge of ipsec?
Ambrose: oh yea
Ambrose: that would be lovely
Darren: random question, i suppose. i just feel like going over ipsec. ;)
Darren: wanna start with what you know so far? take it away.
Ambrose: Here's as much as I know.... It's a protocol used for encrypting IP traffic between 2 endpoints. A protocol called ISAKMP is usually used to authenticate the session, and an encryption protocol like 3des can be used to encrypt the session.
Ambrose: tada
Ambrose: not much...hehe
Darren: not bad actually.
Darren: so i would add that not only one host to another, but network to network (which you already know, because that's what you're doing with enschede.)
Ambrose: ah yes...
Darren: so some theory might help.
Darren: IPsec can work in 2 modes, 'tunnel' and 'transport'. transport mode is usually used for host to host connections - like one system on a LAN wants to secure communications with another.tunnel mode is more commonly used, and is what supports the network-to-network mode that is commonly used for B2B VPNs. It does this by defining IPsec 'gateways' which negotiate a tunnel between themselves and tunnel traffic for the secured networks over the tunnel. Incidentally, tunnel mode "hides" the addresses that are really communicating over the tunnel, and transport mode naturally does not.
Darren: you basically got ISAKMP. its good to understand how it works. in tunnel mode, isakmp is responsible for dynamic keying of security associations. (you can also use static keys in some implementations). isakmp is the standard which basically takes a few setup parameters and then builds all the pieces that make a vpn tunnel do what it is supposed to.
Ambrose: Interesting....I never knew there was a difference. It makes sense though. I'm guessing that microsoft's version of IPSEC between machines on a domain would be transport mode.
Darren: i think so, and it complicates it a bit more because tehy do L2TP+IPsec, which hasn't really caught on except for MS and Cisco (co-creators of the scheme)
Darren: isakmp works over udp port 500. isakmp needs settings which define the tunnel parameters, namely:
- a shared secret or x.509 certificate used to authenticate the remote isakmp peer,
- what subnets are defined as the tunneled networks on both sides of the tunnel,
- information on transform sets used to provide the security services for the tunnel,
- "other" information like use of perfect forward secrecy, timeouts & rekey periods, whether NAT-traversal (NAT-T) should be used, etc.)
Ambrose: nice. I've set up a few VPN's on our Pix and become aware of those pramaters, but I didn't know they were all part of isakmp
Ambrose: then, openvpn doesn't use isakmp at all, does it? It must just use TLS for ssl key exchange to establish and maintain the tunnel?
Ambrose: or something like that...
Darren: yeah, that's right.
Darren: assuming that isakmp can negotiate identical settings with and authenticate the remote peer, the tunnel setup proceeds and the vpn happens. this is all from a high level, of course. It gets more interesting when you look at the underbelly.
Darren: Conceptually, SSL/TLS works similarly. openvpn in tls mode (2.0+) usually uses RSA to securely exchange a session key with the authenticated peer, and the session key is used for bulk encryption of the resulting data. RSA is asymetric, so it can encrypt the session key to the peer using the peer's public key. a symetric cipher such as RC4 or AES (Rijndael) is used for the actual encryption (much faster than asymetric.)
Darren: the concepts are the same at a high level for all of these solutions, be it isakmp (which typically uses Diffie-Helman for key exchange), or SSL (typically RSA) or SSH (DH or RSA or ...) or PGP, or ... it's called a hybrid scheme because it uses a combination of a public key cipher to exchange a key and a secret key cipher for bulk encryption.
Darren: so back to isakmp; the tunnel negotiation works in two phases, which result in two sets of Security Associations (SAs).
Darren: phase 1 is what happens when the IKE portions (isakmp) negotiate a secure tunnel to carry on communications over. a phase 1 SA, in a manner of speaking, is host to host between ipsec gateways.
Darren: is this useful?
Ambrose: yea, this is good stuff
Ambrose: you mentioned phase 1, so what is phase 2?
Darren: phase 2 is where the real purpose of the tunnel is carried out, or where the tunneled networks have security associations built that allows their traffic to be tunneled over the untrusted network securely. so in order for a phase 2 SA to be built, remote and local subnet definitions have to agree on hosts that can encrypt packets over the tunnel.
Darren: so in order for an ipsec vpn to "work", you have to have successful phase 1 and phase2 proposals negotiated. a successful phase 1 pretty much means your peers have been able to authenticate and negotiate initial settings (common transforms, rekey timeouts, etc.) a successful phase 2 happens when an authorized host or subnet on one end of the tunnel is allowed to send traffic to an authorized host or subnet on the other end. the rest just happens.
Ambrose: Is a phase 1 generally maintained, or cached for a certain amount of time, until the phase 1 SA expires?
Ambrose: I've also seen the 2 phases SA's differentiated as isakmp sa and ipsec sa
Darren: it'll expire, or more accurately have a rekey timeout.
Darren: yeah, that's right, isakmp vs. ipsec sa would work.
Darren: another important concept is the protocols used after isakmp does its stuff. once the actual encapsulation begins, IPsec defines ESP and AH to be used. this probably isn't news, but they are protocols that provide different "security primitives" for the encapsulated data.
Darren: ESP is the Encapsulating Security Payload and guarantees confidentiality and integrity of the encapsulated traffic. confidentiality = only the intended recipient of the data can get the data (read: encryption). integrity = assurance that the data has not been modified after it was transmitted. they can work independent of each other or in conjunction with each other. ESP also provides data orginin authentication (read: the data actually did originate from who you think it did.)
Ambrose: geez, there's a lot to ipsec
Darren: truly. :)
Ambrose: underneath the surface
Darren: AH is the Authentication Header. Like its name says, it authenticates the header of transmitted data, or in other words guarantees authenticity. So #1 it does something that ESP can already do, and #2 it doesn't do what most people use ESP to do (and what they generally want a VPN to do), which is provide confidentiality. so for these reasons AH isn't commonly used. The other big thing with AH is that it can't be used in a NAT environment, and this is because it authenticates the actual headers, so the address tampering that NAT does naturally breaks this. (AH makes this happen in a different way than ESP does; strictly speaking, it authenticates "better" although you have to understand where its header information is placed relative to how ESP provides its origin authentication in order to see why it's "better" and why NAT breaks AH but not ESP.
Darren: It's useful to understand that ESP and AH have to "port numbers" to speak of, as they are not transport layer protocols within TCP or UDP, but they are IP protocols (50 and 51), and are therefore "parallel to" ICMP, UDP, TCP, or OSPF in the stack. Furthermore, for isakmp, UDP 500 has to be open on both hosts (HostA:500/udp <--> HostB:500/udp), so no random high port numbers stuff, and ultimately, it means that isakmp daemons run as root on unix boxen (privileged process to bind to privileged port)
Darren: *no* port numbers, that is
Darren: so then, log files serve as some pretty good examples of what this looks like. i can show you some logs from our Contivities, its pretty high level showing phase 1 and 2 negotiations...
Darren: so here's the log entries showing what's happening...
Darren: 10:47:07 tEvtLgMgr 0 : Security [11] Session: IPSEC[70.141.178.209] attempting login
10:47:08 tEvtLgMgr 0 : Security [11] Session: IPSEC[70.141.178.209]:1986605 authenticated using LOCAL
10:47:08 tEvtLgMgr 0 : Security [11] Session: IPSEC[70.141.178.209]:1986605 bound to group /Base/B2B_Sonicwall/Luosys - Martin
10:47:08 tEvtLgMgr 0 : Security [11] Session: IPSEC[70.141.178.209]:1986605 authorized
10:47:08 tEvtLgMgr 0 : Security [11] Session: network IPSEC[70.141.178.209-255.255.255.255] attempting login
10:47:08 tEvtLgMgr 0 : Security [11] Session: network IPSEC[70.141.178.209-255.255.255.255] logged in from gateway [70.141.178.209]
10:47:08 tEvtLgMgr 0 : Security [12] Session: IPSEC[70.141.178.209]:1986605 physical addresses: remote 70.141.178.209 local 162.135.0.15
10:47:08 tEvtLgMgr 0 : Security [12] Session: IPSEC[-]:1986606 physical addresses: remote 70.141.178.209 local 162.135.0.15
Darren: lines 1-4 comprise the Phase 1 setup. the rest is phase 2. notice how the remote address is a single ip in phase 1 (the ipsec peer gateway) and in phase 2 we have a subnet declaration (happens to be the same IP address, and a /32 netmask. This tells you they are NATting their connections to a public address prior to encapsulation.)
Darren: so i think yesterday i was about to share some information from the list of active SAs on our vpn concentrator.
Ambrose: oh yes
Ambrose: let's see it
Darren: i threw out some log file snippets yesterday, yeah?
Ambrose: yea, we saw phase1 and phase 2 negotiations and the public address everything was being NATed through before hitting the VPN.
Darren: cool.
Darren: so here are the resulting SAs that were built as the tunnel was negotiated and traffic traversed the tunnel
Darren: ISAKMP security association established with 70.141.178.209
Local address: 162.135.0.15
Local Udp Port:500 Remote port:500
Initiator cookie: D1258A30B3ED27A7
Responder cookie: B23640961F4E9EED
IKE encryption: Triple DES with Diffie-Hellman group 2 (MODP 1024-bit prime)
Darren: nothing much to that. endpoint addresses and port numbers, and it notifies us that DH is used for the key agreement algorithm in IKE.
Darren: since the Nortel calls this the ISAKMP SA we know it is phase 1 (also because it has gateway IPs and not our tunneled subnet addresses.
Darren: Next comes the phase 2 SAs:
Darren:
IPSec tunnel mode security associations established:
Local host 10.2.31.114
Remote host 70.141.178.209
ESP 3DES-CBC-HMAC-SHA outbound SPI 0xAE3A201B software session
29 packets sent
ESP 3DES-CBC-HMAC-SHA inbound SPI 0x1F8E30 software session
36 packets successfully received
0 packets truncated
0 packets failed replay check
0 packets failed authentication
0 packets with invalid pad length (decryption failure)
Expires on TUE JUN 13 00:20:16 2006

IPSec tunnel mode security associations established:
Local host 10.2.31.117
Remote host 70.141.178.209
ESP 3DES-CBC-HMAC-SHA outbound SPI 0xD95A8667 software session
0 packets sent
ESP 3DES-CBC-HMAC-SHA inbound SPI 0x9E1D4 software session
0 packets successfully received
0 packets truncated
0 packets failed replay check
0 packets failed authentication
0 packets with invalid pad length (decryption failure)
Expires on TUE JUN 13 00:20:05 2006
Darren: 10.2.31.114 <-> 70.141.178.209 and 10.2.31.117 <-> 70.141.178.209 are the secured flows over the tunnel.
Darren: The key thing about phase 2 SAs is the local and remote subnet definitions have to match. In other words, if I say that:

- local subnets = 10.0.1.0/24 10.0.5.2/32
- remote subnets = 192.168.0.0/16

...and you say that:

- local subnets = 192.168.0.0/16
- remote subnets = 10.0.0.0/16

...you will have a negotiation failure because the subnet list mismatches (even though your addresses still "match" due to the subnet definition, they don't result in a SA match and therefore fail.
Ambrose: Yea, I found out about that requirement the hard way....
Darren: haha, bites everyone once. :)
Ambrose: what determines the phase 2 key lifetime? Is that usually user determined as well?
Darren: yes, i think so - should also be a lifetime definition for your choice of data amount or time passed rekeying.
Darren: i.e. after 1024 MB or after 28,800 seconds (8hrs)
Darren: other things that can affect successful negotiation:
- mismatches in supported encryption or hash algorithms
- mismatch in pfs settings
- network lists match up perfectly, but someone is unknowingly NAT'd to a different address than defined (or not NAT'd)
Darren: we've basically adopted the policy here that new tunnels *require* the remote side to use non-rfc1918 addresses. we've had too much trouble with vendors that want to use thier public range which inevitably overlap with our LAN or try to get us to handle the NAT on our side of the tunnel for addresses on thier side. it got to the point of being unmanageable, and since we're a 700 lb gorilla, and since we're basically all allocated out on RFC1918 addresses anyway, they have to interoperate with us to play (plus, we're usually the customer, so if they want our money they can play by our rules, no matter how stupid they are. :)
Ambrose: yea, that's always a pain...
Ambrose: overlaps...
Darren: nothing a good nat can't take care of, when you have the liberty.
Darren: something that understanding ipsec better has led me to understand is why every opensource developer out there thinks the IETF overengineers everything they put together. it's difficult to have to re-implement protocols like this when they aren't simple, especially if your goal is to do it securely (complexity is security's bane.)
Darren: wpa is another one that falls in that category. while it's easy to use from the admin or user's standpoint, the implementation is really a biatch (and you wonder why the OSS community is slow to adopt it.) 802.11i is sure to be along the same lines.
Ambrose: yea, that's the beauty of openVPN, 'simplicity'
Darren: bingo
Darren: anyway, for that reason, I tend to prefer openvpn over ipsec where i have the choice. there's absolutely no difference in security other than possibly TLS is easier (and therefore more likely be be securely implemented) than IPsec and for that reason, also easier for an admin to configure correctly as well.
Darren: have you used openvpn 2.x?
Ambrose: not yet
Darren: put simply, it's brilliant. TLS mode is introduced, and making a secure road-warrior setup using X.509 authentication is cake.
Ambrose: mmmmmm..... nice.
Darren: ssl vpns have been one of the biggest buzzwords lately, but not all are created equal. here's a great read on what's up with them and how openvpn fits in, mostly superior in about every way actually. (pdf warning.)
Darren: http://www.sans.org/rr/whitepapers/vpns/1459.php
Darren: so was the ipsec stuff useful?
Ambrose: It was awesome.
Darren: cool
Darren: so what's next?
Darren: ;)