IP tunnels I have known and loved

Today we'll talk about the "classic" IP tunneling protocols.

GRE is often seen as a one size fits all solution when it comes to classic IP tunneling protocols, and for a good reason. However, there are more specialized options, and many of them are supported by VyOS. There are also rather obscure GRE options that can be useful.

All those protocols are grouped under "interfaces tunnel" in VyOS. Let's take a closer look at the protocols and options currently supported by VyOS.

MTU considerations

One issues that often comes up in tunneled setups is that of the MTU and MSS. Generally, the kernel is capable of setting the correct MTU on its own, and as long as end to end ICMP works, there should be no MSS issues either, but if you are in doubt, or simply curious what the total overhead of a tunnel will be, I made a tool for quickly calculating MTU and MSS for any combination of encapsulating and encapsulated protocols. Your contributions and corrections to it are always welcome.

If you want to do MSS clamping, here's an example:

set policy route MSS-CLAMP rule 10 protocol 'tcp'
set policy route MSS-CLAMP rule 10 set tcp-mss '1400'
set policy route MSS-CLAMP rule 10 tcp flags 'SYN'

set interfaces ethernet eth1 policy route MSS-CLAMP
Alternatively, you can insert a global rule like "iptables -I FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu" and make it persistent across reboot by placing it in /config/scripts/vyatta-postconfig-bootup.script

IPIP

This is the simplest tunneling protocol in existence. It is defined by RFC2003. It simply takes an IPv4 packet and uses sends it as a payload of another IPv4 packet. For this reason it doesn't really have any configuration options by itself.

An example:

set interfaces tunnel tun0 encapsulation ipip

set interfaces tunnel tun0 local-ip 192.0.2.10
set interfaces tunnel tun0 remote-ip 203.0.113.20
set interfaces tunnel tun0 address 192.168.100.200

If tunneling IPv4 traffic in IPv4 is really all you want, then it's a pretty good and a very lightweight choice.

IP6IP6

This is the IPv6 counterpart of IPIP. I'm not aware of an RFC that defines this encapsulation specifically, but it's a natural specific case of IPv6 encapsulation mechanisms described in RFC2473.

It's not likely that anyone will need it any soon, but it does exist.

An example:

set interfaces tunnel tun0 encapsulation ipip

set interfaces tunnel tun0 local-ip 2001:db8:aa::1/64
set interfaces tunnel tun0 remote-ip 2001:db8:aa::2/64
set interfaces tunnel tun0 address 2001:db8:bb::1/64

IPIP6

I'm pretty sure in a few decades this is going to be a very useful protocol (though there are other proposals).

As the name implies, it's IPv4 encapsulated in IPv6, as simple as that.

An example:

set interfaces tunnel tun0 encapsulation ipip6

set interfaces tunnel tun0 local-ip 2001:db8:aa::1/64
set interfaces tunnel tun0 remote-ip 2001:db8:aa::2/64
set interfaces tunnel tun0 address 192.168.70.80

SIT (6in4)

I believe SIT stands for "Simple Internet Transition". This protocol is defined by RFC4213, but curiously that RFC or any of its predecessor do not refer to it as SIT, so I have no idea where that nickname actually comes from (if you know its origin, tell me).

It encapsulates IPv6 packets in IPv4, as the name suggests. Unlike two previous protocols, it's very useful right now, as it's used by a number of IPv6 tunnel brokers such as that of Hurricane Electric.

An example:
set interfaces tunnel tun0 encapsulation sit

set interfaces tunnel tun0 local-ip 192.0.2.10
set interfaces tunnel tun0 remote-ip 192.0.2.20
set interfaces tunnel tun0 address 2001:db8:bb::1/64

GRE

GRE stands for Generic Routing Encapsulation, and it lives up to its name as it can encapsulate many other protocols at more than one OSI layer. It is defined by RFC2784.

Due to kernel driver layout reasons, in VyOS it comes in two flavours: "gre" and "gre-bridge". The difference is that while "gre" is layer 3 only, "gre-bridge" is layer 2 and can encapsulate ethernet frames, thus it can be bridged with other interfaces to create datalink layer segments that span multiple remote sites. GRE is also unique in that it can encapsulate more than one protocol at the same time, so it's the only way to create dual stack IPv4 and IPv6 tunnels in a single interface.

Layer 3 GRE example:

set interfaces tunnel tun0 encapsulation gre

set interfaces tunnel tun0 local-ip 192.0.2.10
set interfaces tunnel tun0 remote-ip 192.0.2.20
set interfaces tunnel tun0 address 10.40.50.60/24
set interfaces tunnel tun0 address 2001:db8:bb::1/64

Layer 2 GRE example:

set interfaces bridge br0 

set interfaces tunnel tun0 encapsulation gre-bridge
set interfaces tunnel tun0 local-ip 192.0.2.10
set interfaces tunnel tun0 remote-ip 192.0.2.20
set interfaces tunnel tun0 parameters ip bridge-group bridge br0

set interfaces ethernet eth1 bridge-group br0

As you can see, the bridge-group option for tunnels is in a rather unusual place, different from all other interfaces. I can't remember why is that, and we may make that CLI more consistent in the future even though it will take quite some effort to make it backwards-compatible.

GRE is also the only classic protocol that allows creating multiple tunnels with the same source and destination due to its support for tunnel keys. Despite its name, this feature has nothing to do with security: it's simply an identifier that allows routers to tell one tunnel from another.

An example:

set interfaces tunnel tun0 local-ip 192.0.2.10
set interfaces tunnel tun0 remote-ip 192.0.2.20
set interfaces tunnel tun0 address 10.40.50.60/24
set interfaces tunnel tun0 parameters ip key 10

set interfaces tunnel tun0 local-ip 192.0.2.10
set interfaces tunnel tun0 remote-ip 192.0.2.20
set interfaces tunnel tun0 address 172.16.17.18/24
set interfaces tunnel tun0 parameters ip key 20

Conclusion

Classic IP tunneling protocols are often not very flexible, but a lot of time they do their job very well, and are easy to use in conjunction with IPsec. For a more modern and flexible option you may consider L2TPv3 or VXLAN — but that's a story for future posts.

Setting up GRE/IPsec behind NAT

In the previous posts of this series we've discussed setting up "plain" IPsec tunnels from behind NAT.

The transparency of the plain IPsec, however, is more often a curse than a blessing. Truly transparent IPsec is only possible between publicly routed networks, and the tunnel mode creates a strange mix of the two approaches: you do not have a network interface associated with the tunnel, but the setup is not free of routing issues either, and it's often hard to test whether the tunnel actually works or not from the router itself.

GRE/IPsec (or IPIP/IPsec, or anything else) offers a convenient solution: for all intents and purposes it's a normal network interface and makes it look like the networks are connected with a wire. You can easily ping the other side, use the interface for firewall and QoS rulesets, and setup dynamic routing protocols in a straightforward way. However, NAT creates a unique challenge for this setup.

The canonical and the simplest GRE/IPsec setup looks like this:

interfaces {
  tunnel tun0 {
    address 10.0.0.2/29
    local-ip 192.0.2.10
    remote-ip 203.0.113.20
    encapsulation gre
  }
}
vpn {
  ipsec {
    site-to-site {
      peer 203.0.113.20 {
        tunnel 1 {
          protocol gre
        }
        local-address 192.0.2.10

It creates a policy that encrypts any GRE packets sent to 203.0.113.20. Of course it's not going to work with NAT because the remote side is not directly routable.

Let's see how we can get around it. Suppose you are setting up a tunnel between routers called East and West. The way to get around it is pretty simple even if not exactly intuitive and boils down to this:

  1. Setup an additional address on a loopback or dummy interface on each router, e.g. 10.10.0.1/32 on the East and 10.10.0.2/32 on the West.
  2. Setup GRE tunnels that are using 10.10.0.1 and .2 as local-ip and remote-ip respectively.
  3. Setup an IPsec tunnels that uses 10.10.0.1 and .2 as local-prefix and remote-prefix respectively.

This way when traffic is sent through the GRE tunnel on the East, the GRE packets will use 10.10.0.1 as a source address, which will match the IPsec policy. Since 10.10.0.2/32 is specified as the remote-prefix of the tunnel, the IPsec process will setup a kernel route to it, and the GRE packets will reach the other side.

Let's look at the config:

interfaces {
  dummy dum0 {
    address 10.10.0.1/32
  }
  tunnel tun0 {
    address 10.0.0.1/29
    local-ip 10.10.0.1
    remote-ip 10.10.0.2
    encapsulation gre
  }
}
vpn {
  ipsec {
    site-to-site {
      peer @west {
        connection-type respond
        tunnel 1 {
          local {
            prefix 10.10.0.1/32
          }
          remote {
            prefix 10.10.0.2/32
          }

This approach also has a property that may make it useful even in publicly routed networks if you are going to use the GRE tunnel for sensitive but unencrypted traffic (I've seen that in legacy applications): unlike the canonical setup, GRE tunnel stops working when the IPsec SA goes down because the remote end becomes unreachable. The canonical setup will continue to work even without IPsec and may expose the GRE traffic to eavesdropping and MitM attacks.

This concludes the series of posts about IPsec and NAT. Next Friday I'll find something else to write about. ;)