I have two IPv6 tunnels with subnets, one from SixXS and one from Hurricane Electric. (Why? Eh, I'm a tinkerer.) I had them both active and responding to tunnel broker pings on my Cisco E2000 with DD-WRT v24-sp2 rev 14929 firmware, but I couldn't route from both my subnets at the same time.
My SixXS tunnel won't accept packets from my HE subnet, and my HE tunnel won't accept packets from my SixXS subnet, but I can only route by destination...with normal routing.
The answer—besides living happily with one subnet and tunnel—is policy routing. With policy routing I can set up more than one routing table and choose which routing table to use based on the source of the packet, so I should be able to send SixXS-subnet-sourced packets to the SixXS tunnel and HE-subnet-sourced packets to the HE tunnel.
Unfortunately my DD-WRT version doesn't support policy routing for IPv6. So I designated my Linux server as the DMZ host under the DMZ tab under the NAT / QoS tab and set up my tunnels and routing on the Linux box.
Instead of setting up a piece at a time and testing it as I go, I brazenly typed up my whole configuration before trying any of it out on the Linux box. That cost me a few hours of troubleshooting, but the funny part is I mostly had it right the first time. The problem was that policy rule flushing works slightly differently with IPv4 and IPv6, at least on Ubuntu Lucid 10.04.2.
The default rules are as shown:
The "gotcha" is that I decided to flush the rules before adding my own. I did this because I was duplicating rules when I brought the interface down and up again.
Flush works a little too well as it takes away the rule that uses the main routing table! Flushing the IPv4 rules puts the default rules back, but not so with IPv6. So when flushing IPv6 rules remember to add the main rule back:
Here is my working policy routing setup that routes to the proper tunnel based on the source address. I have chosen to set up the HE tunnel and subnet as normal and make policy routing decisions for the SixXS tunnel. Other things I did to make this work was to enable IPv6 routing in sysctl and add "200 sixxs" to /etc/iproute2/rt_tables so I could have a routing table named "sixxs", but I could have used a numbered table instead. Ubuntu/Debian /etc/network/interfaces file (partial):
I null-route my assigned prefixes/subnets to avoid sending inappropriate traffic back into the internet which will just get routed right back to me. When I add routes for my /64 prefixes they will override the null route because they more specifically match the destination addresses. I am also routing to 2000::/3 (all currently assigned global unicast addresses) instead of a default route so I don't send out multicasts or other address ranges I don't intend.
If you're wondering about "|| true", if I don't have that and the command generates an error code, the interface set-up would stop. The "|| true" causes the command to return a "success" code to the ifup/ifdown scripts even if the command fails so that the ifup/ifdown script can continue with the rest of the setup.
My plan is that the main routing table will handle all cases except from the SixXS subnet to the SixXS tunnel, in my case from the subnet to external global unicast addresses. I could just make sure each table has all the routes I need, but in my case I think it's easier this way. Here are the resulting rules from the above configuration:
Rules 100, 200 and 300 are my local prefixes. They don't need to go out the SixXS tunnel, so I divert them to the main routing table. Rule 600 is the reserved documentation prefix. I think I may try to use that address range when making how-to videos, so I want to divert that to the main routing table and null route it there to prevent sending invalid traffic out to the Internet. Rule 32000 catches all remaining packets from my SixXS subnet that are to the global unicast address range and tells Linux to use the routing table I named "sixxs". The last rule is what sends everything else to the main routing table. The last rule should exist by default, but "ip -6 rule flush" deletes it, and that's the "gotcha" that cost me a few hours' troubleshooting.
The sixxs routing table as configured above: