Behind The Firewall: Connecting Service Despite Restrictions

Here's the problem:

You have spent a lot of money for some internet service, and being the good administrator you are you'd like to share this service among your very small collection of machines.  You also want to put a firewall in the way of unnecessary and/or unwanted network traffic, because after all, the more hoops people have to jump through to hit your systems, the better.

Now, you've got your firewall connected, and you can ping the world and even browse to places on the firewall.  But when you connect a system behind the firewall, every request out seems to die.  You've checked that the firewall was properly configured, and you had even tested it back at the office.  You can still reach the world from the firewall, and yet from your client machine you can't go anywhere.

After a Wireshark capture of the traffic, you see your client machine traffic going out, and some interesting ICMP messages that say "Time to live exceeded in transit" or "TTL expired in transit."  What's more, you notice that your firewall is sending these messages back out to the servers you're trying to talk to!  It is as though no matter what comes in, its TTL exceeded.

And that is exactly what is happening.  Let's create this situation on purpose.  If we have a NATing firewall that is to be the only NAT device in the network, we don't want to allow any other NAT devices to deliver their traffic to their clients.  It's endpoints or nothing at all.  In iptables, the following rule should accomplish this:

iptables -t mangle -A PREROUTING -i eth0 -j TTL --ttl-set 1

(Refer to this page for more information: http://www.linuxtopia.org/Linux_Firewall_iptables/x4799.html)

The above command should mangle the TTL such that there is only one more hop it can go before it runs out of life.  I haven't tested that - if it doesn't work, try 2 instead of 1.  The above command is really based upon the fix for the problem the above command causes.  Basically, if you're experiencing an issue with the TTL expiring too soon, just rewrite it!

iptables -t mangle -A PREROUTING -i eth0 -j TTL --ttl-set 255

Now we can do another 255 hops before we run out of TTL.  As this is done in the prerouting, the packet is fixed before any checks are made as to whether or not it should be dropped for TTL expiration.  A similar fix can be done with pfSense (search in google for the appropriate terms and make sure to tread carefully when mucking around with the filter code - note that there IS NO GUI OPTION FOR CHANGING THE TTL IN PFSENSE!!).  An alternative to the above is to increment the TTL by 1 or more.

If you have a pfSense VM running on a Linux hypervisor, you can make the fix right in Linux (for the bridge adapter, of course).  IPTables in this way really saves the day.  The fix in pfSense is not terrible, but also not convenient and is not officially supported by their community.

Now that I think about it, this would be a great way to stop people from using unauthorized wireless access points around the office....

Ubuntu Server 12.04 - Waiting for Network Configuration

Just ran across an interesting issue, and the forums I've read so far don't provide a real clear answer.  I don't know that this is the answer, either, but it may be worth pursuing.  This is a bit of stream-of-consciousness, by the way - my apologies.

I just set up some new servers and was in the midst of securing them.  The first server I started on has a static IP, a valid gateway, valid DNS server, and all the networking checked out.  On reboot, however, it would take forever to kill bind9, and then I'd see almost two minutes worth of "Waiting for network configuration."  Well, there are only statically-assigned adapters present, and the loopback (which was left in its installer-default state).

I had introduced a slew of rules via iptables and I suspect they were wreaking havoc with the boot/shutdown procedures.  If someone else is experiencing this problem, try nuking your iptables and make sure it doesn't reload on reboot - hopefully you'll see everything come back up quickly.  UFW users would obviously need to disable ufw from operating.  FWIW, I placed my iptables loader script in the /etc/network/if-pre-up.d/ folder, so it's one of the first things to crank up when networking starts.

Now, I have similar iptables configurations present on other machines, and I don't know that those machines specifically have the same problem.  That being said, I really haven't rebooted them frequently enough to notice.

* * * * *

After a bit more experimentation, it appears there is some dependency on allowing OUTPUT to the loopback.  Specifically, I'm looking at logs that note packets being sent from my machine's configured static address to the loopback, and consequently they're being dropped by my rules.  They're TCP packets to port 953.  This apparently rndc, and related to BIND, which makes sense since my other machines do not run BIND daemons.

This rule, while not the most elegant, and probably not the most correct, fixes the issue for now:

-A OUTPUT -m comment --comment "rdnc" -o lo -p tcp --dport 953 -j ACCEPT

It is probably important to note that this machine is not a gateway and so drops any packets that would be forwarded.  I suppose I'm hoping this will be secure, but I just get a strange feeling something more needs to be done.

More on this later, hopefully.