Debugging NAT / prerouting issues (iptables)

[Vlad]

Hello,

Recently I discovered an issue with packet routing in the latest Android releases (4.4+ KitKat & Lollipop).
It seems that the problem Android specific, but essentially it comes from the Linux kernel.

I already filed a bug report to Google. You can see the details by searching for 'Android Issue 78910' - it's the first result that pops up.
However, I will be very grateful if anyone could suggest how to debug this in order to resolve it faster.

In a nutshell, my setup consists of 2 interfaces = wlan0 (WiFi AP) and tun0 (VPN tunnel).
My goal is to provide full VPN access to one of the WiFi clients, while additionally forwarding all traffic coming from a specific IP in the VPN to the same client as well.
In my test case the WiFi client address is 192.168.100.50 and the VPN address in question is 172.28.26.36.

I am adding the following IPTABLES rules in order to do that:

echo 1 > /proc/sys/net/ipv4/ip_forward;
iptables -t nat -A POSTROUTING -o tun0 -j MASQUERADE;
iptables -A FORWARD -m state --state RELATED,ESTABLISHED -j ACCEPT -i tun0 -o wlan0;
iptables -A FORWARD -j ACCEPT -o tun0 -i wlan0; iptables -t nat -A PREROUTING -s 192.168.100.50 -j DNAT --to-destination 172.28.26.36;

This was working perfectly fine before upgrading to Android 4.4.
When I check the IPTABLES stats I can see that my rules are no longer matching any of the packets.
Monitoring the traffic with TCPDUMP shows that the packets are received and have the correct parameters, so it seems like they are not handled properly by the kernel.

I have downloaded the Android kernel source code, but a lot of things have changed between 4.3 and 4.4, so it won't be easy to find the cause there without strong networking knowledge (which I don't have). The main thing is that Google added UNIX-like multi user support to the OS, which required all of the previously used routing rule setting mechanics to be changed.

As I said, I am quite new to Linux networking, so any help and suggestions will be greatly appreciated.
I am not relying on Google to solve this, as this bug has been out there since last December and apparently the impact is not big enough for them to prioritize (or even acknowledge) it.
However, this is a major setback and a show stopper for my product.

Thanks!

 

[ryanvade]

According to this page:
https://source.android.com/devices/tech/security/enhancements44.html
VPN routing is done on a per-user basis which is probably causing some issues since this change was just made (as you pointed out). I do know that IPtable rules are reset after every reboot on Android so that may be an issue. Echoing 1 to anything in /proc is reset as well.

 

[Vlad]

I have made an application that enables / disables the routing on demand, so having the rules cleaned up on restart is not really an issue.

In version 4.3 all of the VPN routes were set with 'ip route' and 'route' commands, but in order to support a multi user environment Android now makes use of 'iptables' instead.

It seems very weird that the packets coming on wlan0 would be handled by the 'iptables' of another user, but it is one possible scenario (it would also explain why none of the packets are matched by the rules and the 'iptables' counters stay at 0).

I still need to prove however whether that is the case. Could you suggest any way to test this? As far as I can tell, it might have something to do with which user is running the iptables service, so I will check that first.

However, all the commands that I am executing are with 'root' and there are no other user profiles on the device (system services like iptables are also managed by 'root' anyway).