Firejail with Tor HOWTO

A few years ago I created a set of scripts to start applications inside a linux namespace and automatically “Tor-ify” their network traffic. The main reason behind this effort was to provide some isolation and Tor support for applications that don’t have socks5 support, for example claws-mail. While this worked it was hard to keep adding sandboxing features like the ones firejail already provided. So I decided to take a look at how I could automatically send/receive traffic from a firejail-ed application through Tor.

This blog post is NOT meant to be used as copy/paste commands but to explain why each step is needed and how to overcome the problems found in the path.
If you have reasons to proxy all your traffic through Tor as securely as possible use Tails on a different machine, this guide is NOT for you.

A dedicated bridge
First of all create a Linux bridge and assign an IP address to it. Use this bridge to attach the veth interfaces that firejail creates when using the ‘net’ option. This option creates a new network namespace for each sandboxed application.

# brctl addbr tornet
# ip link set dev tornet up
# ip addr add 10.100.100.1/24 dev tornet

NAT
Then enable NAT from/to your “external” interface (eno1 in my case) for tcp connections and udp port 53 (DNS) and enable IP(v4) forwarding, if you don’t already use it. Some rules about sane default policy for FORWARD chain are added here well, modify to your needs.

# sysctl -w net.ipv4.conf.all.forwarding=1
# iptables -P FORWARD DROP
# iptables -A INPUT -m state --state INVALID -j DROP
# iptables -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
# iptables -A FORWARD -i tornet -o eno1 -p tcp -j ACCEPT
# iptables -A FORWARD -i tornet -o eno1 -p udp --dport=53 -j ACCEPT
# iptables -t nat -A POSTROUTING -s 10.100.100.0/24 -o eno1 -j MASQUERADE

This configuration is enough to start a sandboxed application that will have it’s traffic NAT-ed from the Linux host.

$ firejail --net=tornet /bin/bash
Parent pid 26730, child pid 26731

Interface        MAC                IP               Mask             Status
lo                                  127.0.0.1        255.0.0.0        UP    
eth0             72:cc:f6:d8:6a:09  10.100.100.29    255.255.255.0    UP    
Default gateway 10.100.100.1

$ host www.debian.org
www.debian.org has address 5.153.231.4
Host www.debian.org not found: 3(NXDOMAIN)
Host www.debian.org not found: 4(NOTIMP)
$ host whoami.akamai.net
whoami.akamai.net has address 83.235.72.202
$ curl wtfismyip.com/text
3.4.5.6

(where 3.4.5.6 is your real IP and 83.235.72.202 should be the IP address of the final DNS recursive resolver requesting information from whois.akamai.net)

So NAT works and the shell is sandboxed.

“Tor-ify” traffic
Edit /etc/tor/torrc and enable TransPort and VirtualAddrNetwork Tor features to transparently proxy to the Tor network connections landing on Tor daemon’s port 9040. DNSPort is used to resolve DNS queries through the Tor network. You don’t have to use IsolateDestAddr for your setup, but I like it.

TransPort 9040
VirtualAddrNetwork 172.30.0.0/16
DNSPort 5353 IsolateDestAddr

Then use iptables to redirect traffic from tornet bridge to TransPort and DNSPort specified in torrc. You also need to ACCEPT that traffic in your INPUT chain if your policy is DROP (it is right ?)

# iptables -t nat -A PREROUTING -i tornet -p udp -m udp --dport 53 -j DNAT --to-destination 127.0.0.1:5353
# iptables -t nat -A PREROUTING -i tornet -p tcp -j DNAT --to-destination 127.0.0.1:9040
# iptables -A INPUT -i tornet -p tcp --dport 9040 -j ACCEPT
# iptables -A INPUT -i tornet -p udp --dport 5353 -j ACCEPT

Run your sandbox again and try to access the same website:

$ firejail --net=tornet /bin/bash
$ curl wtfismyip.com/text
curl: (7) Failed to connect to wtfismyip.com port 80: Connection timed out

aaaand nothing happens. The problem is that you have tried to route traffic from a “normal” interface to loopback which is considered a “martian” and is not allowed by default by the Linux kernel.

sysctl magic
To enable loopback to be used for routing the route_localnet sysctl setting must be set.
# sysctl -w net.ipv4.conf.tornet.route_localnet=1

Try again:

$ firejail --net=tornet /bin/bash
$ host whoami.akamai.net
whoami.akamai.net has address 74.125.181.10
Host whoami.akamai.net not found: 3(NXDOMAIN)
Host whoami.akamai.net not found: 4(NOTIMP)
$ curl wtfismyip.com/text
176.10.104.243
$ host 176.10.104.243
243.104.10.176.in-addr.arpa domain name pointer tor2e1.digitale-gesellschaft.ch.

it works!

You can actually run any program you want like that:
$ firejail --net=tornet google-chrome

Accessing onion services
There’s one problem left though, accessing onion services.
If you try and access www.debian.org onion service from your firejail+tor setup you will get an error.

$ firejail --net=tornet /bin/bash
$ curl http://sejnfjrq6szgca7v.onion/
curl: (6) Could not resolve host: sejnfjrq6szgca7v.onion

To fix that you need to modify /etc/tor/torrc again and add AutomapHostsOnResolve option.
AutomapHostsOnResolve 1

$ firejail --net=tornet /bin/bash
$ curl -I http://sejnfjrq6szgca7v.onion/
HTTP/1.1 200 OK
Date: Fri, 09 Dec 2016 12:05:56 GMT
Server: Apache
Content-Location: index.en.html
Vary: negotiate,accept-language,Accept-Encoding
TCN: choice
Last-Modified: Thu, 08 Dec 2016 15:42:34 GMT
ETag: "3a40-543277c74dd5b"
Accept-Ranges: bytes
Content-Length: 14912
Cache-Control: max-age=86400
Expires: Sat, 10 Dec 2016 12:05:56 GMT
X-Clacks-Overhead: GNU Terry Pratchett
Content-Type: text/html
Content-Language: en

Accessing onion services works as well now.

Applications supporting socks5
If you already have some of your applications proxying connections to tor using 127.0.0.1:9050 then you need to add another iptables rule to redirect the socks traffic from inside firejail’s namespace to Tor SocksPort.
# iptables -t nat -A PREROUTING -i tornet -p tcp -m tcp --dport 9050 -j DNAT --to-destination 127.0.0.1:9050

Resolving OSSEC active response iptables issues

The past few days some of my servers are having difficult times due to the increase of spam by some botnet(s). From around 600-700 emails per day for unknown addresses/recipients on local domains, this number reached a peak of 8.000 emails 2 days ago. In order to reduce further botnet attempts I’m having ossec to engage, which in turn tries to firewall hosts.

That worked quite ok for a while but then I’ve started seeing errors in the active-response.log like the ones below:

Unable to run (iptables returning != 3): 1 – /var/ossec/active-response/bin/firewall-drop.sh delete – 91.121.21.8 1310919172.51029 31106
Unable to run (iptables returning != 1): 1 – /var/ossec/active-response/bin/firewall-drop.sh delete – 79.149.198.149 1310919524.52191 3302
Unable to run (iptables returning != 1): 2 – /var/ossec/active-response/bin/firewall-drop.sh delete – 79.149.198.149 1310919524.52191 3302
Unable to run (iptables returning != 1): 3 – /var/ossec/active-response/bin/firewall-drop.sh delete – 79.149.198.149 1310919524.52191 3302
Unable to run (iptables returning != 1): 4 – /var/ossec/active-response/bin/firewall-drop.sh delete – 79.149.198.149 1310919524.52191 3302
Unable to run (iptables returning != 1): 5 – /var/ossec/active-response/bin/firewall-drop.sh delete – 79.149.198.149 1310919524.52191 3302
Unable to run (iptables returning != 4): 1 – /var/ossec/active-response/bin/firewall-drop.sh add – 115.242.188.157 1310969220.1045522 3302

Obviously iptables is busy doing something else at the time, adding or deleting some other rule, so the loop inside firewall-drop.sh sometimes fails. That was a bit worrying, I had to fix ossec so one way or another so that iptables rules would eventually be applied. I’ve faced the same issue with iptables in the past, trying to simultaneously add multiple (>5) iptables rules at exactly the same time is very error prone, there’s no way to tell which of those rules will be applied. In order to circumvent the issue, I added locking to the active response script.

Whenever it comes to locking with shell scripts I am using a set of four functions inside a file that I source when I need to. I place this file usually inside /usr/local/bin/ under the lock.sh filename.

lockme () {
    if [ -z "$1" ];then
        echo " o Use an argument to lock"
        return 1
    fi
    if [ -z "$2" ];then
        PID=$$
    else
        PID=$2
    fi
    LOCK_PID_FILE=/var/lock/$1
    if [ -f $LOCK_PID_FILE ];then
        sleep 1
        echo " o Lock file found"
        if [ ! -d /proc/`cat $LOCK_PID_FILE 2>/dev/null` ];then
            echo " o Stale lock file ignoring..."
            rm -f $LOCK_PID_FILE
        else
            return 1
        fi  
    fi  
    #temp file
    echo -n $PID > $LOCK_PID_FILE.$PID
    ln -s $LOCK_PID_FILE.$PID $LOCK_PID_FILE && return 0
    rm -f $LOCK_PID_FILE.$PID
    return 1
}

lockme_wait () {
    if [ -z "$1" ];then
        echo " o Use an argument to lock"
        return 1
    fi  
    if [ -z "$2" ];then
        PID=$$
    else
        PID=$2
    fi  
    while [ 1 ];do
        lockme $1 $PID && break
        sleep 4
    done
    return 0
}

unlockme () {
    if [ -z "$1" ];then
        echo " o Use an argument to unlock"
        return 1
    fi
    #remove pid file
    rm -f /var/lock/$1.`cat /var/lock/$1 2>/dev/null`
    rm -f /var/lock/$1
    return 0
}   

kill_locked () {
    if [ -z "$1" ];then
        echo " o Use an argument to kill_locked"
        return 1
    fi
    if [ -e /var/lock/$1 ]; then
        kill `cat /var/lock/$1 2>/dev/null`
    fi
    rm -f /var/lock/$1.`cat /var/lock/$1 2>/dev/null`
    rm -f /var/lock/$1
}

You can also use %s/var\/lock/tmp/g if you prefer having the locks on the /tmp which is usually ramfs, partition.

Afterwards I edited /var/ossec/active-response/bin/firewall-drop.sh to just add 3 lines. (I only edited the relevant Linux section of the script, since I haven’t tested, or don’t even know if it’s needed on the BSD, SunOS sections, I left those unedited):

  • Add . /usr/bin/lock.sh right after the “# Checking for an IP” section (around line 45)
  • Right after “# Executing and exiting” add lockme_wait active-response (around line 75)
  • Right after the second while loop finishes, after “done” and before “exit 0” add unlockme active-response (around line 110)
  • That’s it…just 3 lines added and the errors have completely stopped since then.

    P.S. Yes, I could have used lockfile-progs to achieve the same result, but I (also) use lock.sh file in embedded systems when needed, and it’s far more portable and easy.

    Stopping Plesk Panel attacks with OSSEC

    During the past few weeks I’ve noticed increased brute forcing activity on various servers that I manage and run Plesk Panel. Most of the entries look like this:

    189.205.227.115 - - [30/Jan/2011:07:14:19 +0100] "GET /login_up.php3?passwd=setup&login_locale=default&login_name=admin HTTP/1.1" 200 5852
    189.205.227.115 - - [30/Jan/2011:07:14:19 +0100] "GET /login_up.php3?passwd=setup&login_locale=default&login_name=admin HTTP/1.1" 200 5852
    189.205.227.115 - - [30/Jan/2011:07:14:19 +0100] "GET /login_up.php3?passwd=setup&login_locale=default&login_name=admin HTTP/1.1" 200 5852
    189.205.227.115 - - [30/Jan/2011:07:14:21 +0100] "GET /login_up.php3?passwd=setup&login_locale=default&login_name=admin HTTP/1.1" 200 5852
    189.205.227.115 - - [30/Jan/2011:07:14:21 +0100] "GET /login_up.php3?passwd=setup&login_locale=default&login_name=admin HTTP/1.1" 200 5852
    189.205.227.115 - - [30/Jan/2011:07:14:23 +0100] "GET /login_up.php3?passwd=setup&login_locale=default&login_name=admin HTTP/1.1" 200 5852
    189.205.227.115 - - [30/Jan/2011:07:14:23 +0100] "GET /login_up.php3?passwd=setup&login_locale=default&login_name=admin HTTP/1.1" 200 5852
    

    The side effect of all these attacks is increased server load.

    Since I already have ossec monitoring these servers the solution was quite simple. I just added a couple more rules to ossec in order to stop these attacks.

    Two steps are necessary to stop these attacks:
    1) Add plesk panel https log to monitor list in /var/ossec/etc/ossec.conf

      <localfile>
        <log_format>apache</log_format>
        <location>/opt/psa/admin/logs/httpsd_access_log</location>
      </localfile> 
    
      <localfile>
        <log_format>apache</log_format>
        <location>/opt/psa/admin/logs/httpsd_error_log</location>
      </localfile>
    

    2) Create some custom rules to block (and notify me) of these attacks.

    <rule id="100144" level="1">
        <if_sid>31100</if_sid>
        <id>200</id>
        <url>/login_up.php3</url>
        <description>Plesk Login.</description>
      </rule>
    
    <rule id="100145" level="12" frequency="3" timeframe="60">
        <if_matched_sid>100144</if_matched_sid>
        <same_source_ip />
        <description>Attack on plesk panel.</description>
        <group>attack,</group>
      </rule>
    

    That’s it. Ossec now monitors these files and blocks through iptables any attacks with active-response.

    Example notification mail:

    Received From: foo->/opt/psa/admin/logs/httpsd_access_log
    Rule: 100146 fired (level 12) -> "Attack on plesk."
    Portion of the log(s):
    
    189.205.227.115 - - [02/Feb/2011:20:19:56 +0100] "GET /login_up.php3?passwd=setup&login_locale=default&login_name=admin HTTP/1.1" 200 5852
    189.205.227.115 - - [02/Feb/2011:20:19:55 +0100] "GET /login_up.php3?passwd=setup&login_locale=default&login_name=admin HTTP/1.1" 200 5852
    189.205.227.115 - - [02/Feb/2011:20:19:54 +0100] "GET /login_up.php3?passwd=setup&login_locale=default&login_name=admin HTTP/1.1" 200 5852