The Armored Blog

Know what we are thinking and doing

June 08, 2016 / by Ghaith /

DIY Home Network Security with NxFilter and Security Onion

As the title suggests, let’s go through how to set up a DIY home NSM (network security monitoring) solution on the cheap (read: free).

Note 1: The tutorial assumes you have basic computer and networking knowledge and doesn’t go over every configuration detail. You are expected to do your research and configure accordingly.

Note 2: Our solution uses one NIC for both monitoring and management. The IP address of our NSM is going to be Not ideal in a production setup, but perfectly fine for home use. OK, onto the tutorial!

Software you will need:

Hardware you will need:

  • A Linux-based router (running Tomato, OpenWRT, or any other flavour of Linux where you can use iptables to control traffic. I’m using an ASUS rt-ac66u with the stock software.)
  • A spare machine (physical or virtual) to install your NSM tools on. This machine should be connected via Ethernet to your router for optimum results.

Once you have all the tools, proceed as follows.

Step 1: Install and configure Security Onion on your spare (virtual) machine

If you’re installing to a VM on Virtualbox, Doug Burks has an excellent howto here.

If you’re installing onto a physical machine, burn the ISO, boot to Live CD, and install. Again, I won’t repeat those instructions because Doug Burks already did a great job here.

Don’t forget to follow the instructions for setup, in that same link above.

Step 2: Install and configure NxFilter on the same machine

Security Onion is based off Ubuntu, so you can use the Linux installation instructions for NxFilter.

Again, NxFilter has some excellent documentation, so I won’t repeat it here.

You’ll know it’s working properly when you point one of the other machines on your network to use it as a DNS server and name resolution works properly.

Once NxFilter is set up and running correctly, set your router to hand out the NxFilter IP address as a DNS server, from LAN > DHCP Server > DNS Server:

Step 3: Configure your router to mirror all traffic to your new NSM

This is done using 2 commands to iptables. On the ASUS rt-ac66u, you can do it by first enabling telnet using the web interface (Administration > System > Enable telnet):

Once you have telnet or SSH access into your firewall, log into it and issue the follow 2 commands (replace the IP address with that of your Security Onion installation):

iptables -A PREROUTING -t mangle -j ROUTE –gw –tee

iptables -A POSTROUTING -t mangle -j ROUTE –gw –tee

The above commands will make a copy of all of the traffic on your network to the IP address

Your router is now mirroring a copy of all traffic to your NSM installation. Let it run for a little while and then browse to your NSM e.g From there you can browse to the various tools: Squirt, ELSA, and Snorby, and learn all sorts of cool stuff about your network that you didn’t know.

Using these tools is beyond the scope of this installation tutorial, so again do your research to configure them to drop all the noise and log only what matters to you. I personally found this page helpful.

Step 4: Block outbound DNS queries, except for NxFilter

NxFilter’s power shines when it is the choke point for outbound DNS queries. If it can be bypassed, it can be defeated. We want to prevent any hosts or application on the network from being able to go around it.

This is accomplished by denying outbound DNS queries on the firewall to all hosts, except the NxFilter IP address.

On the RT-AC66U this is done via the web interface. Go to Firewall > Network services filter and add whitelist rules such as:

The above rules essentially say:

  • permit only to make outbound connections to UDP port 53 (DNS)
  • permit everyone else to make outbound connections to all other ports UDP and TCP

You’ll have to apply similar rules to your firewall to limit outbound DNS queries to the NxFilter machine only.

In conclusion

At this point we’ve:

  • Installed NxFilter, and made it our local LAN DNS server
  • Configured the firewall so that only NxFilter can make outbound DNS queries
  • Configured Security Onion such that the firewall is logging a copy of all packets to it

Have fun!