What is all about
short story ...
This post will help you to configure a linux PC in order to function as a router too.
long story ...
If you like me have a very low energy consumption PC (a NAS equivalent) running all the time you might prefer it to act as a router too. This way you'll be able to:
- use the full Linux power to control the network traffic (especially the malicious connections)
- use the better performing PC hardware (compared to one of a dedicated router) to deal with the network traffic
- have fun because you're a linux enthusiast :)
The setup explained below uses NetworkManager.service; if you use something else the main difference will be related to configuring the pppoe connection while the other aspects should be the same or anyway helpful for your setup.
But using your PC as a router doesn't mean you won't be able to use it for something else too. I for example use my PC as a router while also as a desktop PC, as a server (for this blog, Transmission, ssh, nginx, etc) and as HTPC (Plex based).

What's to achieve
In the end you'll achieve these:
- connect directly to Internet using your PC router
- Internet users directly access your websites running on your PC router
- when having at least 2 ethernet cards you'll use one for Internet access while the other to setup a LAN
- with 2 ethernet cards one could be connected to a dedicated wireless router; its wireless users could be considered part of a LAN accessing the Internet through the PC router (the gateway for the dedicated router)
- secure your PC router against malicious Internet access
- setup other goodies e.g. dnsmasq with or without dhcp, sshttp, Plex

How to do it
In order to achieve the above you'll have do these:
- secure the access to your PC router
- setup a pppoe connection in order to access the Internet
- share the Internet access
- setup dnsmasq (NetworkManager's plugin) in order to ... long story, I'll explain later
- setup a dedicated wireless router in order to have wireless access to Internet when your PC router isn't able to provide by itself wireless access
- solve miscellaneous other issues e.g. dealing with sshttp and/or Plex

Secure the access to your PC router
This is a vital step!
You should do this first before having your PC accessed from all over the Internet.
You can do this by using the default firewall of your Linux distribution, e.g. for Ubuntu is UFW (Uncomplicated Firewall) while for RedHat/CentOS/Fedora is firewalld (check firewall-cmd man page and usage examples here and here).
Before continuing just check your opened ports with the commands below.
List opened ports using UFW:
sudo ufw status numbered
List opened ports using firewalld:
firewall-cmd --get-active-zones
firewall-cmd --list-ports

Setup a pppoe connection in order to access Internet
Use your graphical NetworkManager connection editor (nm-connection-editor on Ubuntu) in order to create a DSL connection (e.g. named RDS). In General tab check the options Automatically connect to this network when it is available and All users may connect to this network. In DSL tab fill in the username and password handed to you by your Internet provider. In Ethernet tab let MTU to automatic (it won't apply to pppoe connection) and choose the card which will be used for Internet access (e.g. eth0). In IPv6 Settings tab disable ipv6 connections if you don't have a reason to use it; if you intend to use it then this post won't help you.

Check the pppoe setup
On Ubuntu you'll be able to see your configuration from the command line:
sudo cat /etc/NetworkManager/system-connections/RDS
or using the graphical NetworkManager applet (Connection Information menu).

With the ifconfig command you'll see a new network interface (e.g. ppp0) when the pppoe connection is active.
Using the command below:
nmcli connection show
you'll also see that the pppoe connection is related to eth0 (chosen by you when creating the RDS connection).
With the command below:
nmcli device show
you'll see that eth0 has as IP4.GATEWAY the ip of your internet provider.
Check the pppoe connection with these commands too:
ifconfig ppp0
netstat -i

The MTU configuration
When MTU of your pppoe connection is not correctly set you'll experience internet web pages hanging/loading forever. 1500 is the maximum MTU possible and seems to be the default for the ethernet devices. For pppoe connections the maximum MTU is 1492. Check more about these at http://www.dslreports.com/faq/695.

You'll have to edit manually the [ppp] section in /etc/NetworkManager/system-connections/RDS in order to add/change it:

[ppp]
mru=1492
mtu=1492
With mtu=1492 the commands below:
sudo ip route flush cache
ping -c 1 -M do -s 1464 8.8.8.8
should yield among other:
1 packets transmitted, 1 received, 0% packet loss
or an error similar to the below:
ping: local error: Message too long, mtu=1492
1 packets transmitted, 1 received, 0% packet loss
If ping with 1464 (1464 = 1492 - 28) value yields an error then change it to a lower value e.g. subtract 10 then try again and so on. When found the maximum working value add 28 to it then use it for [ppp] section in RDS and restart RDS connection (use the NetworkManager applet to disconnect then reconnect).

When an ip package flows through e.g. eth1 (another ethernet card on your PC router) to ppp0 a MTU conversion must be done. This is accomplished with iptables or with the help of the firewall e.g. UFW. After finding the proper MTU you'll have to put this in /etc/ufw/before.rules: 

-A ufw-before-forward -p tcp -i eth1 -o ppp0 --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1452
-A ufw-before-forward -p tcp -i eth1 -o ppp1 --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1452

just before # ufw-not-local though when left at the end I guess it will work too. Replace 1452 with your pppoe MTU (e.g. 1492 for RDS) minus 40 (TCP and the IP header, 20 bytes each).

Why adding 28 and why using 1464 in the first place? check at http://www.dslreports.com/faq/695.

There are other commands that show the MTU value:
ip ad
netstat -i
ifconfig ppp0 | grep MTU
but they don't provide you an option to test a wrong MTU value (as ping does).
The MTU for eth0 (used by ppp0) should be 1500.

There's another way of testing MTU value but with a more complicated setup. It works like this: on another computer (PC2) using e.g. eth0 (192.168.0.1) and connected to your PC router on e.g. eth1 run the command below in order to check received network packets:
sudo tcpdump -i eth0 --direction=in -n ip proto \\icmp
then from your PC router send network packets like this:
ping -c 1 -s 1472 -I eth3 192.168.0.1
ping -c 1 -s 1464 -I eth3 192.168.0.1
For any packet received on PC2 you'll get one line of console output so when the ping value (1464, 1472) is too large you'll see more than one line in PC2's console. You should change the ping value till you reach the maximum one while still showing only one line in PC2's console. Then to that maximum value add 28 and that's will be the MTU for the connection PC router on eth1 to PC2 on eth0.
check also http://ping.online-domain-tools.com/

I have no idea how to check the current MRU value but seems a good idea to set it to the same value as MTU; please post a comment when you have a clue about it.

Share the Internet access
You'll have to enable packet forwarding by editing /etc/sysctl.conf:
net.ipv4.ip_forward=1
then activate the new sysctl configuration with:
sudo sysctl -p
Check current configuration with:
sysctl net.ipv4.ip_forward

Also you'll have to configure your firewall to allow ip forwarding.
e.g. with UFW you'll have to edit /etc/default/ufw to have this:

DEFAULT_FORWARD_POLICY="ACCEPT"

In /etc/ufw/before.rules you'll need:

*nat
:PREROUTING ACCEPT [0:0]
:POSTROUTING ACCEPT [0:0]
# when having no other *nat rules uncomment the line below:
# -F
-A POSTROUTING -o ppp0 -j MASQUERADE
-A POSTROUTING -o ppp1 -j MASQUERADE
COMMIT

At this point when having multiple ethernet cards you'll be able to share the internet connection with them. This means that a PC2 directly connected to PC router's eth1 will be able to access the Internet but only with a proper configuration: 
- PC2 must have an ip in the same network class as PC router's eth1
- PC2 must have the gateway pointing to PC router eth1's ip
- PC2's DNS servers must be the same as those used by PC router (check nmcli device show eth0 | grep '.DNS')
This setup is an annoying complication mostly because of the DNS setup which might change depending on the Internet provider. The following section solves this with the help of a DNS and DHCP server.

Internet connection sharing: the big picture
Let's suppose that your PC router has an additional network interface (e.g. eth1). You could connect to it:
a) another PC on a wired connection when eth1 is wire only accessible
b) many other wireless devices when eth1 is a wireless device
c) a dedicated wireless router (when eth1 is wire only accessible) in order to share the Internet connection with other wireless and wired devices
For the b case you'll need to setup dnsmasq as a DNS and DHCP server. For a and c you won't really need the DHCP server but won't harm you anyway.

Setup dnsmasq as a DNS and DHCP server
When using NetworkManager then dnsmasq is already used as a plugin; just check /etc/NetworkManager/NetworkManager.conf for something like dns=dnsmasq. You'll need to customize dnsmasq's configuration; create the file /etc/NetworkManager/dnsmasq.d/custom-dnsmasq.conf with the following content:

addn-hosts=/etc/hosts-dnsmasq.conf
local-ttl=3600
log-facility=/var/log/dnsmasq/dnsmasq.log
interface=eth1
except-interface=eth0
except-interface=ppp0
strict-order
all-servers
clear-on-reload
cache-size=5000
dhcp-range=192.168.0.2,192.168.0.255,255.255.255.0,192.168.0.255,1h
dhcp-option=option:router,192.168.0.1
dhcp-option-force=option:mtu,1500
dhcp-lease-max=1
log-dhcp
dhcp-leasefile=/var/log/dhcpd.leases.log

Make sure to create /var/log/dnsmasq/ (owned by root only) used for keeping dnsmasq.log.

Be aware to exclude with except-interface at least the pppoe connections (e.g. ppp0) and the network interfaces used by them (e.g. eth0). You can change the cache-size in case you want less RAM to be used. Related to dhcp-range I assume you have only one network interface available (e.g. eth1) besides the one used for the pppoe connection (e.g. eth0). So when something is connected to eth1 it will automatically get the proper ip (between 192.168.0.2 and 192.168.0.255) and the DNS configuration. On your side eth1 should have the ip 192.168.0.1 and no gateway or DNS configured. 

I don't know what one should do when having multiple network interface available; the problem is with the dhcp-option=option:router,192.168.0.1 which should be different for every interface.

Sometimes you'll notice that the network won't start with dnsmasq complaining that can't bind port 53 to 192.168.0.1 (see interface=eth1 option). This happens because sometimes eth1 (having 192.168.0.1 ip) is activated after dnsmasq. The solution I found is to start with the "interface=eth1" option commented; after eth1 is started I uncomment it then kill dnsmasq which will then be restarted automatically by NetworkManager. On PC router shutdown or eth1 down I'll have to comment again the "interface=eth1" option and do again the uncommenting-kill-dnsmasq after restarting eth1.

For the uncommenting and dnsmasq killing part I use /etc/network/if-up.d/eth1-up:

For the commenting part I use /etc/network/if-post-down.d/eth1-post-down:

I notice anyway that when shutdowning PC router the eth1-post-down script above doesn't work so I also use /etc/systemd/system/NetworkManager.service.d/network-manager-override.conf:

You'll also have to open the DNS (53) and DHCP (67) ports only on eth1:

sudo ufw allow in on eth1 to any port 53 comment 'allow DNS access from LAN'
sudo ufw allow in on eth1 to any port 67 comment 'allow DHCP access from LAN'

Useful commands:

to be continued ...

see also https://www.incapsula.com/blog/mtu-mss-explained.html