Own Homeserver - Setup Part 2 Raspberry Pi

This series of instructions is about setting up your home server environment. So that one blog post becomes manageable, I have divided the setup into several posts. Here I’ll be looking at my Raspberry Pi. In particular, it will serve as a proxy manager and ad blocker.

Requirements

You don’t necessarily need a Raspberry Pi for a proxy manager or ad blocker; in principle, it should work with any device Docker can be installed.

A Raspberry Pi has the advantage that it is inexpensive and consumes relatively little power. A Raspberry Pi 3b+ requires just under 2 watts in idle mode, a Raspberry Pi 4b just under 3 watts. Under load, the values increase to approx. 5-6 watts (3b+) and 6-7 watts (4b) respectively. If a device runs 24 hours a day, 365 days a year, each watt costs around €2.45 per year at an electricity price of 28 cents/kWh. This means that even under constant full load (quite unlikely), the electricity costs would increase by a maximum of €17 per year. Added to this are the connected appliances.

Enter data here to calculate the electricity costs:

Install OS

First, an operating system must be installed on the Raspberry Pi. A USB stick (or USB hard disk) or a microSD card can be used as a storage medium. With an older model than the Raspberry Pi 4b, limitations may need to be taken into account. Otherwise, just make sure that the power supply is sufficient.

I use a Crucial SSD with 128 GB of storage space packed in an external housing.

It used to work in another device and is therefore not displayed when I want to install an operating system later on. If you also use Windows and have the same problem, open the administration via the control panel.

There we are interested in the link to “Computer Management”. The menu item “Disk Management” is on the left side.

I want to use the disk for my Raspberry Pi, so I delete the individual volumes first.

Now follows the download of the Raspberry Pi Imager and its installation. The download button can be found on this page.

The official Raspberry Pi OS (formerly Raspbian) has three versions: Lite, Desktop (standard), and Full. The Lite version has no graphical user interface and fewer pre-installed programs. In return, it uses less disk space and memory. In addition to the desktop, Full also contains other programs that may be useful. Unfortunately, the 64-bit version is currently only available as a beta. Raspberry Pi OS is also based on Debian.

Should you necessarily use a 64-bit operating system with more than 3 GB of RAM? No, because the system can continue to use the entire RAM.

Our default operating system image uses a 32-bit LPAE kernel and a 32-bit userland. This allows multiple processes to share all 8GB of memory, subject to the restriction that no single process can use more than 3GB

• Raspberry Blog

I choose the Lite version of the Raspberry Pi OS, because an interface is not needed.

I select the hard disk I deleted as the “SD card” and press “Write”. Once the write process is complete, I remove and reconnect the hard disk. I see it in the data explorer with the label “system-boot”.

A drive letter must be assigned to the system-boot partition if the drive is not visible.

After I switch on the display for file name extensions via the view tab, I create a new file.

This file must be named ssh.

The file extension must be deleted, it is not a text file.

The installation is now complete.

Connecting the Raspberry Pi to the network

This guide uses a Raspberry Pi 4b with 8 GB RAM.

I connect the drive I just flashed to the Raspberry’s USB3 port and use a network cable to connect it to my router or a switch. Now I need the IP address of the Raspberry. To do this, I open the terminal (alternatively PowerShell).

I want to use this to find out the gateway address.

ipconfig

In my case, the default gateway should now be “192.168.0.1”. This address must be called up in the browser to reach the router under normal circumstances. After you have logged in there, you will see a list of the devices in the network. The new Raspberry Pi should also be on the list.

IP address of the Raspberry Pi](@assets/images/blog/setup-raspberry/images/raspberry-ip.png)

In my case, it has been assigned the address 192.168.0.47.

Vodafone station problem

My Raspberry Pi is no longer assigned an IP address on the Vodafone station after I reinstalled it. I suspect it has something to do with the Vodafone Station as a DHCP server. I have often read about the problem of Raspberry Pis without an IP address, but I have not found a simple solution.

Fixed IP assignment

It is advisable to assign a permanent IP address in the router settings if this is possible.

In a Fritz!Box clicking on the device’s details is possible in the Home Network/Network/Network Settings menu.

Fixed IP address assignment](@assets/images/blog/setup-raspberry/images/feste-ip.png)

After the IP address is known, it is time to set up the system.

Set up Raspberry Pi OS

Suppose you are not using a Raspberry Pi OS. In that case, most instructions can also be transferred to Ubuntu or Debian.

Connect

First, we need to connect to the Raspberry Pi. The login data for the Raspberry Pi are username: pi and password: raspberry. This is also the reason why ssh is disabled by default. The username and password are not secure. Open a terminal to log in:

ssh pi@192.168.0.47
yes
raspberry

Update

Now we first update the packages.

sudo apt update
sudo apt upgrade
sudo apt dist-upgrade

Confirm with Y if necessary.

Password

Now it is essential to change the password if it was not changed at the beginning of the SSH session.

passwd
raspberry
newPassword
newPassword

The new password should be more secure.

Time setting

Now we set the correct time if required.

date

If the correct time is not displayed:

tzselect

You can select your time zone using the numbers.

sudo timedatectl set-timezone 'Europe/Berlin'

Monitoring

With the command htop, we can see the system load.

htop

This makes it easy to see how much the CPU and RAM are used. Press F10 or CTRL + C to exit the view. However, not only the load but also the temperature is essential.

cat /sys/class/thermal/thermal_zone0/temp

This displays the temperature in degrees Celsius * 100.

Git

Next, we need git.

sudo apt install git

Host Name

We can also rename our Raspberry.

sudo nano /etc/hostname

The file is exited in Nano with Ctrl + X. Nano will ask if you want to save the changes (confirm with Y) and what the file should be called. Since we want to overwrite it, we do not change the name and simply confirm with Enter.

Docker

Now we need Docker.

curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
rm get-docker.sh

Be careful with curl, as you could download something dangerous The command sudo docker version should now output the version of Docker.

Next, we need Docker-Compose. Under this link we can see which version is the latest. Unfortunately, we would get an older version via apt install docker-compose. That’s why we use pip.

sudo apt install -y python3-pip libffi-dev
sudo pip3 install docker-compose

This may take a while. If I now enter docker-compose, an error appears because the program cannot be found.

So I create a link to the real location and give execute permissions for security.

sudo chmod +x /usr/local/bin/docker-compose
sudo ln -s /usr/local/bin/docker-compose /usr/bin/docker-compose

Next, we need a user group for Docker (which may already be created) and add our user account.

sudo groupadd docker
sudo usermod -aG docker pi
newgrp docker
# wait

If the user is not called “pi”, the command must be changed. When entering id, docker should also appear in the list of groups.

Now, you can install Lazydocker, a management program for Docker in the shell.

curl https://raw.githubusercontent.com/jesseduffield/lazydocker/master/scripts/install_update_linux.sh | bash
lazydocker

Zsh

I also want a different shell.

sudo apt install zsh
curl -fsSL https://raw.githubusercontent.com/zimfw/install/master/install.zsh | zsh
chsh -s /usr/bin/zsh
exec bash

Now it should also automatically complete my commands for Docker.

mkdir -p ~/.zsh/completion
curl \ -L https://raw.githubusercontent.com/docker/compose/1.29.1/contrib/completion/zsh/_docker-compose \ -o ~/.zsh/completion/_docker-compose

fpath=(~/.zsh/completion $fpath)

We add autoload -Uz compinit && compinit -i to the configuration.

nano ~/.zshrc

The server should now be restarted.

sudo reboot
### wait ###
ssh pi@192.168.0.47

Proxy Manager

The home folder should only be filled with hidden folders at this point. We can check this with ls -Al (ls lists the files and folders of the current directory)

I would like to create the Docker folder.

mkdir docker
ls -l

The folder now exists I could also delete it again.

rm -r docker
# If the folder already contains files, it can be deleted with rm -rf docker
ls -l

The folder is gone But I need it, so I create it again.

mkdir docker

The Docker group needs the rights to the folder.

sudo apt install acl
sudo setfacl -Rdm g:docker:rwx ~/docker
sudo chmod -R 775 ~/docker

In contrast to my instructions with Traefik, I would instead follow the approach here that each service gets its own folder. So I need one for a proxy manager. There are several proxy managers, such as Traefik, Caddy or Nginx. I want to use Nginx as a proxy. Still, since I don’t feel like manually writing the Nginx configuration file and taking care of my Lets Encrypt certificates, I use the Nginx Proxy Manager.

cd ~/docker
mkdir nginx-proxy-manager
cd nginx-proxy-manager

By the way, commands can be completed with Tab. If I am in the home folder cd ~ I can enter cd d and then press Tab. With the previously installed extension, Docker commands are also quickly selected. If I enter docker with a space at the end and press Tab twice, a list of options appears. You can click through the last commands entered using the up arrow key.

So back to the new Nginx folder

cd ~/docker/nginx-proxy-manager

We need three more folders for persistent data.

mkdir data/mysql -p
mkdir letsencrypt

Now we create a new Docker network for NPM.

docker network create npm_net

We still need an environment file with the passwords for the database.

touch .env
% echo "DB_PWD=$(openssl rand -hex 16)" >> .env
% echo "DB_ROOT_PWD=$(openssl rand -hex 16)" >> .env

Finally, only the Docker-Compose file is missing.

nano docker-compose.yml

version: "3.9"

### Networks ###
networks:
    npm_net:
        external:
            name: npm_net
    internal:
        external: false
    default:
        driver: bridge

### Services ###
services:
    nginx-proxy-manager:
        container_name: nginx-proxy-manager
        image: jc21/nginx-proxy-manager
        restart: always
        networks:
            - npm_net
            - internal
        ports:
            - "80:80"
            - "443:443"
            - "81:81"
        environment:
            DB_MYSQL_HOST: npm_db
            DB_MYSQL_PORT: 3306
            DB_MYSQL_USER: npm
            DB_MYSQL_PASSWORD: $DB_PWD
            DB_MYSQL_NAME: npm
            # DISABLE_IPV6: true
        volumes:
            - ./data:/data
            - ./letsencrypt:/etc/letsencrypt
        depends_on:
            - npm_db

    npm_db:
        container_name: npm_db
        image: yobasystems/alpine-mariadb:10.4.17-arm32v7
        # image: yobasystems/alpine-mariadb
        restart: always
        networks:
            - internal
        environment:
            MYSQL_ROOT_PASSWORD: $DB_ROOT_PWD
            MYSQL_DATABASE: npm
            MYSQL_USER: npm
            MYSQL_PASSWORD: $DB_PWD
        volumes:
            - ./data/mysql:/var/lib/mysql

The line should be commented out if IPv6 is unavailable in the network. Also, the image is only for the Raspberry Pi; if you use another system, you do not need the arm32v7 tag.

Time to test the configuration.

docker-compose up -d

Now we look at the log.

lazydocker

Everything seems to be okay so far. So we can call up the NPM GUI by entering the IP address of the Raspberry Pi followed by :81 in the browser.

This should take us to the login screen.

The user data is as follows:

Email: admin@example.com Password: changeme

You will be asked for a new password the first time you log in. The Proxy Manager is now ready for use.

NPM Dashboard](@assets/images/blog/setup-raspberry/images/npm-dashboard.png)

As there are currently no other devices in the network that should be accessible from outside, we do not need to deal with this any further for the time being.

Updates can also be installed very easily.

cd ~/docker/nginx-proxy-manager
docker-compose down
docker pull jc21/nginx-proxy-manager
docker-compose up -d

It is possible to be notified about new versions on Github. I prefer this to using Watchtower, which automates the updates.

A new release does not mean that you have to update or that the update is already on Docker.hub

Firewall

Now we need a firewall, preferably an uncomplicated one, i.e. the uncomplicated firewall.

sudo apt install ufw
sudo ufw allow 80/tcp #this is the http port
sudo ufw allow 443/tcp #this is the https port
sudo ufw allow 22/tcp #this is the ssh port
sudo ufw logging medium
sudo ufw default deny incoming
sudo ufw enable

Now we need to restart the server for the changes to take effect.

sudo reboot

We test the firewall by trying to call {IP address from server}:81 again and are surprised that it works. This is because Docker changes the iptables when we open a port.

The best solution I have found for this is the repository chaifeng/ufw-docker.

sudo nano /etc/ufw/after.rules

We add this code to the end of this file.

# BEGIN UFW AND DOCKER
*filter
:ufw-user-forward - [0:0]
:ufw-docker-logging-deny - [0:0]
:DOCKER-USER - [0:0]
-A DOCKER-USER -j ufw-user-forward

-A DOCKER-USER -j RETURN -s 10.0.0.0/8
-A DOCKER-USER -j RETURN -s 172.16.0.0/12
-A DOCKER-USER -j RETURN -s 192.168.0.0/16

-A DOCKER-USER -p udp -m udp --sport 53 --dport 1024:65535 -j RETURN

-A DOCKER-USER -j ufw-docker-logging-deny -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -d 192.168.0.0/16
-A DOCKER-USER -j ufw-docker-logging-deny -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -d 10.0.0.0/8
-A DOCKER-USER -j ufw-docker-logging-deny -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -d 172.16.0.0/12
-A DOCKER-USER -j ufw-docker-logging-deny -p udp -m udp --dport 0:32767 -d 192.168.0.0/16
-A DOCKER-USER -j ufw-docker-logging-deny -p udp -m udp --dport 0:32767 -d 10.0.0.0/8
-A DOCKER-USER -j ufw-docker-logging-deny -p udp -m udp --dport 0:32767 -d 172.16.0.0/12

-A DOCKER-USER -j RETURN

-A ufw-docker-logging-deny -m limit --limit 3/min --limit-burst 10 -j LOG --log-prefix "[UFW DOCKER BLOCK] "
-A ufw-docker-logging-deny -j DROP

COMMIT
# END UFW AND DOCKER

Then we restart the Raspberry.

sudo reboot

We try to call up the GUI again and see the GUI again. Did it not work? Yes, it did in part. The code block ensures that the services can still be accessed via 192.168.0.0/16. When sharing to the Internet, access is blocked.

But since 80 and 443 should really be released if necessary, we have to add the firewall.

sudo ufw route allow 80/tcp
sudo ufw route allow 443/tcp
sudo ufw reload

The firewall is now set up.

Adblocker

If possible, I would like to block advertising and malware in my network at the DNS level. The advantage is that the advertising is also blocked on the cell phone. Although there are alternatives such as Pi-Hole and AdGuard, I have opted for Blocky this time.

cd ~/docker
mkdir blocky
cd blocky
mkdir whitelists
mkdir blacklists
mkdir logs

Now we have three folders in the newly created directory. First, I create a new network and fill the Docker Compose file.

docker network create blocky_net
nano docker-compose.yml

version: "3.9"

### Networks ###
networks:
    blocky_net:
        external:
            name: blocky_net
    default:
        driver: bridge

### Services ###
services:
    blocky:
        container_name: blocky
        image: spx01/blocky
        restart: unless-stopped
        networks:
            - blocky_net
        ports:
            - "53:53/tcp"
            - "53:53/udp"
            - "4000:4000/tcp"
        environment:
            - TZ=Europe/Berlin
        volumes:
            - ./config.yml:/app/config.yml
            - logs:/logs
            - ./blacklists:/app/blacklists/
            - ./whitelists:/app/whitelists/

Now we need the config file. We take the example file instead of editing the complete file.

nano config.yml

upstream:
    externalResolvers:
        - 5.9.164.112 #digitalcourage
        - 80.241.218.68 #dismail
        - tcp-tls:fdns1.dismail.de:853
        - https://dns.digitale-gesellschaft.ch/dns-query
blocking:
    blackLists:
        suspicious:
            - https://raw.githubusercontent.com/PolishFiltersTeam/KADhosts/master/KADhosts.txt
            - https://raw.githubusercontent.com/FadeMind/hosts.extras/master/add.Spam/hosts
            - https://v.firebog.net/hosts/static/w3kbl.txt
        ads:
            - https://adaway.org/hosts.txt
            - https://v.firebog.net/hosts/AdguardDNS.txt
            - https://v.firebog.net/hosts/Admiral.txt
            - https://raw.githubusercontent.com/anudeepND/blacklist/master/adservers.txt
            - https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt
            - https://v.firebog.net/hosts/Easylist.txt
            - https://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=0&mimetype=plaintext
            - https://raw.githubusercontent.com/FadeMind/hosts.extras/master/UncheckyAds/hosts
            - https://raw.githubusercontent.com/bigdargon/hostsVN/master/hosts
        tracking:
            - https://v.firebog.net/hosts/Easyprivacy.txt
            - https://v.firebog.net/hosts/Prigent-Ads.txt
            - https://raw.githubusercontent.com/FadeMind/hosts.extras/master/add.2o7Net/hosts
            - https://raw.githubusercontent.com/crazy-max/WindowsSpyBlocker/master/data/hosts/spy.txt
            - https://hostfiles.frogeye.fr/firstparty-trackers-hosts.txt
            - https://raw.githubusercontent.com/Kees1958/W3C_annual_most_used_survey_blocklist/master/TOP_EU_US_Ads_Trackers_HOST
        malicious:
            - https://raw.githubusercontent.com/DandelionSprout/adfilt/master/Alternate%20versions%20Anti-Malware%20List/AntiMalwareHosts.txt
            - https://osint.digitalside.it/Threat-Intel/lists/latestdomains.txt
            - https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt
            - https://v.firebog.net/hosts/Prigent-Crypto.txt
            - https://bitbucket.org/ethanr/dns-blacklists/raw/8575c9f96e5b4a1308f2f12394abd86d0927a4a0/bad_lists/Mandiant_APT1_Report_Appendix_D.txt
            - https://phishing.army/download/phishing_army_blocklist_extended.txt
            - https://gitlab.com/quidsup/notrack-blocklists/raw/master/notrack-malware.txt
            - https://v.firebog.net/hosts/Shalla-mal.txt
            - https://raw.githubusercontent.com/Spam404/lists/master/main-blacklist.txt
            - https://raw.githubusercontent.com/FadeMind/hosts.extras/master/add.Risk/hosts
            - https://urlhaus.abuse.ch/downloads/hostfile/
        coin:
            - https://zerodot1.gitlab.io/CoinBlockerLists/hosts_browser
    clientGroupsBlock:
        default:
            - suspicious
            - ads
            - tracking
            - malicious
            - coin
port: 53
httpPort: 4000

docker-compose up -d

Now you should see something under port 4000.

The only thing left to do is change the router’s DNS server. With the Fritz!Box, this can be done under IPv4 settings. The IP address of the Raspberry Pi must be entered there as the local DNS server.

Furthermore, we enter the IPv4 address of the Raspberry Pi instead of the previous DNS servers under Access data and DNS server.

This blocks most of the advertising.

Of course, you can edit many more settings, which I won’t go into now. For the change to take effect, you must leave the network and reconnect.

What I would at least do would be to create three bookmarks in my web browser.

http://ipadressevompi:4000/api/blocking/status To see the status.

http://ipadressevompi:4000/api/blocking/enable To switch on Blocky.

http://ipadressevompi:4000/api/blocking/disable To turn Blocky off.

Home page

To test the proxy manager and run something else on Raspberry, I would like to have a dashboard that could serve as a start page in the browser. There are also several options here. I choose Homer.

cd ~/docker
mkdir homer
cd homer
mkdir assets
id

We remember the number of our user (User ID) and the group Docker (Group ID).

nano .env

UID=1000 GID=123

Your own values must be entered. Then we need a Docker-Compose file.

nano docker-compose.yml

version: "3.9"

### Networks ###
networks:
    npm_net:
        external:
            name: npm_net
    default:
        driver: bridge

### Services ###
services:
    homer:
        container_name: homer
        image: b4bz/homer
        restart: unless-stopped
        networks:
            - npm_net
        ports:
            - 8080:8080
        environment:
            - UID=$UID
            - GID=$GID
        volumes:
            - ./config.yml:/www/config.yml
            - ./assets/:/www/assets

The configuration file still needs to be included before deployment. Here, you can see what is available.

nano config.yaml

title: "Homepage"
subtitle: "Homer"
logo: "assets/logo.png"

links:
    - name: "Github"
      icon: "fab fa-github"
      url: "https://github.com/"
      target: "_blank"
    - name: "Deployn"
      icon: "fab fa-octopus-deploy"
      url: "https://deployn.de"
    - name: "Reddit"
      icon: "fab fa-reddit"
      url: "https://www.reddit.com/"

services:
    - name: "Example"
      icon: "fas fa-code-branch"
      items:
          - name: "Startpage"
          - subtitle: "Search engine"
          - url: "https://startpage.com"

Done.

docker-compose up -d

The page is now located under the IP address of the Raspberry Pi with the port number 8080.

Public access

The last thing that needs to be added is the possibility of accessing this homepage from outside the network. To do this, a domain must point to the public IP address of the home network, as shown in the instructions for the Fritz! In the NPM GUI, only the desired subdomain must be entered and forwarded to port 8080.

It is also possible to use a DNS challenge for a domain.

If you have saved the setting and try to access the page within your network, an error message appears because the DNS rebind protection prevents access from your own network via the external IP for security reasons. The error message usually tells you how to change this behavior.

But even if you leave your own network and try to access the subdomain, you will receive an error, this time 504 Gateway Time-out. Why? Because the firewall is working.

sudo ufw allow 8080/tcp

As soon as you send this command, the page should be accessible.

The setup is now complete 😀