The ULTIMATE Guide to Reasonable Security for your Debian/Ubuntu Linux Server for new Linux Admins

[blunix]

PLEASE READ THE SECOND POST IN THIS THREAD BEFORE POSTING

Welcome new Linux Server admins! Take these steps to configure a reasonably secure server and to avoid falling victim to automated bot attacks!

[Security is a process not a task that can be completed, hence this thread will always be WIP]


1) Installing automatic security upgrades

Reason
This will automatically install security upgrades for all installed apt packages once a day.
Action

apt install unattended-upgrades
dpkg-reconfigure unattended-upgrades

Select "Yes" when it asks if you want to automatically install unattended upgrades.
Configuration
No further configuration required.


2) Configure the SSH server public key authentication
Reason
By default (sadly), openssh-server on Debian and Ubuntu allows for authentication by password. This is widely discouraged, as passwords can be guessed.
Action
First generate a secure SSH keypair (private and public key)

ssh-keygen -t ed25519 -a 100 -o[/SIZE]
Generating public/private ed25519 key pair.
Enter file in which to save the key (/home/user/.ssh/id_ed25519):    <== ENTER
Enter passphrase (empty for no passphrase):    <== Enter a secure password or press ENTER to skip
Enter same passphrase again:    <== same here
Your identification has been saved in /home/user/.ssh/id_ed25519
Your public key has been saved in /home/user/.ssh/id_ed25519.pub
The key fingerprint is:
SHA256:VdIgmgM/xJgvgeg1+Mz+VeAB9mFOKrWG+ira4mvHeXM user@host
The key's randomart image is:
+--[ED25519 256]--+
| ....Bo+. oo.    |
|...oB+O=.. o.    |
|. =o.*Boo .      |
| ..++ .+ o       |
| ..  .  S        |
|  ..   .         |
|  ..o .          |
|oo.+ + E         |
|*=+ . o          |
+----[SHA256]-----+
[SIZE=4]

If you do not trust that somebody can steal your laptop while it is running, or turned off if its disk is not encrypted, then you should enter a password for the ssh key. You will need to enter the password every time you use it - Debian and Ubuntu come with the ssh-agent pre-installed in most variants (not minimal), which will cache the password for a couple of minutes.
If you trust your laptop will not be stolen (while it is turned on if it has disk encryption) then you can skip this step.
Show the PUBLIC key and copy it:

[/SIZE]
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKSsgAYltS87EDN5rXy3rTezuBszeEfpUKcKArOYgk27 user@workstation
[SIZE=4]

Now SSH into your server and do the following:

ssh root@server
mkdir ~/.ssh
chmod 700 ~/.ssh
echo ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKSsgAYltS87EDN5rXy3rTezuBszeEfpUKcKArOYgk27 user@workstation >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

Now logout by typing exit or pressing CRTL + d and login to the server again. It should not ask for a password anymore. If it still asks for a password, debug this first and then continue with the following steps.

ssh root@server
echo [/SIZE]PasswordAuthentication no >> /etc/ssh/sshd_config.d/99-custom.conf
systemctl restart sshd.service
systemctl status sshd.service
● ssh.service - OpenBSD Secure Shell server
     Loaded: loaded (/lib/systemd/system/ssh.service; enabled; preset: enabled)
     Active: active (running) since Fri 2024-03-22 20:49:43 CET; 2s ago
       Docs: man:sshd(8)
             man:sshd_config(5)
    Process: 553075 ExecStartPre=/usr/sbin/sshd -t (code=exited, status=0/SUCCESS)
   Main PID: 553076 (sshd)
      Tasks: 7 (limit: 2244)
     Memory: 85.9M
        CPU: 75ms
     CGroup: /system.slice/ssh.service
             ├─553017 "sshd: [accepted]"
             ├─553019 "sshd: [net]"
             ├─553059 "sshd: root [priv]"
             ├─553072 "sshd: root [net]"
             ├─553076 "sshd: /usr/sbin/sshd -D [listener] 1 of 10-100 startups"
             ├─553077 "sshd: root [priv]"
             └─553078 "sshd: root [net]"
[SIZE=4]

Log out and back in again. If this works, you can additionally remove the password for this account. This has no effect in regard to SSH anymore but it doesn't hurt either:

ssh root@server
passwd --lock root

From man passwd:

-l, --lock
           Lock the password of the named account. This option disables a password by changing it to a value which matches no possible encrypted value (it adds a ´!´ at the beginning of the password).

           Note that this does not disable the account. The user may still be able to login using another authentication token (e.g. an SSH key). To disable the account, administrators should use usermod --expiredate 1 (this set the
           account's expire date to Jan 2, 1970).

           Users with a locked password are not allowed to change their password.

 

[blunix]

The first Post of this thread will be constantly updated with the relevant information. All posts below can be used to discuss possible configurations to enhance security.

The second Post of this thread will be constantly updated to describe what is relevant to this thread.



WHAT IS RELEVANT TO THIS THREAD

This thread describes to new users howto take REASONABLE steps to secure their Debian or Ubuntu Server after the initial installation. It is designed to help Users that are new to Linux to avoid simple configuration mistakes so that their servers are not taken over by automated attacks from script kiddys.

This thread is not designed to describe how to make the most paranoid setup possible. Recommendations should be kept simple and efficient, so that they give a real benefit to new Linux users.

Recommendations should focus on Debian/Ubuntu only, as it is rather rare that new Linux Users will go with RedHat / CentOS / Arch / Gentoo or similar.



How to Submit a Recommendation

Please use the following form to submit recommendations:

Action: Set SSH to a random high port, like for example 9876
Reason: Most bots scan for a open SSH port on port 22, hence ~90% of bots will think that this server does not have an open SSH port.
Configuration:

1) nano /etc/ssh/sshd_config.d/99-custom.conf
2) Write Port 9876
3) Save and close the file: CRTL + x -> y -> ENTER
4) Restart the sshd service to apply the changes: systemctl restart sshd.service
5) Check if the sshd service started correctly: systemctl status sshd.service
6) Logout of your current SSH session by typing exit
7) Login to the server with the new port number: ssh username@IP-or-domain -p 9876

 

[vs2-free-users]

Action: Check open ports with running programms
Reason: Sometimes service running that you dont want to have, and every service can be a security risk

Steps:
- Find the services using netstat

netstat -tulpen

- Stop and disable unwanted services

systemctl stop [SERVICENAME]
systemctl disable [SERVICENAME]

 

[blunix]

smooth. Can you add a cool nmap command on how to check this from the outside?

 

[vs2-free-users]

smooth. Can you add a cool nmap command on how to check this from the outside?

I prefer the local version because here you can see witch program hold the port open.

 

[blunix]

Of course you check with netstat / lsof / similar, but there should also be a version to check this from the outside @vs2-free-users - this is the way the script kiddys do it, so anyone reading this should be enabled to do this as well.

 

[dos2unix]

A lot of distro's have something called lynis. It's a pretty good baseline.

 

[blunix]

@dos2unix I know I'm familiar with it. This post isnt about you guys suggesting me what to write about, but for you to write about something and then i'll add it
You can use a LLM to save time

 

[bob466]

1. Keep your system up to date.
2. Enable the Firewall.
3. Change your DNS
4. Don't do anything stupid.

Works for me because simple is best.

 

[blunix]

@bob466 1. we already have - could you write a bit more on 2, 3 and 4 in the format described in the first post?

 

[vs2-free-users]

1. Keep your system up to date.
2. Enable the Firewall.
3. Change your DNS
4. Don't do anything stupid.

Works for me because simple is best.

bob446 for president . Can i be the second best .

 

[blunix]

the idea of this thread is to link users to it that open threads like "my server got hacked" or "how do i secure my server" - and then them not asking any further questions

"install a firewall" is not helpful to a new linux user. you have to explain how to do that.

 

[bob466]

"install a firewall" is not helpful to a new linux user. you have to explain how to do that.

I didn't say "install"...I said "Enable" the Firewall which everyone should do.

 

[Aristarchus]

I would set up server this way:
Read a lot about Linux servers configuration. Then:
1) install Linux off line
2) remove all unused software (remove X for dedicated server)
3) disable USB, bluetooth
4) close all network services
5) enable/configure firewall
6) configure ACL, harden sysctl.conf
7) install integrity check software e.g. AIDE, security benchmarks e.g. Lynis, malware scanner
8) go online and update, configure automatic updates, install/configure hardened kernel, update firmware (including router)
9) configure root (local only), user, passwords (e.g. no empty passwords, no reused passwords, expiring passwords, define password complication)
10) configure backups
11) install/configure vpn/sshd
12) install configure server

Learn, learn, learn

 

[vs2-free-users]

Shutdown servers you don't need as an extension to @Aristarchus post.

 

[blunix]

guys. please read the second post in this thread. it describes what this thread is for and how to post to it.

@Aristarchus 1-6 are off topic. Its not that I find what you are discussing not interesting (quite the opposite, I'd be super happy to discuss setting up a server in a very secure way) it is just out of scope of this thread.

 

[CaffeineAddict]

to help Users that are new to Linux to avoid simple configuration mistakes so that their servers are not taken over by automated attacks from script kiddys.

Action: Set SSH to a random high port, like for example 9876
Reason: Most bots scan for a open SSH port on port 22, hence ~90% of bots will think that this server does not have an open SSH port.

Script kiddies (or more correctly impatient ones) might go straight for port 22, falsely conclude the port is filtered or closed, and go on to scan some other IP's, but a more dedicated haxor will scan all your ports especially if they're targeting you therefore they will find what ever port you set for SSH.

Can you add a cool nmap command on how to check this from the outside?

For SSH it's as simple as running a TCP connect scan:

sudo nmap -sT server.com

Action: Detect and stop TCP port scan with nftables
Reason: To prevent exposing your actual SSH listening port and keep it secret
Configuration:

#!/usr/sbin/nft -f

# IPv4 table
add table filter_4

# prerouting: sees all incoming packets, before any routing decision has been made
# Packets may be addressed to the local or remote systems
add chain filter_4 prerouting_4 {
    # -175 = After conntrack, before mangle
    type filter hook prerouting priority -175; policy accept;
}

# IPv4 port scanning detection chain
add chain filter_4 attacker_4 {
    comment "IPv4 port scanning detection chain"
}

# Prerouting filter
add rule filter_4 prerouting_4 jump attacker_4

# IPv4 attack count
add counter filter_4 attacker_count_4 {
    comment "Count of attacks over IPv4"
}

# A record of current attackers on cooldown
# NOTE: Do not set too high timeout because the attacker might as well spoof it's IP
add set filter_4 port_scanners_4 {
    type ipv4_addr
    flags dynamic
    timeout 10m
    size 256
    comment "IP of the attacker performing port scan"
}

# A record of source IP and local ports being probed
add set filter_4 probed_ports_4 {
    type ipv4_addr . inet_service
    flags dynamic
    timeout 1m
    size 65535
    comment "Potential attacker IP and local port probed"
}

# Detect port scanning and record attacker's IP
add rule filter_4 attacker_4 ct state new ip saddr . tcp dport != @probed_ports_4 meter tcpscan { ip saddr timeout 10s limit rate over 1/hour burst 3 packets } update @port_scanners_4 { ip saddr }

# Drop packets originating from attacker's IP
add rule filter_4 attacker_4 ct state new ip saddr == @port_scanners_4 log flags all prefix "DROP attacker_4 port scanner: " counter name attacker_count_4 drop

# Keep a record of a potential attacker IP and local TCP port being probed
add rule filter_4 attacker_4 ct state new update @probed_ports_4 { ip saddr . tcp dport }

Btw. I've already posted this here some time ago based on original answer that is sadly out of date for modern systems.

This is the most basic firewall defense that lacks additional functionality and won't prevent all kinds of information gathering attacks.

It won't prevent a truly dedicated hacker to break trough this firewall setup and find your SSH port, what they can do next is probe your HTTP\S port for connection, if they fail they'll know they're temporarily banned by firewall and will keep trying in time-decremental fashion until a well known HTTP\S port becomes available again.

This way they'll learn ban timeout and can then auto-launch the scan at specified interval until a port with listening SSH service is found.

But the whole point here is that, even though we're unable to 100% defend we're at least able to get rid of script kiddies because they don't go that far into attack, therefore the attack surface has been reduced.