Set Up SSH Key Authentication on Linux (Step-by-Step Guide)

SSH key authentication is a convenient way of logging into a Linux system. In this guide, we'll walk through the process of creating a user, generating SSH keys, and configuring a Linux server for SSH access.

Let's get started.

Create a New User

If you haven't already created the user that will access the server, create it with:

sudo adduser newuser

Replace newuser with your preferred username.

Switch to the New User

Log in as the newly created user:

sudo su - newuser

This ensures all SSH files are created in the correct home directory.

Create the SSH Directory

Create the .ssh directory inside the user's home folder:

mkdir -p ~/.ssh

Set the Correct Permissions

SSH is very particular about file permissions. Incorrect permissions can prevent key authentication from working.

The permissions should be:  

• .ssh directory: 700
• authorized_keys file: 600

Use the following chmod command to configure the above:

chmod 700 ~/.ssh
touch ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

Generate an SSH Key Pair

On your local machine (not on the Linux server), generate an SSH key pair.

ssh-keygen -t rsa -b 4096 -f ~/.ssh/newuser_key

This creates:

• Private key: newuser_key
• Public key: newuser_key.pub

Important: Never share your private key.

Copy the Public Key to the Server

Transfer the public key to the server:

scp ~/.ssh/newuser_key.pub newuser@your_server_ip:~/.ssh/

Then append it to the authorized keys file:

ssh newuser@your_server_ip "cat ~/.ssh/newuser_key.pub >> ~/.ssh/authorized_keys && rm ~/.ssh/newuser_key.pub"

This authorizes the key for future logins.

After copying the key, ensure permissions remain correct:

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

You can also verify ownership:

chown -R newuser:newuser ~/.ssh

Now you're ready to connect:

ssh -i ~/.ssh/newuser_key newuser@your_server_ip

Replace your_server_ip with your server's IP address or hostname.

If everything is configured correctly, you'll be logged in.