Simple Way to Fix SSH Permission Denied (Public key) error

Most of the time when you encounter permission denied (public key) error, it is not because the public key doesn't exist but because the private key can't find the public key.

If you are connecting from your PC (local) to a server, your PC would be the one storing the private key, and your server would be the one storing the public key. , so, if you connect to a server that has your public key, it will know it’s you because you are the only one that has the private key that matches it. This is why you are able to connect using Putty.

If you want to connect from Server 1 to Server 2, this won't work and it would return Permission denied (publickey), the reason for this is because Server 1 doesn't have a private key, this is no longer a connection from a local computer, this is a connection from a server to server, so, to fix this, the private key must be kept on Server 1 and the public key must be stored on Server 2.

Generate a private key on Server 1

ssh-keygen -t rsa

The server will ask you for a place to store the keys, and a pass-phrase, hit enter in the first prompt, and input a good pass-phrase for the second, as a way to increase security.

The output should look something like this:

user1@server1:~/.ssh$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/user1/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/user1/.ssh/id_rsa.
Your public key has been saved in /home/user1/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:XaGZuhjuV/6FSRoa9GW6hm0nfZqp8ZEsxI/rMCcW58o user@server1.domain.com
The key's randomart image is:
+---[RSA 2048]----+
|            .    |
|           + .   |
|        . + +    |
|       . = =     |
|      . S O .    |
|     . o @.X +   |
|      o Oo& X o  |
|     . o.O.B B   |
|      ..E.+o*    |
+----[SHA256]-----+

• You will be asked for the directory to save your key files, hit Enter to default to /home/user1/.ssh, keep in mind that the user would be the user you are currently logged in as.
• The next part would tell you to enter the passphrase for the key, this is optional and you can skip by hitting the enter key. I’ll recommend you give it a passphrase (make sure this is different from your system password) as this is add-on security to the keys. Once you are done, the key would be saved in your selected directory.

Before we go further, let’s undertstand the process the ssh-keygen took when generating the key:

The ssh-keygen creates a directory named .ssh in your home directory if it doesn’t already exist, and inside that directory, it will create two files, id_rsa, and id_rsa.pub.

You might have guessed what both files do by their name, the id_rsa is your private key and should never leave your machine (the server 1), if it leaves your machine somehow, then the new owner can connect to your server.

The id_rsa.pub is your public key and can get copied to other servers, this way, you can log in via the key-pair.

Transferring The Generate SSH Public Key To Other Servers

ssh-copy-id -i ~/.ssh/id_rsa.pub xx.xx.xx.xx

The above command would transfer the public key to the target server, replace the xx.xx… with the actual IP of the target server, and you can as well use the hostname of the server. Learn how to simplify ssh connection using a simple config file.

Once you issue the command, you will be asked to login via password, and then the key would be copied over.

Keep in mind that if you set up a passphrase during the key generation, you will be asked to enter it in order to open your public key.

Also, authorisation via public key must be allowed for the ssh daemon, open nano

sudo nano /etc/ssh/sshd_config

Change

PubkeyAuthentication no

to

PubkeyAuthentication yes

Now, you can safely disable Passwordauthentication

Change

PasswordAuthentication yes

to

PasswordAuthentication no

TroubleShooting 1

In case you are still unable to connect from Server 1 to Server 2, and you instead get Permission denied (publickey).

Open up the config in the .ssh directory, and add the following information

Host name_to_identify_server (it can be anything)
     hostname ip_or_realdomain_of_server
     port Server_Port_Number
     user username_of_the_server(the user that has the public key)
     IdentitiesOnly yes (to ensure ssh uses the IdentityFile and no other keyfiles during authentication)
     IdentityFile ~/.ssh/id_rsa3 (the private key)
     PubKeyAuthentication yes

Now retry to connect, and if it fails again, then go to server 2, and copy the details of the public key from server 1 to authorized_keys in server 2. If there is already a key there, then just append a new one.

Retry and it should work now.