In a typical scenario of using ssh keys for authentication, you would generate a key pair on your system, deploy public key on all servers you need to access and use your single private authentication key to access all servers. The process is described here. There are however cases where a single private key is insufficient. For example when working with cloud servers, deploying your public key to servers which may be frequently launched and terminated is a nuisance. An alternative is to download server specific private keys to your system and use them to authenticate on those servers. On Amazon for example you can generate a number of SSH key pairs and have separate key pairs associated with different servers or groups of servers. Public key is deployed on a server at launch and you can download a private key to your system when it is created.

Let’s assume that you already have a private key stored in the ~/.ssh/id_dsa file. Let’s also assume that you downloaded two private keys from Amazon one of which is associated with application servers and another with database servers.

To use multiple private keys in such scenario simply follow the instructions below:

  1. Move application servers private key to ~/.ssh/ file.
  2. Move database servers private key to ~/.ssh/id_rsa.db file.
  3. chmod 600 ~/.ssh/id_rsa.*
  4. Create ~/.ssh/config file with the content outlined below.
  5. chmod 600 ~/.ssh/config

config file content:

IdentityFile ~/.ssh/id_dsa
IdentityFile ~/.ssh/
IdentityFile ~/.ssh/id_rsa.db

This assumes that your regular private key is stored in the id_dsa file. Replace id_dsa above with the name of your previously generated private key file if different from id_dsa.

Once the config file is set up, the ssh authentication process will attempt to use all three keys.