SSH & SCP
SSH (Secure Shell)
SSH (Secure Shell) is a protocol that allows you to securely connect to remote computers over a network. It is commonly used to access servers, transfer files, and execute commands remotely.
Opening a Remote Connection
We connect to the machines remotely through Secure Socket Shell (SSH). This is done using ssh command in the terminal. This command can be used to connect to a remote machine in a variety of ways, however the two most commonly used forms are as follows: ssh [username]@[hostname] and ssh [hostname] -l [username]
Example
ssh compute0.ada.davidson.edu -l anscott@davidson.edu
Warning
To to access compute0.ada.davidson.edu or any other compute resources, you must be connected eduroam or using a VPN.
After executing this command you will be prompted for your password. Note that no characters will be displayed on the screen as you type in your password (not even dots), but this is expected behavior and the characters are being entered nevertheless. After successfully entering your password, you will be sent a Duo Push notification requesting your approval for login. Once the Duo request has been approved, you will be connected to the machine.
Troubleshooting
In the event that you cannot connect to a machine, here are some things to consider:
Trying to Connect to an Offline Machine
It could be the case that the machine is offline for some reason. An easy way to check this is by pinging it. This is achieved with the ping [hostname] command.
Example
The output of a successful ping will look something like this:
[anscott@compute0 ~] $ ping compute0.ada.davidson.edu
PING compute0.ada.davidson.edu (152.42.97.202) 56(84) bytes of data.
64 bytes from 152.42.97.202 (152.42.97.202): icmp_seq=1 ttl=63 time=31.2 ms
64 bytes from 152.42.97.202 (152.42.97.202): icmp_seq=2 ttl=63 time=29.3 ms
64 bytes from 152.42.97.202 (152.42.97.202): icmp_seq=3 ttl=63 time=30.1 ms
64 bytes from 152.42.97.202 (152.42.97.202): icmp_seq=4 ttl=63 time=37.8 ms
^C
--- compute0.ada.davidson.edu ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3004ms
rtt min/avg/max/mdev = 29.259/32.077/37.825/3.386 ms
The output of a failed ping will look something like this:
[anscott@compute0 ~] $ ping compute0.ada.davidson.edu
PING compute0.ada.davidson.edu (152.42.97.202) 56(84) bytes of data.
Request timeout for icmp_seq 0
Request timeout for icmp_seq 1
Request timeout for icmp_seq 2
Request timeout for icmp_seq 3
^C
--- compute0.ada.davidson.edu ping statistics ---
4 packets transmitted, 0 packets received, 100.0% packet loss
Note that the ping command will run infinitely unless manually terminated with control + c keybind.
Entering a Non-existent Username
ssh will not tell you if you are trying to login through a nonexistent username. You will still be prompted for a password, but you will never be allowed in (since the user doesn’t even exist). So if you find that your password is not being accepted, check to make sure you have entered the correct username.
SSH Public-Key Authentication for Mac/Linux
Constantly having to respond to a Duo push request can get frustrating quickly. Thankfully there is a way to connect to the cluster which does not subject you to constantly having to verify your identity.
The SSH command is actually only one part of a suite of tools (which includes scp) designed to enable secure access to remote machines. When used in conjunction with one another these tools can securely verify your identity with a trusted machine (so much so, you won’t need to be prompted for a password or sent Duo notification).
Setting up this trusted identity verification mechanism is done using the two commands ssh-keygen and ssh-copy-id. The first command, ssh-keygen, generates a pair of keys which combined will serve as your “identity.” The second command, ssh-copy-id, copies one of the keys to the remote machine and ensures that anyone who can prove they hold the other half is trusted and assumed to be you.
In most cases, running the following from a local terminal and using the suggested path to save the keys will generate a sufficiently secure “identity” you can use to identify yourself while on the computer you’re using: ssh-keygen -N "" -q -t rsa -b 2048
When you execute this command, the ssh-keygen utility prompts you to indicate where to store the key. Press the ENTER key to accept the default location. Once the keys are generated (and in the default location) you’ll need to inform the remote machine that the identity is trusted. This is done by executing ssh-copy-id using the same argument patterns that we saw with ssh: ssh-copy-id [username]@[hostname]
Example
Once a ssh identity has been created on your local host, you can copy this id to the compute0.ada.davidso.edu:
[anscott ~]$ ssh-copy-id anscott@davidson.edu@compute0.ada.davidson.edu
INFO : Source of key(s) to be installed : "id_rsa.pub "
INFO : attempting to log in with the new key(s), to filter out any that are already installed
INFO : 1 key(s) remain to be installed -- if you are prompted now it is to
install the new keys
anscott@davidson.edu@compute0.ada.davidson.edu’s password:
Number of key(s) added: 1
Now try logging into the machine, with: ssh compute0.ada.davidson.edu -l anscott@davidson.edu
At that point you should follow the directions and attempt to log into the machine as indicated. Once you connect to the machine again, you should not be prompted for a password or receive a Duo push notification and the terminal prompt should indicate you are connected to the remote machine.
SSH Public-Key Authentication for Windows
Windows 10 and later come with OpenSSH built-in, so you can use these tools directly from PowerShell or Command Prompt. If you are using an older version of Windows, you may need to install OpenSSH or use Git Bash, PuTTY, or WSL (Windows Subsystem for Linux).
Open PowerShell (or Command Prompt) and run: ssh-keygen -N "" -q -t rsa -b 2048. When prompted for a location to save the key, press ENTER to accept the default (C:\Users\YourUsername.ssh\id_rsa).
This will create two files: - id_rsa (private key) – Do not share this file. - id_rsa.pub (public key) – This will be copied to the remote machine.
Unlike Linux, Windows does not have ssh-copy-id by default. Instead, you can manually copy the public key using the following method:
Display the contents of your public key by running Get-Content $env:USERPROFILE\.ssh\id_rsa.pub. Copy the entire output. Log in to the remote server with the ssh [username]@[hostname] command using your password. Once logged in, append the copied public key to the ~/.ssh/authorized_keys file: echo "your-public-key" >> ~/.ssh/authorized_keys; chmod 600 ~/.ssh/authorized_keys.
SCP (Secure File Copy)
It is often necessary to either copy files from the cluster to your local machine or from your local machine to the cluster. There are several ways this can be accomplished. First of all, it can be tempting to use Git as a mechanism for file transfer. It is entirely feasible to commit local changes, push to GitHub, then pull the update to the cluster. Speaking from experience, this is how you end up with hundreds of frivolous commits in your repository.
While it works, this is not what Git was designed for — there are much better options. For those who are GUI-oriented, there are several server browsers freely available for download (Cyberduck is a good option). It is also possible to use JupyterLab’s built-in file browser to download individual files. However, the easiest and most robust method is to use Secure File Copy.
Using SCP
SCP is available through the console and allows you to copy files between machines. The command we use is scp, which is in many ways analogous to the cp command used for copying files on a single machine. The syntax for common use cases is below. Note that when running any of the following commands, you will be prompted to enter your password, just as with ssh.
Copy a file from the cluster to your local machine
scp [username]@[hostname]:[path to file] [path to destination]
Note that [path to file] points to a location on the cluster and [path to destination] points to a location on your local machine.
Example
scp anscott@compute0.ada.davidson.edu:/home/anscott/results.txt ~/Downloads/results.txt
Copy a file from your local machine to the cluster
scp [path to file] [username]@[hostname]:[path to destination]
Now, [path to file] points to a location on your local machine and [path to destination] points to a location on the cluster.
Example
scp ~/Documents/data.csv anscott@compute0.ada.davidson.edu:/home/anscott/data.csv
Copy a directory from the cluster to your local machine
scp -r [username]@[hostname]:[path to directory] [path to destination]
Example
scp -r anscott@compute0.ada.davidson.edu:/home/anscott/example-dir ~/Downloads/project
Copy a directory from your local machine to the cluster
scp -r [path to directory] [username]@[hostname]:[path to destination]
Example
scp -r ~/Documents/research anscott@compute0.ada.davidson.edu:/home/anscott/research