This (longer than expected) post explains how to transfer files securely between your device and an external storage. The first part may be useful for you if you only have little knowledge of terms like (S)FTP(S) and want to learn something about widely used technologies. The second part will help you to mount an external storage so you can manage all files as if they are on your local device and the third, fourth and fifth part will concentrate on easing the mounting process by the help of hostnames, Private/Public Keys and a shell script.
This guide will be very detailed and is also (and especially) suited for beginners. Maybe also some advanced users can learn something or give hints for improvements.
Update: With improving Bash skills and more time, I was able to heavily improve the script in the end. Have a look at my Git instance to download the latest version.
But let’s be honest: All in all, this post will show you again, why Free Software, GNU/Linux and Open Standards are great, easy to use and why Windows users are to be pitied.
(Nearly) everybody knows FTP. FTP is a protocol which enables you to transfer files between your device and a remote space. Maybe you want to present your documents or images to visitors of your homepage and simply want to upload these files on your webspace. In most cases this could be done by the use of a seperate program like FileZilla.
So far so good, but there’re several problems. Two of them:
Now we know why FTP is insecure. So what alternatives do we have?
There is FTPS – using the FTP protocoll in connection with SSL. Well, that sounds great because, you know, SSL is great to ensure the safety of your online banking and frightens away sniffing script kiddies. Forget that. FTPS is only securing the authentication process, the transfer of all files remains in plain text. So if you upload a sensitive document, it is fairly easy to intercept it on different levels in the uploading/downloading process. There is also the fact that FTPS is horrible to implement, not very compatible, and not very smart designed.
The best alternative in my opinion is SFTP (you are confused by all the abbreviations now? Wait for it, it gets better). SFTP uses a SSH connection to build a safe tunnel between your device and the remote storage. SSH is very smart, widely used and on most servers running a Free operating system it is preinstalled and needs no further configuration. SFTP enforces you to transfer your files fully encrypted through a secure channel.
But how to use it? You can use clients like FileZilla but do you remember the first problem I mentioned? Correct, an external editor is not what suits the needs of someone who wants to work with his files comfortably.
This sounds more complicated than it really is. The theory behind it is very easy: You mount (link) a remote directory over SSH (a secure channel) in any desired directory on your machine. For instance, you can simply edit all files in /home/user/remote/ with your programs, but finally all changes will happen (nearly) instantly on your remote storage. This even works with watching a video file that is located on the server: Open it like a normal video file in a player like VLC and it behaves like it is locally on your PC (if your internet connection is fast enough, just test it!).
The only prerequisite: You need a server/webspace/storage with full SSH access. Unfortunately many webhosters don’t provide SSH access. If you belong to the unlucky ones, I recommend you to look for alternatives – it is worth it! (below this post, I added a very small list of webhosting providers offering SSH access)
Now we come to the technical part. For this post, following data is used. Most likely, this will look different in your case.
SSH-Server: server1.net
Username on server: client
Home directory of user on server: /home/client
Username local machine: user
Local mount directory: /home/user/remote/server1
On your GNU/Linux, please assure that following programs are installed. Some of them (ssh-askpass, zenity) are only used in step IV and V:
openssh-client, sshfs, ssh-askpass, zenity
Now create a directory in which the remote storage should be mounted in. Don’t worry about the space on your hard disk: This directory is remote and there are only temporary files stored on your local device. In this example I’ll use /home/user/remote/server1. It could also be /fsfe/lol/wtf/nsa/ if this directory would exist.
Now, start a terminal/shell to fill in following commands as a normal user.
ssh -p 22 client@server1.net
If asked for a password, fill in the password of ‚client‚ (the webhoster should have given it to you). If you are now connected to the server1.net, you are just one step away from mounting it. Close the connection or terminal, open it again and type:
sshfs -p 22 client@server1.net: /home/user/remote/server1/ -o follow_symlinks
This command connects to ‚server1.net‚ as the user ‚client‚ using the SSH protocol over the standard port 22. Then it mounts the home directory of ‚client‚ on the local directory ‚/home/user/remote/server1‚ which we created beforehand. Additionally we added the option ‚follow_symlinks‚ so that links of the server work on our local machine as well. If you have a look at /home/user/remote/server1, you should see the exact same content than in the home directory on your server or if you connect via FTP/SFTP. Congratulation! Now play around with it, try to edit, upload and download files.
To unmount (speak: disconnect) the directory, type
fusermount -u /home/user/remote/server1
Now you mount the home directory of ‚client‘. If you want to mount another directory on the server (e.g. /home/client/exchange), use this modified command:
sshfs -p 22 client@server1.net:/home/client/exchange/ /home/user/remote/server1/ -o follow_symlinks
In the next step, we will make the connection and mounting more comfortable, even if you are handling with more than one server.
Now that you know how (and that) the system works, we will make it easier. Of course it is quite annoying to type in the whole server-address and port each time. Instead of sshfs -p 22 client@server1.net[…], you can simply type sshfs server1. How? Just open the SSH configuration file /home/user/.ssh/config with you desired text editor and add:
Host server1
HostName server1.net
Port 22
User client
Save it and try to use this shortcut. It also works for normal SSH connection like ‚ssh server1‚. For the Host variable, you could use any name you can remember easily, for instance privateserver.
Pretty smooth, isn’t it? Well, it gets even better! You have to admit that typing a password each time you connect is quite harrassing. This gets even worse if you are used to connect to many servers. On the one hand, you could use password managing tools… or Public/Private key authentication! For those who don’t know much about it, let me say: This sounds terribly complicated, but you don’t have to understand it completely.
For this guide, you only have to know: There are two keys (files). The one is private, you should never give it to anyone! It’s like a key to your safe, if someone achieves it, he can get everything that is secured in this safe. But the other file is public, you can give it to everyone if you want so. Now it gets a bit weird, so I try to make it as simple and abstract as possible.
Let’s say, the public key is a chest that no one except the owner can open. Inside this chest, there is a supersecret word that no one except the owner knows. You put an exact copy of this chest on your server and each time you try to connect, you say „Hey, I’m the legitimate user of this server. Give me my chest, I will open it with my secret key and tell you, what the supersecret word is!“. The server will give you the chest (public key), you open it with your private key and tell the server the supersecret word. If you were right, the server lets you in.
Note: This example is so ridiculously abstract that I had to laugh while writing it down. Every security expert, IT guy and cryptography scientist would knock me down for this but I hope it made you understand the principle of this very smart method. And if you ask yourself now „Why does the server know the supersecret word? Isn’t this insecure?“ or „Isn’t is insecure to send such a sensitive code word over the internet?“: You’re absolutely right! But be assured, that the public-key cryptography uses some awesome tricks to avoid security leaks like these and others. If you want to, read about it!
It is quite easy to make this system happen. Again we need a terminal to generate the two keys:
cd ~/.ssh/
ssh-keygen -t dsa
This neat script asks a.) in which file you want to save the new keys. The standard is set to /home/user/.ssh/id_dsa and I recommend this. Most programs automatically search for keys in this destination. In step b.) it asks you for a password. This is quite important because if someone achieves your private key, he still needs a password to use it. After this short procedure, you have two new files in /home/user/.ssh – id_dsa and id_dsa.pub. As you can imagine, id_dsa.pub is your public key, the other is the secret one.
Now we have to put the public key (chest) on the server we want to connect to. This could be done by a simple command:
ssh-copy-id -i /home/user/.ssh/id_dsa.pub client@server1.net
By this, you automatically connect as a legitimate user to your server (with the password used in step II) and put the public key in the /home/client/.ssh/authorized_keys. This file simply collects all public keys (chests) of users that are allowed to connect to the server. If a stranger tries to connect to your server but there is no chest his secret key belongs to, he of course cannot open the connection to your server.
Now disconnect from your server and try to connect via ssh server1. Your device should automatically search for your private key and open the connection with it. As you notice, you are still asked for a password. This is because you (hopefully) put a password on your private key (remember?). But after you typed in the password once, it should be saved for duration of your local session in a local keyring.
If this is not the case or you want to make your configuration even better, add the following lines on the bottom of your /home/user/.bashrc (or if you’re not using bash, the respective rc file of your shell):
alias ssh='ssh-add -l > /dev/null || ssh-add -t 7200; ssh'
By this, you say to your shell: „Everytime I type the command ’ssh‘, you should instead (alias) 1.) check if I already added my key to the local keyring. If not so (||), 2.) add my key to the keyring, but only keep it there for 7200 seconds (you can also keep this away or increase it). Only if you made this sure, use the command ssh to connect to my server“. Every command line program that uses ssh to connect to a server (and there are some, e.g. git, sshfs…) now uses this procedure – no matter if you close the terminal or lock the screen.
Of course, this also works with sshfs to mount your remote storages. If you remember our first line to mount the server, we have a much shorter line now without any password request:
sshfs server1: /home/user/remote/server1/ -o follow_symlinks
And these steps were quite important for the next section where we will write a shell script to make it much more easier – and even graphical!
Our setting is very smooth now, but it could still be improved. If you want to connect to many servers and don’t want to use your shell every time or don’t want to remember the HOSTs you used in your .ssh/config, you’re free to modify and use this shell script:
#!/bin/bash
### VARIABLES TO BE CHANGED ###
# Preconfigured HOSTs in ~/.ssh/config that should be used
PRESSH[0]=server1
PRESSH[1]=server2
PRESSH[2]=server3
# Local directory where the remote storages should be mounted to
LOCALMOUNTDIR=/home/user/remote
### THE SCRIPT BEGINS HERE ###
# Add SSH key to local keyring if not already happened
function sshadd {
ssh-add -l > /dev/null || ssh-add
}
# Choose preconfigured HOST to mount
function mount {
if ! SSH=$(zenity --list \
--height=300 \
--text="Please choose. Cancel to unmount drives." \
--title="Choose SSH server" \
--column "Preconfigured SSH servers" \
${PRESSH[*]}); then
unmountquestion # If you press cancel, it should ask you to unmount all drives
fi
# If you double click on an entry, it gives 'server1|server1' as result instead of 'server1'
# This command cuts of everything after |
SSH=$(echo $SSH | awk -F\| '{ print $1 }')
# Make a local directory if not available
if [ ! -e "$LOCALMOUNTDIR"/"$SSH" ]; then
mkdir -p "$LOCALMOUNTDIR"/"$SSH"
fi
# Command to mount actually
sshfs "$SSH": "$LOCALMOUNTDIR"/"$SSH"/ -o follow_symlinks &
quitquestion # one more ssh server or quit?
}
# Ask if all preconfigured SSHFS drives should be unmounted
function unmountquestion {
zenity --question --text="Unmount all preconfigured\nSSHFS drives now?"
if [ "$?" = "0" ]; then
unmount # unmount function
else
exit 0
fi
}
# Procedure to unmount all preconfigured SSHFS drives and exit program afterwards
function unmount {
for ((i = 0; i < ${#PRESSH[*]}; i++))
do
fusermount -u "$LOCALMOUNTDIR"/"${PRESSH[$i]}"
echo ""${PRESSH[$i]}" unmounted."
done
exit 0
}
# Should another SSHFS storage be mounted?
function quitquestion {
zenity --question \
--text="Mount another SSHFS storage?"
if [ "$?" = "1" ]; then
exit 0
fi
}
sshadd # sshadd function
# Loop for endless mounts until stopped by unmount or unmountquestion
while :
do
mount # mount function
done
Save this script to some place (e.g. /home/user/sftp-mount.sh) and make it executable (chmod +x /home/user/sftp-mount.sh). For example, you are now ready put a shortcut in your panel or on your desktop to make it easily accessible. Please note that zenity, sshfs and ssh-askpass should be installed.
Please test the procedures written above on your servers and (if they have SSH access) webspaces. On my 4 webspaces and my vServer it works perfectly with my Debian Sid system. Please contact me or write in the comments if you have any problems or if some steps could be improved – nobody’s perfect :)
I know following hosters that provide webspace with SSH access. This list is very short so please add more by writing in the comments!
Comments