SSH and SSL

This isn't a reasonable comparison to make. SSL is a general method for protecting data transported over a network, whereas SSH is a network application for logging in and sharing data with a remote computer.

• AWS use .pem key (seems to related to SSL standard) to login (via SSH)
  • created by AWS and downloaded from AWS when created. Once created, .pem key can be used for multiple AWS services.
• Github use SSH key (e.g. id_rsa and id_rsa.pub) to login (also via SSH)
  • created using ssh-keygen. Github has good tutorial for it.
• All these files, including know_hosts, are within ~/.ssh folder

SSH tips

https://serversforhackers.com/ssh-tricks

SSH works by connecting a client program to an ssh server. In the commands, ssh is the client program. The ssh server is already running on the remote_host that we specified.

ssh remote_username@remote_host (use -vv for debug)
e.g: ssh -Y j68lin@linux.student.cs.uwaterloo.ca

$ ssh user@hostname  # login with password

$ ssh -p 2222 user@hostname

$ ssh -i /path/to/identity.pem username@hostname  # preferred, login with security key

Can use ssh to directly run command on remote machine:

$ ssh -p 2222 username@hostname pwd
/home/username

$ ssh -p 2222 username@hostname ls -la

format of .ssh/config file:

# within ~/.ssh/config:
Host namenode
        HostName ec2-52-71-80-173.compute-1.amazonaws.com
        User ubuntu
        Port 22
        IdentityFile ~/.ssh/jinchao-lin.pem

Host datanode1
        HostName ec2-52-23-30-73.compute-1.amazonaws.com # this can be public DNS or public IP
        User ubuntu
        Port 22
        IdentityFile ~/.ssh/jinchao-lin.pem

SSH Tunneling

---- Local Port Forwarding

ssh -L 3306:localhost:3306 username@hostname
or
ssh -L 3307:localhost:3306 username@hostname

-L - Setup local port forwarding
3306 - The local port to forward
localhost:3306 - Within the remote server, what address and port to forward traffic to. Since the MySQL server is on the remote server, we're tunneling to the remote server's "localhost" on port 3306, which MySQL is listening to.
username@localhost - The SSH username and host to connect to

--- Remote Port Forwarding

# Still on our local machine:
ssh -R 9000:localhost:8001 username@hostname

Let's go over this command:

-R - Using remote port forwarding
9000 - The remote server's port to use (not our local server this time!)
localhost:8001 - The local address to forward to. Since our webserver is on localhost port 8001, that's what we specify here. (Yep, the order of those arguments changed for -R over -L!)
username@hostname - SSH access to the remote server
If our remote server's IP address was 123.123.123.123, then our friends can access our website at 123.123.123.123:9000, which will forward to our local site at localhost:8001!

SCP transfer files

scp file from hadoop to local:
scp cluster:/home/hadoop/jinchao_analysis/res2.txt ./

e.g: scp j68lin@linux.student.cs.uwaterloo.ca:~/cs246/1145/a1/a1.pdf ~/Desktop

• If you need to resume an scp transfer from local to remote, try with rsync:
  rsync --partial --progress --rsh=ssh local_file user@host:remote_file

wget

# 1. Download a single file from the Internet
wget http://example.com/file.iso*

# 2. Download a file but save it locally under a different name
wget ‐‐output-document=filename.html example.com*

# 3. Download a file and save it in a specific folder
wget ‐‐directory-prefix=folder/subfolder example.com*

# 4. Resume an interrupted download previously started by wget itself
wget ‐‐continue example.com/big.file.iso*

# 5. Download a file but only if the version on server is newer than your local copy
wget ‐‐continue ‐‐timestamping wordpress.org/latest.zip*

# 6. Download multiple URLs with wget. Put the list of URLs in another text file on separate lines and pass it to wget.
wget ‐‐input list-of-file-urls.txt*

# 7. Download a list of sequentially numbered files from a server
wget http://example.com/images/{1..20}.jpg*

Network testing

telnet edhpen1131.kdc.capitalone.com 21000