Run Linux Instances in Red Cloud 2

Red Cloud 2 supports several Linux distributions via these official images running on a variety of instance configurations, or flavors.

This documentation assumes a basic understanding Linux system administration for maintaining your Linux instance. If you are new to Linux, read the Linux Tutorial first. Additionally, you may find the Introduction to Linux topic on the Cornell Virtual Workshop helpful.

Creating a New Linux Instance

  1. Log in to the Horizon Web Interface.
  2. If you have not already, create a key pair or import a public key
  3. If you have not already, create a security group. Note that your security group should include the inbound SSH rule so you can connect to it.
  4. Optional: Create a Private Network and Router.
  5. Create an instance: select one of the Rocky or Ubuntu Linux images under the Source tab. If you created or have a private network for the new instance, select the private network under the Networks tab.
  6. Double check the new instance's security group rules to make sure you can access the new instance.
  7. Optional: Create and attach a Volume
  8. Optional: Associate a Floating IP address if the new instance is on a private network.

Now that you have created and launched an instance, your next steps will be to connect to it and set up a new user account. See the CentOS steps or Ubuntu steps for more information on how to set up a new user, update, and install software for each distribution.

Accessing Instances

First, establish access to your instance using Secure Shell (ssh), possibly including X Windows for remote graphical display. If you are at all unfamiliar with Linux, we recommend following the Linux Tutorial. If you would like to have a desktop environment (not available by default for Linux instances), then you may want to follow the steps for XFCE Desktop. Establishing an SSH connection is a prerequisite for creating a Linux desktop environment such as XFCE.

Secure Shell (SSH)

The main requirements for logging in to your instance using secure shell are:

  • The security group for your instance must permit SSH connections (TCP port 22) from your current IP address.

  • You must use the private key that matches the public key in the key pair you specified when launching the instance.

  • You must log in to your instance using the correct initial username:

    • For Rocky Linux 8 and 9, the user name is rocky,
    • For Ubuntu, it is ubuntu.

To log in through SSH, specify the private key file from your SSH key pair (for example my_key.pem), along with username and IP address as follows:

 ssh -i my_key.pem <username>@<IP address of your instance>

You may need to supply the full path to your private key, e.g., /Users/local_username/my_key.pem. For more information on how to use SSH, see the Connect to Linux page.

Note: Transferring files can also be done over ssh. See the scp and sftp commands, or programs like WisnSCP and similar software for Mac OS X.

Troubleshooting

There are several common error messages you may see when trying to access your Linux instance using SSH. These are the most likely causes for each common message.

  • "Connection timed out" means that your SSH command can't communicate at all with the instance.

    • Note: It may take a while for the connection time out to occur, so it may seem that the system is not responding at all initially. However, this is still likely a "Connection timed out" error.
    • Has the instance been started? Check the instance's console in the Horizon Web Console. Did the instance boot successfully?
    • Do the security group's rules allow incoming ssh connections (TCP port 22) from an address range (CIDR) that includes your current IP address? Double check that you followed the instructions for security groups.
    • Remember, if you are not on campus but the security group is configured for ingress from 10.0.0.0/8 and 128.84.32.0/22, you would need to be connected to the Cornell VPN in order to SSH into the instance.
    • Is your instance on the public network? If the instance is on a private network, does it have an associated floating IP address and are you using the correct address?
    • Here is a tool for finding your current IP address. There are also port connection tools like ping and telnet you can use for troubleshooting. (To use ping, make sure your security group has the "ALL ICMP" rule enabled for an address range that includes your current IP.)

  • An error like "Permissions <4-digits> for are too open" means you can reach the instance, but your private key file has improper permissions.

    • Make sure your private key file is saved as a ".pem" extension and that it has the proper permissions:

      chmod 600 <key name>.pem

  • "Permission denied ()" means that the combination of the username and SSH key you are providing are not correct for this instance.

    • Make sure you are using the correct username:
      • If you are using Ubuntu, did you login as the ubuntu user? For more information on that, see the Linux Tutorial.
      • If you are using Rocky Linux 8 or 9, did you login as the rocky user?
      • If you are using a CentOS MATLAB instance, did you login as the root user?
    • Did you create a key pair and make sure to select it when creating the instance?
    • Are you supplying your key pair in the command? See the SSH instructions above for an example.
  • If you get an unexpected password prompt:
    • Did you use the correct user name? See suggestions above.
    • Make sure your private key matches the public key selected when you created your instance.
      1. On your computer, run: ssh-keygen -y -f <private key file>
      2. Confirm that the output matches the public key on Red Cloud at https://redcloud2.cac.cornell.edu/dashboard/project/key_pairs<your keypair name>.
  • "Warning: Identity file my_key.pem not accessible: No such file or directory" means that the ssh client can't find your private key.
    • Provide the full path to your private key, or check to see if there is a typo in the path you provided.

VNC with XFCE Desktop

In case you would prefer a desktop environment over a command-line, your Linux instance needs to have a VNC (Virtual Network Computing) server and a desktop environment installed on it. Red Cloud's "gpu-accelerated" images come with the TigerVNC server preinstalled, as well as the XFCE Desktop Environment, making it relatively easy to use a VNC client to connect to a Linux instance that is based on one of the "gpu-accelerated" images.

For Linux instances based on other images, certain packages must be installed on the instance first. This section details the steps to setting up an XFCE desktop environment for use with VNC on a CentOS 7.4 or Rocky instance. Other typical Linux desktop environments, such as GNOME, are also available, but XFCE is used as an example here. Setting up a desktop environment should work similarly on Ubuntu instances as well, with some differences. Once the environment is set up, you can launch a VNC server on the instance and connect to it using a VNC client through an ssh tunnel.

VNC and XFCE Installation on CentOS and Rocky Linux

  1. Log in via ssh as described above.
  2. Install needed packages (note, in Rocky Linux, dnf is preferred over yum):
    • sudo yum install tigervnc-server
    • sudo yum install epel-release
    • sudo yum groupinstall xfce
  3. Install some additional software that most users will want. These are only suggestions, and this is not a comprehensive list
    • sudo yum install gedit
    • sudo yum install firefox
    • LibreOffice

VNC and XFCE User Setup

For each user that will want to use the XFCE Desktop, you will need to set up VNC capability. To do this, follow the directions below. Alternatively, there is also an Easy guide to secure VNC server with TLS encryption.

  1. Open a shell as that user
  2. vncpasswd
    • Sets the user's VNC password
    • This step is not necessary for read-only VNC
    • This creates a ~/.vnc folder
  3. vim ~/.vnc/xstartup
    • This step is not needed for Rocky Linux, as the file is created automatically upon starting a vncserver
    • Do not change this file on "gpu-accelerated" instances (as commands in it prevent Anaconda from interfering with dbus)
    • On other instances, paste this text into the file:
      #!/bin/bash
      xrdb $HOME/.Xresources
      startxfce4 &
  4. chmod 755 ~/.vnc/xstartup
  5. mkdir ~/bin
  6. vim ~/bin/start_vncserver.sh
    • Paste this text into the file:
      #!/bin/bash
      vncserver -geometry 1680x1050
  7. chmod 755 ~/bin/start_vncserver.sh

Using VNC

A brief overview for users is provided here, and for more information please see the Using VNC section on our Getting Started page.

Manage the VNC Server

Whenever an instance gets rebooted, you can restart the VNC server by doing the following

  1. ssh into the instance
  2. run ~/bin/start_vncserver.sh

On Rocky Linux, starting the vncserver manually will generate a warning message, but it should still work. On multi-user systems, it is generally better to start vncservers through systemd as the message suggests.

To find the port, you can run e.g. ps gxu | grep vnc.

You can stop the VNC server by running vncserver -kill :1. If this doesn't work, you can try pkill Xvnc.

To change or reset the VNC password, you can simply run vncpassword

To change the screen resolution:

  • Permanently: edit the script ~/.vnc/xstartup
  • For this session only: use Applications->Display
View the Desktop
  1. Open a local shell and ssh to establish the tunnel (recommended local port 10000):
    • ssh -L 10000:localhost:5901 <username>@<remote_ip>
    • Note When this shell is closed, the VNC viewer will have to close, too, though VNC is still running.
  2. In a VNC viewer app, connect to the VNC server localhost:10000 (or whichever port you chose above).
    • This should open a window showing the desktop
    • The first time you do this:
      • For the remote computer, you may have to dismiss a warning dialog
      • You will need to initialize a "panel". Click "Use default config"

Note for Windows 10 users You can also find useful instructions on the PuTTY and Xming page. You may also want to look into OpenSSH or Bash on Ubuntu on Windows.

Instance Maintenance

All self-managed desktops, laptops, servers, and Red Cloud instances, both Windows and Linux, should be updated with Operating System and Acrobat Reader critical and security updates on a monthly basis.

For Linux instances:

  1. Check for updates
    • Ubuntu: sudo apt update
    • CentOS 7: sudo yum check-update
    • Rocky 8 or 9: sudo dnf check-update
  2. Install updates
    • Ubuntu: sudo apt upgrade
    • CentOS 7: sudo yum update
    • Rocky 8 or 9: sudo dnf upgrade
  3. Reboot the instance with sudo reboot

Before rebooting make sure to save all active work. Rebooting will disconnect you from the instance. Wait a minute or two to allow the instance to restart before reconnecting. When you reconnect, you should verify that the updates were applied by repeating step 1.

Initialize and Mount a Volume

WARNING: FILE SYSTEM INITIALIZATION OVERWRITES AND DESTROYS PREVIOUS DATA.

The instructions here are for formatting and mounting attached volumes, though steps like these can only be performed if you have allocated and attached the volume through OpenStack, which can be done while the instance is running.

Note These instructions assume you are the root user. If you are not (such as on Ubuntu), then you may need to prepend sudo where appropriate.

  1. Identify the device name of the drive you wish to format and mount.
    • Run lsblk to see which /dev/vdX is the likely candidate (for some character 'X'). The following directions assume you identified vdb as the drive of interest, and ext4 as the file system you wish to initialize on it. On Rocky Linux, you may wish to substitute xfs in place of ext4 in the instructions that follow.
  2. Set up file system:
    • sudo mkfs.ext4 /dev/vdb
  3. Make a directory where the device will be mounted, for example "mountpoint" in the "/mnt" directory:
    • sudo mkdir /mnt/mountpoint
  4. Protect the mount point directory from accidental writes (Optional, but prevents a common user error):
    • sudo chattr +i /mnt/mountpoint
  5. Mount the device:
    • sudo mount /dev/vdb /mnt/mountpoint
  6. To have the mount automatically renewed after reboot, add an fstab entry (this is a little dangerous)
    • sudo nano /etc/fstab
    • Add a line with tab separations between four fields: disk device, mounted location, "ext4", "defaults":
      • /dev/vdb /mnt/mountpoint ext4 defaults,nofail

Migrate an Instance to a New Project

Occasionally, you may have an instance in one Red Cloud project that you would like to migrate to a different project. If you have been working in an exploratory project and are transitioning to using a permanent project, you may want to bring along the instances you have created. Or, you may want to share an instance with someone who is working in another project. The steps to perform such migrations are not difficult and can be performed through the Red Cloud (Horizon) web interface.