This page describes methods that lets you connect remotely to the graphical interface of a machine. The main assumption here is, that we want to get the actual output of the graphics card, so basically everything that you would see on the screen of the PC. This has the benefit that you can also use the remote desktop for 3D applications like Unreal which are based on vulkan, but is a bit more intense on your network connection.

Package https://github.com/LibVNC/x11vnc How to use x11vnc

On remote machine (server)

Gnome Display Manager makes problems for starting a x11 session because of authorizations. Install and use lightdm instead.

sudo apt-get install lightdm

Use dpkg to set the display manager to lightdm.

sudo dpkg-reconfigure lightdm

With lightdm set as display manager we can install x11vnc on the server:

sudo apt-get install x11vnc

On local machine (client)

Install a vncviewer. x11vnc recommends tightvnc.

sudo apt install xfce4 xfce4-goodies xtightvncviewer

Preparing the server

Before the remote client can connect to the server, the server needs to be prepared. On the server a monitor has to be connected. This enables the server to open an actual display, to which the remote client can attach itself to.

If anything else than lightdm is used: Once after every reboot, a user must log in to the server on site, because the virtual display is only launched after the first user's login.

Start server and connect to it (from client)

Launching the x11vnc server and connecting to it via tightvnc is done from the client's site.

The following command launches x11vnc on the server through ssh. Replace the <server_user> with the name of a user with root-rights, and the <server_ip> the the server's IP address.

ssh -t -L 5900:localhost:5900 <server_user>@<server_ip> 'sudo x11vnc -localhost -display :0 -auth guess'

You'll be asked for the user's password first, then for a password of your choice for the vnc session. It's best to store this command in a bash alias, since it is required every time you want to establish a vnc connection. The first part connects to the server via ssh, the latter launches the x11vnc server on display :0, which is the same display as can be seen on the server's physical monitor.

Connecting visually to the x11vnc server from your client machine is done with the following command. As with the previous command, it is advised to create a bash alias for it.

vncviewer -encodings "copyrect tight zrle hextile" localhost:0

Important notes on how to use

If work is done on the remote server, leave the server unlocked and simply close the tightvnc window, this will terminate the client's connection as well as the x11vnc server. It is advised to install xscreensaver on the server, which allows the server to lock and become password-protected after some idle time. Do not lock the screen or log out, because it will close display :0, which prevents any control via vnc.

If any other user is currently connected to the server, it is indicated by your x11vnc-server's console output after entering the ssh password and before setting the vnc password:

bind: Address already in use
channel_setup_fwd_listener_tcpip: cannot listen to port: 5900
Could not request local forwarding.

Furthermore, if this is ignored and the x11vnc server is launched anyway, the port will be set to the next free (5901) and display :1, but display :1 doesn't exist. If a second connection it attempted to the same server, it is refused with the following information:

Connected to RFB server, using protocol version 3.8
vncviewer: VNC server closed connection

If a visual connection is temporarily not required, meaning if CLI suffices, use ssh to connect to the server and run byobu for a comfortable, multi-terminal environment. This is especially advised when monitoring rosnodes running on the server. The byobu instance is shared among all clients connected to the server. By detaching from the byobu session (F6) other users can re-use the shell command, executed by previous users.

This method has just recently been discovered and tested. If you encounter any issues, feel free to edit this section if some steps require workarounds or similar.

Test platform:

1. Ubuntu 20.04
2. Default Window Manager

Update your apt list and install the required packages. It is important that you use the tigervnc-scraping-server, as this is the one that really gets the output of your graphics card instead of a virtual desktop.

# on your remote machine
sudo apt-get update
sudo apt-get install tigervnc-scraping-server openssh-server

Create a VNC password that will be requested when you connect to the machine.

# on your remote machine
mkdir -p ~/.vnc
vncpasswd

The next steps will be done on your client machine. VNC itself comes per default without any encryption. This would mean that your keypresses and the screen could be sniffed. This is why we'll create a SSH tunnel to the remote machine and pipe the whole VNC traffic through it. Please go to your client machine and execute the following command to create the SSH tunnel and start the VNC server on the remote machine.

# on your client machine
ssh USERNAME@REMOTE.SERVER.IP -t -t -L 5900:localhost:5900  "x0vncserver -localhost -passwordfile ~/.vnc/passwd -display :1"

Please let this command and the terminal running it open while you want to work with VNC.

The next step is to get the TigerVNC viewer on your client machine. This can be installed with:

# on your client machine
sudo apt-get update
sudo apt-get install tigervnc-viewer
vncviewer

In the tigervnc-viewer, first open the options and set the compression to 'auto'. Afterwards, you can enter 'localhost' in the connection dialog and hit Connect. Please note: Please do not enter the IP address of your remote machine here, but really 'localhost'. This works, because we have a SSH tunnel running (see above).

Important: Once after every reboot, a user must log in to the server on site, because the virtual display is only launched after the first user's login. This is similar to the 18.04 method mentioned above.

Tested with GDM3 (20.04 default display manager).

After reboot the desired display :1 is not loaded until you logged in on display :0. We need to connect to display :0 once after rebooting. With x11vnc you can easily open :0. A newly set-up system needs one UI login to establish general display settings.

# on the remote machine
sudo apt-get update
sudo apt-get install x11vnc

Get onto the remote machine and look for the Xauthority file in /run/user/

# on the remote machine
sudo find /run/user/ -name Xauthority

This will yield at least two paths, one with a 4-digit named directory, and another with 3 digits. We need the path with 3-digits. Now from the client you can launch x11vnc similar to the x0vncserver. Specify the path to Xauthority and replace '125' with yours.

# on your client machine
ssh USERNAME@REMOTE.SERVER.IP -t -t -L 5900:localhost:5900 \
  "sudo x11vnc -display :0 -auth /run/user/125/gdm/Xauthority"

Connect to x11vnc with vncviewer, it will show the login display :0. After login, close the connection and use x0vncserver to launch display :1.