vnc4server runs Xvnc, which is a virtual (headless, software) X server. To use the graphics card, you need to start a hardware X server then share access using tools such as gnome-remote-desktop, vino or x11vnc.
For assistance, you’re more likely to get usable advice from forums dedicated to EC2, Ubuntu, Gnome or X.org.
The Xorg server (or XWayland, if using Wayland). It will need the appropriate DRM (kernel) and DRI (user-space) drivers for the graphics hardware. AFAIK, Nvidia hardware needs an Nvidia-specific version of the OpenGL libraries.
DirectFB provides access to the video hardware without using X11. It’s mostly used on embedded systems, e.g. Linux-based smart TVs.
Thanks. I may have found the perfect fit for my purpose: “VirtualGL” (virtualgl dot org): “VirtualGL works fine with headless nVidia GPUs (Tesla), but there are a few additional steps that need to be performed…”
Ran the scripts linked to by Dark_Photon (thanks). I got the VirtualGL / TurboVNC working. OpenGL shows the glxgears fine (with vglrun), the “core” OpenGL version is reported as 4.6. However the VirtualGL “vglrun” command won’t start the software I’m interested in (Unity game IDE), so I’ll probably drop the whole remote desktop idea.
BTW, if someone wants to run the scripts above, note that there are some hard-coded device ID’s for Nvidia Tesla 4, which don’t match what the ID I see on the instance. You may want to simplify the expression to just “Tesla 4”. This refers to “configure.sh”, “start_vnc_server.sh” and the “fix_xorg_conf.py” scripts. Also, in one script, it checks for “VGA controller: Nvidia” whereas Nvidia is listed separately as a “3D accelerator”