is there already a timeline for something akin to a Metacontext extension, or a “CLUT”, for the compute clouds & GPU clusters? (It appears that some redundant work among the HPC folks could be saved by taking action earlier rather than later…)
You might be interested in CLuMPI download | SourceForge.net, it presents MPI nodes as a single device.
It’s not a complete, conforming implementation by any means yet and clearly there is significant overhead involved which you must be aware of to benefit from using it.
Patches to add missing features or improve performance very welcome!