Manipulate a cube by dragging it's corners

I’m trying to create a simple modeller application that uses a tool to draw geometric shapes (box, sphere etc) as show in the following image.

The tool basically creates a placeholder cube from two points (start and end vectors) as shown. I’m trying to figure out a way to be able to drag any corner to resize the cube. I already have the mouse coordinates in world space and depending on the corner it should modify the start position, end position or both. Basically it works fine if I only drag the start or end position but can’t figure out a neat way to handle the rest of the corners.
Any help would be appreciated!