Select single vertex with the mouse


I am working on a programm where I can manually shape a terrain after I loaded it from a heightmap. For this reason I want to be able to select a vertex on the screen with the mouse in order to change its height.

What I did untill now is following:

First I get the screen coordinates with GetCursorPos(), then I translate them into coordinates for my application window with ScreenToClient(). After that I call twice gluUnproject() with first winZ = 0 and then winZ =1. This way I get two points which define a line. Now I test all vertices of the mesh in worldcoordinates if they lie on the line and output the one with the lowest z-value.

Something doesn’t work here. So if you have done this before or see an error please tell me!! Other suggestions are welcome as well!



PS: In order to check if the vertices lie on the line I implemented a little test DistancePointLine(). If the distance gets very small I argue that the point is on the line. Is there a better way to do this?

first check if your ray is correct… if you draw it and you see nothing, then its possible its correct

second: don’t try to find a point, but bether intersect with the landscape… find the nearest trianglehit, and then there you get the barycentric coordinates (normally done while intersecting so for free) and take the vertex with the biggest barycentric value…

for intersecting ray->heightmap, you’ll find tons of posts about this… preferably on flipcode for example… take a look around… (-limp bizkit)

you can draw each vertex with a different color, do a glReadPixels( mouseX, mouseY… ) and see what color you read.