16 July 2019

Math - Calculate the 3D world position of a 2D window vector

I render a 3D triangle. When I click on it, I want to know the point on the triangle that was clicked. To calculate this point, you need:
  • Point position: the window position of the mouse click. You get this point directly from the event raised by the operating system.
  • Matrix projection and camera (also called modelview): the matrices used to transform from object space to world space and then to 2D clip coordinates.
  • Rectangle viewport: the position and size of the viewport on which to project.
  • An InvertMatrix method. You can find an implementation here.
Then you calculate the line that contains all the points that project to the clicked position as follows:
public (Vector3 near, Vector3 far) GetWorldPosition(Point position)
{
    position.Y = viewport.Height - position.Y; //For OpenGL, the bottom is the origin.
    position -= viewport.Position; //Adjust to the viewport's position on the window.
    var p = position.Normalized(Size); //Normalize the point to the range [-1;1].
    var inv = (Projection * Camera).Invert(); //Calculate the inverse matrix.
    var near = new Vector4(p.X, p.Y, -1, 1) * inv;
    var far = new Vector4(p.X, p.Y, 1, 1) * inv;
    near.PerspectiveDivide(); //Divide X, Y and Z by the W component.
    far.PerspectiveDivide();
    return (near, far);
}
If you draw this line, it should appear as a dot until you move the camera, because every vector on this line projects on the same screen pixel.
Intersecting this line with the triangle then gives me the 3D point I'm looking for.

See also

Calculate the 2D window position of a 3D vector