How to transform around a point using glm

[hydrodog]

We have a coordinate system for a web page in which top left corner is (0,0) with pixel coordinates, x increasing to the right and y increasing going down.

For laying down a character, I compute x0, y0, x1, y1, a rectangle and copy the glyph of the character.
This has worked for a long time.
For rotating all text the same way, we can apply a transform and change the projection, but if we just want to rotate one piece of text, we decided to do it by changing the x,y locations of the 4 corners of the character:

Given that we have x0, y0, x1, y1, what is wrong with the following code?

glm::mat4 t(1);
t = glm::translate(t, glm::vec3(-x,-y,0.0f));
t = glm::rotate(t, ang, glm::vec3(0,0,1));
t = glm::translate(t, glm::vec3(x,y,0)); 
glm::vec4 vec0(x0, y0, 0, 0);
vec0 = vec0*t;
x0 = vec0.x, y0 = vec0.y;
glm::vec4 vec1(x1, y1, 0, 0);
vec1 = vec1*t;
x1 = vec1.x, y1 = vec1.y;  

I thought this should bring (x,y) to the origin, rotate by angle, then translate back

[GClements]

@hydrodog

glm::vec4 vec0(x0, y0, 0, 0);
...
glm::vec4 vec1(x1, y1, 0, 0);

These should have 1 in the W component, otherwise the translation won’t be applied.

@hydrodog

vec0 = vec0*t;
...
vec1 = vec1*t;

These should have the matrix on the left and the vector on the right; otherwise the matrix is effectively transposed:
v^T=u^TM <=> v = (u^TM)^T = M^Tu.

When multiplying a matrix and a vector, GLSL (and GLM, which mimics it) treat the vector as a row vector if it appears on the left and a column vector if it appears on the right. The matrices generated by GLM (and also by the legacy OpenGL and GLU matrix functions) are intended to be multiplied with a column vector on the right.

This is the usual mathematical convention, but the opposite of DirectX which expects a row vector on the left. DirectX transposes the matrices, but uses row-major order (OpenGL and GLM use column-major order), so there’s actually no difference in terms of memory layout or the calculations involved, only how they are represented mathematically.

[hydrodog]

Thanks, I was desperately trying permutations after finding code showing the multiplication the other way.
The code still does not work. The text is going down and to the right because y is positive downward, which is fine, but the letters are in their original orientation. I want to rotate the entire path of letters so the line is oriented at the angle.

  glm::mat4 t(1);
  t = glm::translate(t, glm::vec3(-x,-y,0));
  t = glm::rotate(t, ang, glm::vec3(0,0,1));
  t = glm::translate(t, glm::vec3(x,y,0.0f));

    float x0 = x + glyph->bearingX,
              x1 = x0 + glyph->sizeX;  // TODO: Not maxwidth, should be less for
                                       // proportional fonts?
    float y0 = y - glyph->bearingY, y1 = y0 + glyph->sizeY;
    glm::vec4 vec0(x0, y0, 0, 1);
    vec0 = t*vec0;
    x0 = vec0.x, y0 = vec0.y;
    glm::vec4 vec1(x1, y1, 0, 1);
    vec1 = t*vec1;
    x1 = vec1.x, y1 = vec1.y;

In the debugger, transforming the first coordinate (100, 277) turns into -365,250
I can do the transform myself, I would just rather understand how glm::mat4 works.

[GClements]

@hydrodog

  t = glm::translate(t, glm::vec3(-x,-y,0));
  t = glm::rotate(t, ang, glm::vec3(0,0,1));
  t = glm::translate(t, glm::vec3(x,y,0.0f));

Note that this rotates about (-x,-y); reverse the order to rotate about (x,y).

As for why the glyphs themselves aren’t rotated, you’d need to show the relevant code. But you appear to be transforming only two of the corners. You need to transform all four; you can’t construct the transformed positions of (x0,y1) and (x1,y0) from the transformed positions of (x0,y0) and (x1,y1).

[hydrodog]

The bottom left point is (x0,y0). The top right is (x1,y1)

addPoint(x0, y0, /* fontLeft */ glyph->u0, glyph->v1);
addPoint(x0, y1, /* fontLeft */ glyph->u0, glyph->v0);
addPoint(x1, y1, /* fontRight */ glyph->u1, glyph->v0);
addPoint(x0, y0, /* fontLeft */ glyph->u0, glyph->v1);
addPoint(x1, y1, /* fontRight */ glyph->u1, glyph->v0);
addPoint(x1, y0, /* fontRight */ glyph->u1, glyph->v1);

The code works for horizontal, and the plan was just to compute the 4 points on an angle.
The math is definitely wrong, because I wrote it by hand and get the same outcome, so the problem is that what I intended is wrong, it isn’t a problem with the matrix implementation:

Given a starting point xc, yc, I thought to rotate the line of text by angle a.
Here is the entire function:

inline void rotateAround(float xc, float yc, float cosa, float sina, float& x, float& y) {
  const double dx = x - xc, dy = y - yc;
  x = xc + dx * cosa - dy * sina;
  y = yc + dx * sina + dy * cosa;
}

void MultiText::add(float x, float y, float ang, const Font* f,
    const char s[], uint32_t len) {
  float startx = x, starty = y;
  double cosa = cos(ang), sina = sin(ang);
for (uint32_t i = 0; i < len; i++) {
    const Font::Glyph* glyph = f->getGlyph(s[i]);
    float x0 = x + glyph->bearingX,
          x1 = x0 + glyph->sizeX;  // TODO: Not maxwidth, should be less for
                                   // proportional fonts?
    float y0 = y - glyph->bearingY, y1 = y0 + glyph->sizeY;
    rotateAround(startx, starty, cosa, sina, x0, y0);
    rotateAround(startx, starty, cosa, sina, x1, y1);
    addPoint(x0, y0, /* fontLeft */ glyph->u0, glyph->v1);
    addPoint(x0, y1, /* fontLeft */ glyph->u0, glyph->v0);
    addPoint(x1, y1, /* fontRight */ glyph->u1, glyph->v0);
    addPoint(x0, y0, /* fontLeft */ glyph->u0, glyph->v1);
    addPoint(x1, y1, /* fontRight */ glyph->u1, glyph->v0);
    addPoint(x1, y0, /* fontRight */ glyph->u1, glyph->v1);
    x += glyph->advance;
  }
}

[GClements]

@hydrodog

    rotateAround(startx, starty, cosa, sina, x0, y0);
    rotateAround(startx, starty, cosa, sina, x1, y1);
    addPoint(x0, y0, /* fontLeft */ glyph->u0, glyph->v1);
    addPoint(x0, y1, /* fontLeft */ glyph->u0, glyph->v0);
    addPoint(x1, y1, /* fontRight */ glyph->u1, glyph->v0);
    addPoint(x0, y0, /* fontLeft */ glyph->u0, glyph->v1);
    addPoint(x1, y1, /* fontRight */ glyph->u1, glyph->v0);
    addPoint(x1, y0, /* fontRight */ glyph->u1, glyph->v1);

This won’t work. You’re rotating two points then constructing an axis-aligned rectangle which has those two points for two if its corners. You have to construct all four corners then rotate them:

inline void rotateAround(float xc, float yc, float cosa, float sina,
                         float x, float y, float& xout, float& yout) {
  const double dx = x - xc, dy = y - yc;
  xout = xc + dx * cosa - dy * sina;
  yout = yc + dx * sina + dy * cosa;
}

...
    float x00, y00, x01, y01, x10, y10, x11, y11;
    rotateAround(startx, starty, cosa, sina, x0, y0, x00, y00);
    rotateAround(startx, starty, cosa, sina, x0, y1, x01, y01);
    rotateAround(startx, starty, cosa, sina, x1, y0, x10, y10);
    rotateAround(startx, starty, cosa, sina, x1, y1, x11, y11);
    addPoint(x00, y00, /* fontLeft */ glyph->u0, glyph->v1);
    addPoint(x01, y01, /* fontLeft */ glyph->u0, glyph->v0);
    addPoint(x11, y11, /* fontRight */ glyph->u1, glyph->v0);
    addPoint(x00, y00, /* fontLeft */ glyph->u0, glyph->v1);
    addPoint(x11, y11, /* fontRight */ glyph->u1, glyph->v0);
    addPoint(x10, y10, /* fontRight */ glyph->u1, glyph->v1);

[hydrodog]

Ugh, thanks! That was a stupid one. I still have a bug but the angle is clean.