I have read this (some years ago)

3D Math Primer for Graphics and Game Development

You can find more information on the book site

The bad thing about this book is that all matrix are directX

style, so you have to transpose every matrix and invert every operation order.

It speak about vector, matrix, quaternion, geometric test…ecc…

The only part that I miss is floating point numerical robustness that is used for physics and AI tests.

Luckily I found a good chapter about numerical robustnessin the book “real time collision detection” by Christer Ericson, it speak about collision detection algorithms, Math (a lot of math) and numerical robustness. If you want to create a physics engine is a must.