2d RTS (starcraftish) in openGL

i’m new to openGL, I’m currently trying to port a 2d sprite based RTS game like starcraft to openGL because I want to use some 3d features and maybe convert some objects to 3d, can someone tell me what the best way is to do this?
I currently have the map displayed on the screen, I got that part down just fine using quads, but I have a couple thousand very small sprites. Is it a good idea to just place these all into textures? or is there a better way? and what are my texture dimension limitations? anyone have any source I can view for a 2d RTS in openGL?

Texture dimension limitations:
1.Both width and height must be power of 2 dimensions (e.g. 64x128).
2.Most recent GFX cards support textures up to 1024x1024, otherwise, it’s 256x256.

To render the sprites, the only way I can think of is by using textures (on quads) with an alpha channel for the transparent bits and using alpha test to discard those transparent texels.
Also, when rendering, sort your sprites by textures (if you have to use several textures) and render the ones with the same texture in one burst to minimise texture changes (since texture changes can be very expensive).

I was thinking of doing it somewhat like this:
take all sprites of the same dimensions and put them a singular texture and then draw them from that singular texture onto quads, can i do this? draw onto a quad a portion of a texture instead of the whole thing?
did that make sense?
is this possible? is this something that I would want to do? would it be faster than having thousands of textures and switching between them? well, thousands of sprites aren’t going to be on the screen at the same time though.

If you can make that optimization, then that is certainly something I would suggest you do. Remember to get the maximum texture size first of course before tiling the sprites into textures.