I am forming 1024*1024 texture only at a time.
Create several 1024x1024 textures and preload them. Use separate thread to load pages from disk in RAM. In main render thread refresh existing textures from RAM.
sry i have no idea about a free library for that purpose.
but the problem you have with your “megatextures” is a common one in flight simulators or similar.
be aware an unpreprocessed uncompressed file format is not very efficient and u wont get it much faster if u go that way.
i dont either recommend tiling the large texture into smaller ones, because u have then to fight with tri-linear filtering problems between tiles, which means u need redundant borders (very problematic with mipmaps) or some heavier fragment shader work.
but the common approach to that is called “texture clipmapping”
(u should find something useful with googling it),
which allows u to display !any! window-size of almost-infinite (if u would use progressive procedural synthesis) textures inclusive correct mipmap-antialiasing.
i.e. there are some approaches that use wavelet compression
, so u can progressively decode from lower to higher detail mipmaps. (in fact in “clipmapping” there do not exist mipmaps anymore, only clipmaps that have the same size as the toplevel)
there is one critical flaw in your approach,
you should detach the texture loading process from the rendering loop,
one of the best examples of that, u can find in unreal tournament 3: the game begins, even before the textures are fully loaded, only lower mipmap levels are available at level start, and while playing the game progessively loads the higher detail mipmaps.
mapped to ur problem means that: if u cant load the 1024x1024 fast enough, at least display a 512x512 or 256x256 version
but well, i need more information how u want to display ur 1024x1024 window of ur 5gb file, maybe u dont need mipmapping at all (cause maybe u dont do any 3D-transformations/projections or 2d-scalings).
if this is so, i would go the way yooyo suggested, with some extensions:
if i had to display any portion of a 5gb file with the requirement of low-latency access. i would tile that large texture into smaller tiles (half or quarter your windowsize seams reasonable), compress each tile using a compression library, so u can squeeze that 5gb easily into 500mb memory.
btw, 5gb is not “that” large in today terms, using DXT1 u have a compression ratio of 1:8 (from RGBA8 to DXT1_RGB) means 0.625gb which fits completely in today top cards.