Apparently, setting the GL_TEXTURE_MAG_FILTER and GL_TEXTURE_MIN_FILTER to GL_NEAREST is essential for the FS to be able to read depth texture.
And with that, I believe my problem is solved.
Thanks for taking the time to help 
Apparently, setting the GL_TEXTURE_MAG_FILTER and GL_TEXTURE_MIN_FILTER to GL_NEAREST is essential for the FS to be able to read depth texture.
And with that, I believe my problem is solved.
Thanks for taking the time to help 