I have a mesh with triangle and vertex information. Now how to extract a inside triangles of a submesh, whose boundary triangles and its vertices are known.

I assume you have vertex and face arrays. Do the boundary triangles give you the face indices? Also, do vertices give you their coordinate values or their reference numbers (i.e indices).

If you have indices, it is simple to get the corresponding triangles. If by vertices you mean coordinates, then you may have to search for range of vertices, in the cloud point, such that their coordinates fall between the desired interval along all axes.

To lessen search time, you may need to organize your data to aid fast search - sorting or arrange your data using K-d tree structure.