Retrieving pointer created/consumed by VkAllocationCallbacks

Suppose I have a handle to an object allocated with VkAllocationCallbacks I wrote myself. Using only the object handle, is there any way to retrieve the void* pointer returned by pfnAllocation, the same pointer that would be passed to pfnReallocation or pfnFree, but without actually resizing or destroying the object?

I want to experiment with allocating/storing extra metadata along with the object data using the same block of memory while still simply passing the object handle around. Something similar to cbWndExtra + GetWindowLongPtr/SetWindowLongPtr for Win32 windows.


To me, the most effective way to do that is to have a map from handles to the data.