Add TLS cache to tracking provider/critnib

[lplewa]

At this moment, if a user wants to free an allocation from the proxy pool, they can do something like this:

umf_pool_by_ptr(ptr, &pool);
umf_pool_free(pool, ptr);

Internally, proxy_free() calls umfMemoryTrackerGetAllocInfo(ptr, &allocInfo); to retrieve the allocation size. Then, the free function from the tracking provider is called, which removes the entry for ptr from the critnib tree (the removal process itself involves a tree search).

In this example, the same pointer is being searched for three times (noting that the removal process also performs a tree search).

This issue proposes introducing caching of the last search in the tracking provider. We have two approaches:

Extend Critnib with a Caching Feature
This approach requires using functions like pthread_setspecific (though the performance impact of this function is uncertain).
With this method, the pointer lookup could be reduced to one search.

Add Cache at the Tracking Provider Level
Since the tracking provider is static, a TLS cache can be added easily using __thread.
However, with the current implementation of critnib, a tree search is still required for removal, so this approach would reduce the number of searches from three to two

[vinser52]

I think the issue you described is only applicable to the proxy pool implementation which is not a real pool but just a thin proxy on top of the memory provider.
As I can see only the disjoint pool uses the same approach, but jemalloc_pool and scalable_pool do not use umfMemoryTrackerGetAllocInfo they store size internally.

It looks like a premature optimization that only suites proxy_pool needs. Also the performance impact need to be evaluated.

To be clear I am not against this feature, but we need to weigh performance gain vs code complexity before applying such enhancement.