Fix tcmalloc compatibility issue with aligned_alloc #683
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Replace aligned_alloc() with posix_memalign() to fix crashes when using tcmalloc. The issue occurs because aligned_alloc() uses glibc's allocator while tcmalloc replaces free(), causing "invalid pointer" errors when freeing memory.
posix_memalign() is more compatible with tcmalloc and other memory allocators, as it properly integrates with the malloc/free interface that tcmalloc intercepts.
Changes:
This fixes crashes during index construction when InMemDataStore objects are destructed after saving the index.
Reference Issues/PRs
Related to #527. This issue has been reported by multiple users experiencing the same "Attempt to free invalid pointer" error when using tcmalloc. This fix addresses the root cause of the compatibility issue between
aligned_alloc()and tcmalloc'sfree()function.What does this implement/fix? Briefly explain your changes.
This PR fixes a crash that occurs when building DiskANN indexes with tcmalloc enabled. The crash manifests as "Attempt to free invalid pointer" errors when
InMemDataStoreobjects are destructed after saving the index.Root Cause:
aligned_alloc()(C11) uses glibc's memory allocator directlyfree()functionfree()tries to free memory allocated byaligned_alloc(), it detects that the pointer was not allocated by tcmalloc and reports an "invalid pointer" errorSolution:
aligned_alloc()withposix_memalign()in thealloc_aligned()functionposix_memalign()is older (POSIX standard) and better supported by memory allocators like tcmallocposix_memalign()properly integrates with the malloc/free interface that tcmalloc interceptsposix_memalign()return value (it returns an error code instead of NULL)Changes:
#include <stdlib.h>forposix_memalign()functionaligned_alloc()call withposix_memalign()inalloc_aligned()functionAny other comments?
This change maintains backward compatibility as
posix_memalign()has been available since POSIX.1-2001 and is widely supported. The function signature and behavior are equivalent for the use case inalloc_aligned(), with the only difference being the return value (error code vs. NULL pointer).The fix has been tested and resolves the crash during index construction when using tcmalloc.