Skip to content

Conversation

@wupq2022
Copy link

@wupq2022 wupq2022 commented Dec 9, 2025

Replace aligned_alloc() with posix_memalign() to fix crashes when using tcmalloc. The issue occurs because aligned_alloc() uses glibc's allocator while tcmalloc replaces free(), causing "invalid pointer" errors when freeing memory.

posix_memalign() is more compatible with tcmalloc and other memory allocators, as it properly integrates with the malloc/free interface that tcmalloc intercepts.

Changes:

  • Add #include <stdlib.h> for posix_memalign()
  • Replace aligned_alloc() with posix_memalign() in alloc_aligned()
  • Add proper error handling for posix_memalign() return value

This fixes crashes during index construction when InMemDataStore objects are destructed after saving the index.

  • Does this PR have a descriptive title that could go in our release notes?
  • Does this PR add any new dependencies?
  • Does this PR modify any existing APIs?
    • Is the change to the API backwards compatible?
  • Should this result in any changes to our documentation, either updating existing docs or adding new ones?

Reference Issues/PRs

Related to #527. This issue has been reported by multiple users experiencing the same "Attempt to free invalid pointer" error when using tcmalloc. This fix addresses the root cause of the compatibility issue between aligned_alloc() and tcmalloc's free() function.

What does this implement/fix? Briefly explain your changes.

This PR fixes a crash that occurs when building DiskANN indexes with tcmalloc enabled. The crash manifests as "Attempt to free invalid pointer" errors when InMemDataStore objects are destructed after saving the index.

Root Cause:

  • aligned_alloc() (C11) uses glibc's memory allocator directly
  • tcmalloc replaces the standard free() function
  • When tcmalloc's free() tries to free memory allocated by aligned_alloc(), it detects that the pointer was not allocated by tcmalloc and reports an "invalid pointer" error

Solution:

  • Replace aligned_alloc() with posix_memalign() in the alloc_aligned() function
  • posix_memalign() is older (POSIX standard) and better supported by memory allocators like tcmalloc
  • Memory allocated with posix_memalign() properly integrates with the malloc/free interface that tcmalloc intercepts
  • Added proper error handling for posix_memalign() return value (it returns an error code instead of NULL)

Changes:

  1. Added #include <stdlib.h> for posix_memalign() function
  2. Replaced aligned_alloc() call with posix_memalign() in alloc_aligned() function
  3. Added error handling to check return value and set pointer to nullptr on failure

Any other comments?

This change maintains backward compatibility as posix_memalign() has been available since POSIX.1-2001 and is widely supported. The function signature and behavior are equivalent for the use case in alloc_aligned(), with the only difference being the return value (error code vs. NULL pointer).

The fix has been tested and resolves the crash during index construction when using tcmalloc.

Replace aligned_alloc() with posix_memalign() to fix crashes when using
tcmalloc. The issue occurs because aligned_alloc() uses glibc's allocator
while tcmalloc replaces free(), causing "invalid pointer" errors when freeing
memory.

posix_memalign() is more compatible with tcmalloc and other memory allocators,
as it properly integrates with the malloc/free interface that tcmalloc
intercepts.

Changes:
- Add #include <stdlib.h> for posix_memalign()
- Replace aligned_alloc() with posix_memalign() in alloc_aligned()
- Add proper error handling for posix_memalign() return value

This fixes crashes during index construction when InMemDataStore objects
are destructed after saving the index.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant