core: harden trace lifecycle and serialization

[edsiper]

Summary

• preserve trace metadata across MessagePack and OTLP round trips, including span/link flags, link trace state, and dropped-link counts
• harden trace ownership, replacement setters, random IDs, MessagePack validation, and OTLP allocation/error handling
• support long MessagePack attribute keys plus CFL uint, null, byte, array, and kvlist values consistently
• reject malformed offsets, numeric overflow, excessive nesting, duplicate fields, invalid enums, opaque references, and partial traces
• make the text encoder safe for optional and long values
• expand regression coverage across lifecycle, APIs, MessagePack, OTLP, and text encoding

Root causes and impact

Several fields in the CTraces model were not serialized, causing trace metadata to change after buffering or forwarding. This included protocol flags used by sampling and remote-context handling.

Ownership operations could retain freed pointers during self-assignment or attach a span to two different trace contexts. MessagePack decoding also accepted offsets and numeric values that could wrap, used unbounded recursive attribute decoding, and allowed duplicate fields to overwrite earlier allocations.

The OTLP encoder built nested protobuf objects using best-effort void helpers. Allocation or conversion failures could leave nonzero element counts paired with null arrays, leak previously constructed objects, or pass incomplete payloads to protobuf-c. Construction now fails transactionally and unwinds all completed children.

Fluent Bit compatibility

• resource-span destruction remains compatible with Fluent Bit's historical manual-unlink sampling path
• MessagePack and OTLP round trips now retain sampling-relevant flags instead of silently resetting them
• public structure layout additions are appended, and existing successful API behavior is preserved

Validation

• ASAN_OPTIONS=detect_leaks=1 ctest --test-dir build --output-on-failure — 4/4 suites passed
• GCC AddressSanitizer and UndefinedBehaviorSanitizer build with leak detection — passed
• Clang 18 release build and tests — 4/4 suites passed
• ctraces-simple-c-api release example — passed
• git diff master...HEAD --check — passed

New tests cover direct and manual-unlink destruction, self-assignment, cross-context rejection, flags and dropped-count round trips, long keys, uint/null/byte values, excessive nesting, duplicate fields, invalid offsets/ranges/enums, opaque references, empty contexts, zero-length bytes, and incomplete nested payload rejection.