Skip to content

Conversation

@vadorovsky
Copy link
Member

@vadorovsky vadorovsky commented Nov 5, 2025

LLVM 21.1.5 contains fixes that emit correct BTF[0] without the need to sanitize it, except from the name fixup that:

  1. Is going to be fixed up in Aya:
  2. Should be fixed in the kernel.

Introduce the di-sanitizer feature, that explicitly activates the use of DISanitizer. Enable it by default for now, but run Aya integration tests without it.

[0] llvm/llvm-project#165154


This change is Reviewable

@vadorovsky vadorovsky force-pushed the disable-di-sanitizer branch 5 times, most recently from 0ad0afb to 18fda8c Compare November 5, 2025 15:43
Copy link
Member

@tamird tamird left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tamird reviewed 2 of 3 files at r1, 7 of 7 files at r2, 9 of 14 files at r3, all commit messages.
Reviewable status: 13 of 18 files reviewed, 1 unresolved discussion


-- commits line 9 at r1:
are you going to send that patch? 🍿


tests/sanitizer/btf/assembly/auxiliary/loop-panic-handler.rs line 1 at r3 (raw file):

// no-prefer-dynamic

can these various files that are copies be symlinks instead?

It was never necessary in the first place. There is no place in the
kernel that enforces BTF map structs themselves to be anonymous. Only
pointer types (that are members of BTF maps) have to be anonymous, but
that's handled by a separate fixup.

BTF maps work just fine without it.
@vadorovsky vadorovsky force-pushed the disable-di-sanitizer branch from 18fda8c to da10a1e Compare November 5, 2025 17:15
@vadorovsky
Copy link
Member Author

vadorovsky commented Nov 6, 2025

Something fishy is going on with apt packages. The job that uses LLVM 21.1.5 from packages:

https://github.com/aya-rs/bpf-linker/actions/runs/19110300914/job/54605380603?pr=320

Get:1 https://apt.llvm.org/jammy llvm-toolchain-jammy-21/main amd64 libllvm21 amd64 1:21.1.5~++20251023083201+45afac62e373-1~exp1~20251023083316.53 [31.1 MB]
Get:2 https://apt.llvm.org/jammy llvm-toolchain-jammy-21/main amd64 libclang-cpp21 amd64 1:21.1.5~++20251023083201+45afac62e373-1~exp1~20251023083316.53 [14.0 MB]
Get:3 https://apt.llvm.org/jammy llvm-toolchain-jammy-21/main amd64 libpolly-21-dev amd64 1:21.1.5~++20251023083201+45afac62e373-1~exp1~20251023083316.53 [1854 kB]
Get:4 https://apt.llvm.org/jammy llvm-toolchain-jammy-21/main amd64 llvm-21-runtime amd64 1:21.1.5~++20251023083201+45afac62e373-1~exp1~20251023083316.53 [645 kB]
Get:5 https://apt.llvm.org/jammy llvm-toolchain-jammy-21/main amd64 llvm-21-linker-tools amd64 1:21.1.5~++20251023083201+45afac62e373-1~exp1~20251023083316.53 [1425 kB]
Get:6 https://apt.llvm.org/jammy llvm-toolchain-jammy-21/main amd64 llvm-21 amd64 1:21.1.5~++20251023083201+45afac62e373-1~exp1~20251023083316.53 [20.3 MB]
Get:7 https://apt.llvm.org/jammy llvm-toolchain-jammy-21/main amd64 llvm-21-tools amd64 1:21.1.5~++20251023083201+45afac62e373-1~exp1~20251023083316.53 [631 kB]
Get:8 https://apt.llvm.org/jammy llvm-toolchain-jammy-21/main amd64 llvm-21-dev amd64 1:21.1.5~++20251023083201+45afac62e373-1~exp1~20251023083316.53 [53.1 MB]

fails with:

failures:

---- tests::sk_storage::sk_storage_connect stdout ----
[DEBUG aya_obj::btf::btf] changing FUNC memcpy linkage to BTF_FUNC_STATIC
[DEBUG aya_obj::btf::btf] changing FUNC memmove linkage to BTF_FUNC_STATIC
[DEBUG aya_obj::btf::btf] changing FUNC memset linkage to BTF_FUNC_STATIC
[DEBUG aya_obj::btf::btf] changing FUNC memcmp linkage to BTF_FUNC_STATIC
[DEBUG aya_obj::btf::btf] [DATASEC] .maps: fixup size to 40
[DEBUG aya_obj::btf::btf] [DATASEC] .maps: VAR SOCKET_STORAGE: fixup offset 0
[WARN  aya::bpf] object BTF couldn't be loaded in the kernel: the BPF_BTF_LOAD syscall returned Invalid argument (os error 22). Verifier output: magic: 0xeb9f
    version: 1
    flags: 0x0
    hdr_len: 24
    type_off: 0
    type_len: 1120
    str_off: 1120
    str_len: 3738
    btf_total_size: 4882
    [1] PTR (anon) type_id=3
    [2] INT i32 size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
    [3] ARRAY (anon) type_id=2 index_type_id=4 nr_elems=24
    [4] INT __ARRAY_SIZE_TYPE__ size=4 bits_offset=0 nr_bits=32 encoding=(none)
    [5] PTR (anon) type_id=2
    [6] PTR (anon) type_id=7
    [7] STRUCT Value size=40 vlen=6
    	user_family type_id=8 bits_offset=0
    	user_ip type_id=9 bits_offset=32
    	user_port type_id=8 bits_offset=192
    	family type_id=8 bits_offset=224
    	type_ type_id=8 bits_offset=256
    	protocol type_id=8 bits_offset=288
    [8] INT u32 size=4 bits_offset=0 nr_bits=32 encoding=(none)
    [9] STRUCT Ip size=20 vlen=1
    	(anon) type_id=0 bits_offset=0 Invalid type_id
    

thread 'tests::sk_storage::sk_storage_connect' (27146) panicked at test/integration-test/src/tests/sk_storage.rs:17:62:
called `Result::unwrap()` on an `Err` value: MapError(CreateError { name: "SOCKET_STORAGE", io_error: Os { code: 22, kind: InvalidInput, message: "Invalid argument" } })
stack backtrace:
   0: __rustc::rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::result::unwrap_failed
   3: integration_test::tests::sk_storage::sk_storage_connect
   4: core::ops::function::FnOnce::call_once
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

Given that the "source" job uses our fork https://github.com/aya-rs/llvm-project, which currently is LLVM 21.1.3 with my patches applied, I though there is some regression in 21.1.5, but... I'm not able to reproduce that with LLVM built from source from the upstream tag https://github.com/llvm/llvm-project/releases/tag/llvmorg-21.1.5. And the BTF of Ip with it looks like:

#9: <STRUCT> 'Ip' sz:20 n:1
        #00 '<anon>' off:0 --> [10]
#10: <UNION> '<anon>' sz:20 n:3
        #00 '<anon>' off:0 --> [8]
        #01 'V4' off:0 --> [11]
        #02 'V6' off:0 --> [12]
#11: <STRUCT> 'V4' sz:20 n:1
        #00 '__0' off:32 --> [8]
#12: <STRUCT> 'V6' sz:20 n:1
        #00 '__0' off:32 --> [13]
#13: <ARRAY> n:4 idx-->[4] val-->[8]

I was thinking about proposing an upgrade to LLVM 21.1.5 to the Rust fork of LLVM https://github.com/rust-lang/llvm-project/tree/rustc/21.1-2025-08-01, but I will do that after solving that mystery.

@vadorovsky
Copy link
Member Author

vadorovsky commented Nov 6, 2025

OK, mystery solved. The current "21.1.5" deb packages from apt.llvm.org are not made from the actual 21.1.5 release (that was cut 2 days ago), but from some git snapshot of the release/21.x branch that was made on Oct 23.

https://apt.llvm.org/jammy/pool/main/l/llvm-toolchain-21/

🤡

I will re-enable fixups for the apt variant and remove them once packages are fresh enough to actually contain my fixes.

@tamird
Copy link
Member

tamird commented Nov 6, 2025

Jeez that's frustrating behavior from the LLVM apt maintainers.

@vadorovsky vadorovsky force-pushed the disable-di-sanitizer branch from da10a1e to 3d8c710 Compare November 7, 2025 08:03
`assembly/auxiliary/loop-panic-handler.rs` and
`btf/assembly/auxiliary/loop-panic-handler.rs` had exactly the same
content. Instead of keeping copies, make a symlink.
LLVM 21.1.5 contains fixes that emit correct BTF[0] without the need to
sanitize it, except from the name fixup that:

1) Is going to be fixed up in Aya:
   * aya-rs/aya#1382
2) Should be fixed in the kernel.

Introduce the `di-sanitizer` feature, that explicitly activates the use
of `DISanitizer`. Enable it by default for now, but run Aya integration
tests without it.

[0] llvm/llvm-project#165154
@vadorovsky vadorovsky force-pushed the disable-di-sanitizer branch from 3d8c710 to 2423a99 Compare November 9, 2025 16:13
Copy link
Member Author

@vadorovsky vadorovsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 4 of 19 files reviewed, 1 unresolved discussion (waiting on @tamird)


-- commits line 9 at r1:

Previously, tamird (Tamir Duberstein) wrote…

are you going to send that patch? 🍿

Yes, probably some time this week. 😅


tests/sanitizer/btf/assembly/auxiliary/loop-panic-handler.rs line 1 at r3 (raw file):

Previously, tamird (Tamir Duberstein) wrote…

can these various files that are copies be symlinks instead?

Yes, done

@vadorovsky vadorovsky marked this pull request as ready for review November 9, 2025 16:15
Copy link
Member

@tamird tamird left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@tamird reviewed 14 of 14 files at r4, 14 of 14 files at r6, 1 of 1 files at r7, 14 of 14 files at r8, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @vadorovsky)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants