[clr-interp] Add support for unmanaged thiscall calling convention #122600
Conversation
davidwrighton
commented
Dec 16, 2025
davidwrighton
commented
- Notably, on Windows platforms, the thiscall calling convention changes the argument order of the return buffer, and changes the rules where the return buffer pointer is placed
- This should also be good prep for the Swift calling convention work, which also needs to identify the swift calling convention at this point
- Also a drive by fix for the behavior of explicit this managed function pointers which return values in the return buffer, we were also missing a test cases for that scenario.
- Notably, on Windows platforms, the thiscall calling convention changes the argument order of the return buffer, and changes the rules where the return buffer pointer is placed - This should also be good prep for the Swift calling convention work, which also needs to identify the swift calling convention at this point - Also a drive by fix for the behavior of explicit this managed function pointers which return values in the return buffer, we were also missing a test cases for that scenario.
|
Tagging subscribers to this area: @BrzVlad, @janvorli, @kg |
1 similar comment
|
Tagging subscribers to this area: @BrzVlad, @janvorli, @kg |
|
Tagging subscribers to this area: @mangod9 |
1 similar comment
|
Tagging subscribers to this area: @mangod9 |
|
@kotlarmilos This change includes logic to discover the correct unmanaged calling convention in the call stub generator. Note that it handles the pinvoke, UnmanagedCallersOnlyAttribute, as well as the calli signature scenarios for specifying the calling convention. This will need to merge with the work you've done around the Swift calling convention. |
There was a problem hiding this comment.
Pull request overview
This PR adds support for unmanaged thiscall calling convention to the CLR interpreter, addressing platform-specific calling conventions (particularly on Windows) where the thiscall convention changes argument order for return buffers. The changes also fix handling of explicit this managed function pointers that return values in a return buffer.
Key Changes:
- Added detection and handling of unmanaged calling conventions (thiscall, CMemberFunction, etc.) in the interpreter call stub generator
- Implemented signature rewriting from ExplicitThis to HasThis for proper return buffer handling on Windows
- Extended test coverage for explicit this instance method calls returning large value types
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
src/tests/JIT/Regression/JitBlue/GitHub_35384/GitHub_35384.il |
Expands test to include large valuetype returns and verifies explicit this calling convention with return buffers |
src/coreclr/vm/siginfo.hpp |
Adds declaration for GetUnmanagedCallConvExtension helper function |
src/coreclr/vm/siginfo.cpp |
Implements GetUnmanagedCallConvExtension to extract calling convention from method signature |
src/coreclr/vm/callstubgenerator.h |
Updates ComputeCallStub signature to accept MethodDesc parameter |
src/coreclr/vm/callstubgenerator.cpp |
Core implementation: detects unmanaged calling conventions, rewrites signatures, and adds helper for native primitive struct detection |
src/coreclr/vm/callconvbuilder.hpp |
Declares new function to parse calling convention from return type position in signature |
src/coreclr/vm/callconvbuilder.cpp |
Refactors calling convention parsing to support starting from return type position |
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
Co-authored-by: Copilot <[email protected]>
2 participants
