Assert in lmem.cpp (page is nullptr) if Lua_State/lua_close are called from an external dll

[Vin789]

Hello !

I'm trying to integrate Luau in my personal project. I have a thirdparty project for it in VS that build Luau as a static library (was not able to build it as a dynamic library btw, seems corrupted each time, even from the cmake example), the same thing as the cmake command. Then I have an other project that reference it and build a dynamic library. Finally I have an application project that reference the dynamic library.

If I'm using luaL_newstate/lua_close directly inside the application project no problem. But if I'm using an helper class that encapsulate these call inside some function defined in a cpp an exposed with __declspec I'm crashing when lua_close is called.

Crashing in lmem.cpp on

LUAU_ASSERT(page && page->busyBlocks > 0);

page is nullptr.

If I move the code that use luaL_newstate/lua_close from the cpp to the h (no export needed) I'm not crashing anymore and everything is fine. I'm trying to fine if a compilation/linker flag set in my project is not compatible with the way Luau is build but I didn't find one yet.

Is there some restriction for referencing luau from a dynamic library ? Or maybe I missed something obvious in the documentation ?

Any help is appreciated.

Thx for your time.

[Vin789]

A little example on how to reproduce the issue with minimal code (the mentionned /h/cpp files are inside the attach zip).

// Step 1
mkdir cmake && cd cmake
cmake .. -DCMAKE_BUILD_TYPE=Debug

// Step 2
Open Luau.sln (tested with Microsoft Visual Studio Community 2022 (64-bit) - Version 17.14.15 (September 2025))
Solution Configuration: Debug
Righ click on Solution 'Luau' -> Build. Should build without issue.

// Step 3
Right click on solution, Add new project,
Empty C++ project, putting it inside the cmake directory
Naming it TestLuauLibrary. Deleting autogenerated header/resource/source filter.

// Step 4
Right click on the new TestLuauLibrary project, Add exiting item, select LuauScript (both h and cpp)
This is a simple class that encapsulate some Luau code.

// Step 5
Right click on the new TestLuauLibrary project, Properties (configuration Debug)

• General - Configuration type: static library
• General - C++ Language standard: ISO C++ 17
• General - C Language standard: ISO C 17
• C/C++ - Additional Include Directories
  -> ....\Compiler\Include
  -> ....\VM\Include
• Linker - General - Additional Library Directories
  -> ..\Debug (for luau libs)
• Input - Additional Dependencies
  -> Luau.Common.lib
  -> Luau.Ast.lib
  -> Luau.Compiler.lib
  -> Luau.VM.lib
• Apply + Ok
• Right click on the new TestLuauLibrary project, build. Should build without issue.

// Step 6
Right click on solution, Add new project,
Empty C++ project, putting it inside the cmake directory
Naming it TestLuau. Deleting autogenerated header/resource/source filter.

// Step 7
Right click on the new TestLuau project, Add exiting item, select main.cpp
Right click on the new TestLuau project, Set as startup project
Copy paste the LuaBridge directory inside the project directory near the main.cpp
This is a modified version of LuaBridge3 (https://github.com/kunitoki/LuaBridge3) compatible with latest Luau

// Step 8
Right click on the new TestLuau project, Properties (configuration Debug)

• General - Configuration type: Application (exe)
• General - C++ Language standard: ISO C++ 17
• General - C Language standard: ISO C 17
• C/C++ - General - Additional Include Directories
  -> ....\Compiler\Include
  -> ....\VM\Include
  -> LuaBridge
  -> ..\TestLuauLibrary
• Linker - General - Additional Library Directories
  -> ..\Debug (for luau libs)
  -> ..\x64 (for TestLuauLibrary libs)
  - Input - Additional Dependencies
  -> Luau.Common.lib
  -> Luau.Ast.lib
  -> Luau.Compiler.lib
  -> Luau.VM.lib
  -> TestLuauLibrary.lib
• Apply + Ok

// Step 9

• Compile and run TestLuau. It should run without assert or issue.
• At this step everything is working. Now let's switch to dynamic.

// Step 10

• Right click on the TestLuauLibrary project
• General - Configuration type: Dynamic library (.dll)
• C/C++ - Preprocessor - Preprocessor Definitions
  -> Add TEST_LUAU_DYNAMIC
  -> Add TESTLUAUEXPORT
• Apply + Ok
• Right click on the new TestLuauLibrary project, build. Should build with 2 warnings (for m_ByteCode/m_ScriptContent) because the __declspec is on the whole class. (Btw if the export is not the class it won't crash, it look like lua_State need to be exported to have the issue)

// Step 11

• Right click on the TestLuau project
• C/C++ - Preprocessor - Preprocessor Definitions
  -> Add TEST_LUAU_DYNAMIC
• Apply + Ok
• Compile and run TestLuau. On my side it's crashing inside lmem.cpp at function freeblock at the end of the program when lua_close is called.

Debug Luau.zip