Migeran LibGodot Feature

[kisg]

Features

• Compile Godot Engine as either static or shared library.
• Expose Godot Engine controls (startup, iteration, shutdown) over the GDExtension API to a host process.
• On Apple Platforms, Godot Engine windows (both the main window, and other windows created after startup) may be rendered into surfaces provided by the host process.
  • This support can be extended to other platforms, read on for details.
• Sample applications (separate repository)
  • C++ Sample (tested on MacOS and Linux)
  • SwiftUI Sample (tested on iOS)

Why LibGodot?

LibGodot has a number of different use cases.

• Control Godot Engine from a host application
  • E.g. Start Godot from a .NET process to use standard .NET tooling
• Use it for automation of development tasks (e.g. building an asset pipeline using a Python GDExtension API)
• Embed Godot into another application while displaying Godot Windows as part of the host application's UI.

How to try it out?

We provide a Github repository with preconfigured build environment, sample apps and more information: https://github.com/migeran/libgodot_project

Migeran LibGodot Design

The core idea of the LibGodot feature is to build upon Godot's strong extension capabilities: its modularity, powerful type system and the GDExtension API.

The main advantage of this approach is, that this way LibGodot may be used from any language that has a GDExtension binding (e.g. C++, Swift, Rust, Python, ... etc.), with only minimal changes to the binding.

Godot Instance Lifecycle Management

The main class added by the LibGodot design is the GodotInstance:

class GodotInstance : public Object {
	GDCLASS(GodotInstance, Object);

	static void _bind_methods();

	bool started = false;

public:
	GodotInstance();
	~GodotInstance();

	bool start();
	bool is_started();
	bool iteration();
	void shutdown();
};

This class is made accessible over the GDExtension API. This class can be used to control a Godot instance.

To actually create a Godot instance a new symbol is added to the GDExtension API:

GDExtensionObjectPtr gdextension_create_godot_instance(int p_argc, char *p_argv[], GDExtensionInitializationFunction p_init_func);

This function can be used to create a new Godot instance and return a GodotInstance object reference to control it. Both samples show how easy it is to bind this function and then use the generated GDExtension API bindings with the returned GodotInstance object.

To properly destroy a Godot instance the GDExtension API is extended by another symbol:

typedef void (*GDExtensionInterfaceDestroyGodotInstance)(GDExtensionObjectPtr p_godot_instance);

This function is made available through the GDExtension API's getProcAddress mechanism.

Note: Due to Godot's internal architecture (the use of global data structures and singletons) only one Godot instance may be created in a process.

Embedding Godot UI

Note: UI embedding is currently implemented for Apple platforms. Please read the Next Steps section for more information on the status of other platforms.

To allow for embedding the Godot UI into a host process the host process needs to be able to pass the necessary data about a native surface where
Godot may render its contents. The form of this native surface is entirely platform and rendering backend specific.

The Migeran LibGodot design adds the following types to Godot to allow this:

• A new DisplayServerEmbedded implementation which uses externally provided native surfaces as its rendering targets.
• A RenderingNativeSurface class and its associated platform specific subclasses, e.g. RenderingNativeSurfaceApple.
• The Window class is extended by a set_native_surface(Ref<RenderingNativeSurface>) method which allows specifying the rendering target of a Window in a typesafe, platform independent manner. It is even possible to change the rendering target of a Window dynamically during its lifecycle.

These classes are all exposed to the GDExtension API, so it is easy to use them from the host process.

The DisplayServerEmbedded class also exposes methods that allow the injection of input events from the host process into the Godot instance.

The RenderingNativeSurface class hierarchy has an additional function: provides a mechanism using the Factory Method design pattern to create compatible RenderingContextDrivers for a native surface.

This allowed us to make the DisplayServerEmbedded class platform agnostic.

We also refactored the other DisplayServer implementations to use RenderingNativeSurface during initialization.

Since all these classes are exposed over the GDExtension API, these can be seamlessly used from any supported language with a GDExtension binding.

Rationale for RenderingNativeSurface Design

For those who are familiar with Godot Engine internals: there was already a way to pass in platform specific native surface data (called WindowPlatformData) during initialization.

Why is this refactoring necessary to a full fledged reference counted object?

• We chose reference counting because it makes it easier to use on the client side, no need to manage the lifecycle manually. Earlier versions of this PR used simple Godot Objects, but they required manual memory management which was error prone.

• While on Apple platforms it is very easy to pass in a CAMetalLayer reference, and Metal (and thus MoltenVk) will happily render into it, other platforms impose way more restrictions.

  • For example: On Windows and Linux a separate Vulkan instance and the use of external textures is required to render Godot on a separate thread from the host application's main renderer thread.

  • This is not just a theoretical option: We already implemented a prototype on Linux and Windows based on Godot 4.2, where the host process and Godot are using Vulkan External Textures to pass the rendered frames from Godot to the host process. We intend to upgrade and refactor this patch to the current version of the LibGodot patch.

  • To use External Textures a thread-safe queue has to be implemented between between the host process and Godot, and a reference-counted RenderingNativeSurface subclass would be an ideal place to implement it, e.g. RenderingNativeSurfaceVulkanExternalTextureWin32.

Next Steps

Open for Discussion

We are happy to discuss any part of the design, naming conventions, additional features ... etc.

Merging for Godot 4.3 as an Experimental Feature

• We propose to merge the current LibGodot patch as an experimental feature in Godot 4.3, because it is already very usable in its current form for many use cases.

Godot 4.4 Developments

During the Godot 4.4 development cycle additional features may be added, for Example:

• Support for additional platforms for UI embedding: Android, Windows, Linux (both X11 and Wayland).
• Support for OpenGL (Native / ANGLE) UI embedding on all platforms.
• Add a Configuration API that can be used on startup instead of supplying command line parameters.

Sponsors & Acknowledgements

• Initial development sponsored by Smirk Software
• Rebasing to Godot 4.3 and further development sponsored by Xibbon Inc.

* The GDExtension registration of the host process & build system changes were based on @Faolan-Rad's LibGodot PR: #72883

About the Authors

This feature was implemented by Migeran - Godot Engine & Robotics Experts.

• Production edit: This closes Add a build-time option to compile Godot as a shared library (for use in other languages) godot-proposals#6267.

[fire]

You can fix some of the github ci actions tests with pre-commit run -a to format the text.

Here are the rest of the errors. https://github.com/V-Sekai/godot/actions/runs/8642913827

[dsnopek]

Thanks! I'm so excited to finally see this PR :-)

I looked only at the GDExtension-related changes so far. There's a bunch of rendering and display server related changes that I haven't looked at.

At high-level, this seems like a pretty good approach. Also, I really appreciate that the code changes to GDExtension are fairly minimal. :-)

On PR #72883, we discussed the possibility of introducing a new concept, perhaps called GDExtensionLoader, that would encapsulate the loading behavior, so that we could have GDExtensionLibraryLoader that used OS::open_dynamic_library() (so the current way) and then maybe a GDExtensionEmbeddedLoader that would just take an initialization function.

However, given how simple the changes are here, I think we could perhaps save something like that for a future refactor? It could make things a little cleaner by not having to constantly check if GDExtension::library is null or not, though.

[dsnopek]

This PR introduces a bunch of new names, and, while I don't necessarily want to start bikeshedding, we probably should discuss if these are the names we want to use, and, in particular, are they internally consistent, and consistent with names already used in other parts of Godot.

So, here we have:

• "lib_godot" as the fake entry point name
• "res://__LibGodot" as the fake library path. (Also, this needs to be unique among all extensions - with this PR, are developers able to register only one faux GDExtension from their application or multiple? In @Faolan-Rad's PR, if I'm remembering correctly, more than one faux GDExtension could be registered. If multiple are allowed, we'll need a unique path for each.)

And elsewhere we have:

• The LIBRARY_ENABLED define
• The "Embedded" feature tag
• The general term "embedded" to describe a faux GDExtension registered through libgodot

These seem a little bit inconsistent, using the terms "libgodot", "embedded" and "library" in different places.

[kisg]

I agree, and we are open to naming suggestions. I agree that it is important to have consistent naming. Should we just try to use LIBGODOT everywhere as the key for this? Or should we move to the more generic "LIBRARY" name?

[dsnopek]

Coming back to the naming question because it seems there's still some mixed names (although, it's much improved from the last time I went through this PR).

Personally, I don't like the super generic "library".

I'd be for either "LibGodot" or "embedded Godot", since they are more descriptive, with maybe a slight preference for "LibGodot" because that sounds more like a product/feature name, making it easier to refer to.

If we did go for "LibGodot", then we'd end up with a LIBGODOT_ENABLED define, libgodot:// path, libgodot feature tag, and "LibGodot" in the docs and other human-readable places.

I'm surprised there hasn't been more bike-shedding about the name :-)

[fire]

I've tried to get the Godot Engine Github actions continuous integration to pass. Most tests pass except for Windows. https://github.com/V-Sekai/godot/tree/vsk-libgodot-migeran-4.3

Edited: Time is limited for me to work on this, so I hope this can help your efforts.

[kisg]

Thanks! I'm so excited to finally see this PR :-)

Thank you, and thank you for taking the time to review it!

I looked only at the GDExtension-related changes so far. There's a bunch of rendering and display server related changes that I haven't looked at.

At high-level, this seems like a pretty good approach. Also, I really appreciate that the code changes to GDExtension are fairly minimal. :-)

On PR #72883, we discussed the possibility of introducing a new concept, perhaps called GDExtensionLoader, that would encapsulate the loading behavior, so that we could have GDExtensionLibraryLoader that used OS::open_dynamic_library() (so the current way) and then maybe a GDExtensionEmbeddedLoader that would just take an initialization function.

However, given how simple the changes are here, I think we could perhaps save something like that for a future refactor? It could make things a little cleaner by not having to constantly check if GDExtension::library is null or not, though.

I think introducing the concept of library loaders would benefit not just this use case, but also other language support that are implemented as GDExtensions. If you recall, I proposed a feature like that in my .NET integration proposal: https://docs.google.com/document/d/1QwZpo3oIKHyita1mDOIB_xIxdwJ5rtAmpY4_vP9S02Y/edit?usp=sharing

However, I am also not sure if we should try to push this whole loader concept into this PR, we could do that in a follow-up PR. We can discuss this more in the GDExtension meeting.

[kisg]

I've tried to get the Godot Engine Github actions continuous integration to pass. Most tests pass except for Windows. https://github.com/V-Sekai/godot/tree/vsk-libgodot-migeran-4.3

Edited: Time is limited for me to work on this, so I hope this can help your efforts.

Thank you @fire ! Would it be ok if I squash your changes into a single commit, and add it to our branch?

[fire]

Do what what works for you.

Edited:

As the initiator of the libgodot-style patch, I've successfully persuaded several individuals, including @Faolan-Rad to contribute to the libgodot project. I am excited about the new places we can use Godot Engine.

• Elixir-02 branch on V-Sekai's Godot repository
• Godot Engine Proposal #4773
• Blog post on Tirania.org

[fire]

@kisg Can you review the changes to libgodot from here V-Sekai#35?

Here's the modified libgodot_project project working on Windows. https://github.com/V-Sekai/libgodot_project. You may need to force mingw.

[kisg]

Just a quick note:

I just pushed an update to our libgodot PR, rebased to the current master, which includes many new features, like:

• start / stop support
• Metal simulator support
• Embedding support using external textures / shared OpenGL contexts (required on platforms like Windows and Linux X11 and for toolkits like Qt/QML)

Additional features like Android support already work as well (as you have seen in our GodotCon presentation), but we intend to submit that as a separate PR, because this PR is becoming pretty huge already. We are also open to submit features not directly related to LibGodot (like Metal Simulator support) separately, but we did not want to delay this update any longer.

We are working on updating the samples + adding new ones in https://github.com/migeran/libgodot_project, you can expect them there next week, will update you here when ready.

We (the Migeran team) intend to attend the upcoming Core, GDExtensions and Rendering meetings in the coming weeks where we can discuss the details of this PR.

Let's get LibGodot into 4.6!

[clayjohn]

I've just taken a quick looks so far. But I can say for certain that this PR is not ready for serious PR review yet. There are hundreds of files touched with changes unrelated to the actual content of the PR. Reviewing this will already be challenging enough without having to sift through hundreds of unrelated changes. To highlight the problem. Github won't even show me all of the changed files because this PR exceeds 300 changed files.

Please trim down the PR to only contain things related to LibGodot in order to ease the burden on PR reviewers and ensure we can review this in a reasonable amount of time. Keep in mind that most of us are doing this in our free time, so anything you can do to reduce the burden will have a direct impact on how soon this can get properly reviewed

I have left a few comments after a preliminary review. There is still a lot of work to do.

[clayjohn]

We are not going to call glGetIntegerv every time we want to bind a framebuffer. That will absolutely kill performance on low end platforms. The GL driver is not a general repository of information that you can request information from when you need, glGetIntegerv can force a flush on some drivers

[clayjohn]

This needs to be renamed. It has nothing to do with the metal simulator.

In OpenGL, we use a fill called std_lib_inc.glsl to contain these kind of workarounds. Although there it was more because we needed to polyfill certain built in functions that aren't available in GLSL 330 / ES 300.

For the RD backend, it could be something like compat_methods_inc.glsl

[clayjohn]

These need better names. "Fix" is only an appropriate suffix for a temporary hack. It is not appropriate for production-ready code

[clayjohn]

This will break the shader baker. Shaders cannot have dependencies on the system from which they are created.

[clayjohn]

This should be in platform code. We shouldn't be leaking platform abstractions into the rendering server

[clayjohn]

Can you provide some context to why this exists?

Right off the bat, its a huge red flag to have so much OpenGL code leaking out of the driver folder.

I am very wary of having an entire system that appears to duplicate what already exists, but sitting in an entirely different part of the code base. I suspect that this design (whatever its intended purpose) will be an enormous maintenance burden and will basically result in LibGodot being extremely brittle and prone to breaking from seemingly unrelated changes

[clayjohn]

This function should be replaced with an LUT

[clayjohn]

I'm very skeptical about handling input through a DisplayServer. Can you provide some context why this is needed and can't use the usual input handling route?

[clayjohn]

These seem unrelated to the PR. Does this fix an unreported bug or something?

[clayjohn]

Can you provide some background on this change. I thought that we already handled error printing from includes in the normal shader error handling code.

For context, users find it very annoying to get spammed by shader include error thats are handled elsewhere, so I would avoid spamming them more without a really good reason

[GeorgeS2019]

@zorbathut

Please trim down the PR to only contain things related to LibGodot in order to ease the burden on PR reviewers and ensure we can review this in a reasonable amount of time. Keep in mind that most of us are doing this in our free time, so anything you can do to reduce the burden will have a direct impact on how soon this can get properly reviewed

You are one of the few perhaps has the overview to contribute/assist how to compartmentalize this PR into the First minimum core changes, independent of platforms, to speed up the review process and to ensure that CI/CD GitHub action passing in agile way.

@dylanthesoldier17
#90510 (comment)

Please, if possible, go through the newly created codes, how to adapt the Displayserver so that there are possibilities available for MAUI, and older e.g. WPF and winform. So other .Net users could figure out a directx friendly API to create a windForm/WPF UserControl, allowing the embedded display of Godot engine within e.g. WPF/Winform to attract many millions of e.g. VB/C#/F# and industry IronPyrhon developers to become Godot user's.

[zorbathut]

compartmentalize this PR into the First minimum core changes, independent of platforms, to speed up the review process and to ensure that CI/CD GitHub action passing in agile way.

That's what #110441 is :)

I'd be interested in @kisg's opinion on my modifications to libgodot_create_godot_instance; instead of adding extra callbacks on every version I've chosen to include the necessary stuff (argc, argv, gdextension callback) and then basically a C-api-esque key/value dictionary for "further parameters". Which will hopefully be more long-term stable than a fixed DLL function.

I think that's the only significant change I made though, the rest is just all streamlining and removing stuff that isn't the core.

That said, I've spent roughly no effort trying to grok the embedded-display stuff; it's not a feature I need, I just want to get the barebones in place. I think this API will be able to reimplement the embedded-display stuff without signature changes, but a second opinion would be good.

[kisg]

Dear @clayjohn,

thank you very much for this early review. We are of course happy to break this up into smaller PRs, which we also offered back when we first submitted this PR.

This code drop was rushed out mostly to show what is in the pipeline (in reference to the other PR), and we even had to leave out some other pieces, like unifying the GL Manager code across all platforms (including EGL, ANGLE and platform specific GL-host integrations), which we also were using successfully in production with LibGodot.

Kind regards,
Gergely

[kisg]

I'd be interested in @kisg's opinion on my modifications to libgodot_create_godot_instance; instead of adding extra callbacks on every version I've chosen to include the necessary stuff (argc, argv, gdextension callback) and then basically a C-api-esque key/value dictionary for "further parameters". Which will hopefully be more long-term stable than a fixed DLL function.

Hi @zorbathut,

I have a solution in the works that integrates better with Godot Core, reduces the maintenance burden going forward, and does not require additional C-style functions in the future. I will be ready to show it before the Core meeting next week - I already added the topic to the meeting's agenda. It would be great if we could discuss it there if you have the time: https://files.godot.foundation/s/7jBFMWXjPg6EDsk?dir=/&openfile=true

Kind regards,
Gergely

[GeorgeS2019]

@kisg

Not to be impolite, I attach a link where I ask ChaGPT the difference in making LibGodot works with WinForn/WPF versus QT/QTL

migeran/libgodot#1 (comment)

The market potential for LibGodot/WinForm to attract corporate customer is very attractive

[zorbathut]

I have a solution in the works that integrates better with Godot Core, reduces the maintenance burden going forward, and does not require additional C-style functions in the future. I will be ready to show it before the Core meeting next week - I already added the topic to the meeting's agenda. It would be great if we could discuss it there if you have the time: https://files.godot.foundation/s/7jBFMWXjPg6EDsk?dir=/&openfile=true

Could you post a summary of it? I'd love to see a better idea!

[dylanthesoldier17]

@zorbathut

Please trim down the PR to only contain things related to LibGodot in order to ease the burden on PR reviewers and ensure we can review this in a reasonable amount of time. Keep in mind that most of us are doing this in our free time, so anything you can do to reduce the burden will have a direct impact on how soon this can get properly reviewed

You are one of the few perhaps has the overview to contribute/assist how to compartmentalize this PR into the First minimum core changes, independent of platforms, to speed up the review process and to ensure that CI/CD GitHub action passing in agile way.

@dylanthesoldier17 #90510 (comment)

Please, if possible, go through the newly created codes, how to adapt the Displayserver so that there are possibilities available for MAUI, and older e.g. WPF and winform. So other .Net users could figure out a directx friendly API to create a windForm/WPF UserControl, allowing the embedded display of Godot engine within e.g. WPF/Winform to attract many millions of e.g. VB/C#/F# and industry IronPyrhon developers to become Godot user's.

Sorry @zorbathut, I likely wouldn’t be much help quickly enough for your team’s push to get the libGodot PRs integrated, given my lack of experience with C++. I was speaking more from the C# side of things, where I see a lot of potential benefits in shifting toward a C#-first execution model.

[EricApostal]

@kisg Is DisplayServer embedding intended to work at this stage? I've been trying to set the native surface to a metal one but it doesn't appear that running with --embedded is respected, at least on macOS, and it never seems to actually show the embedded view. It seems that OS_MacOS_NSApp is being forced regardless? Also, I tried to look at examples for embedding but all I see is SwiftGodotKit (which is obviously on an old 4.4 version, of which I previously used and didn't have problems with). Running it as a linked library under a non embedded mode worked fine.

Is there an active example of something that should work within recent time? Or are these known issues?

Thanks!

[kisg]

@kisg Is DisplayServer embedding intended to work at this stage? I've been trying to set the native surface to a metal one but it doesn't appear that running with --embedded is respected, at least on macOS, and it never seems to actually show the embedded view. It seems that OS_MacOS_NSApp is being forced regardless? Also, I tried to look at examples for embedding but all I see is SwiftGodotKit (which is obviously on an old 4.4 version, of which I previously used and didn't have problems with). Running it as a linked library under a non embedded mode worked fine.

Is there an active example of something that should work within recent time? Or are these known issues?

Thanks!

Hi @EricApostal,

DisplayServer embedding is being used in production on Windows and iOS, Mac should work as well, but we don't have a sample for that yet. We are currently breaking up this PR into multiple interdependent smaller ones, and we are also updating the samples as part of that effort in the next days. I will update this PR when anything new is released.

[kisg]

Just pushed the first PR in the new series: #110863

[EricApostal]

DisplayServer embedding is being used in production on Windows and iOS, Mac should work as well, but we don't have a sample for that yet.

Are you sure this PR is the same as what you run in production? I'm still stuck on getting the DisplayServer running, and it seems as if it's disabled on anything other than MacOS. This line blocks DisplayServer from working at all on platforms other than MacOS, which was indeed hit when I tried to run in embedded display mode on Windows.

@kisg

[kisg]

@EricApostal good catch, this must have crept in with one of the rebases. We will do a fix for this.

[zorbathut]

C# support update:

I've got working libgodot C# integration going. Godot code, working example; note that both of these are branches. This is going to be finicky to get upstreamed because it's seven separate changes that touch multiple areas, some of which are kind of suboptimal due to C#'s current weird implementation in Godot, but I'll see what I can manage; fingers crossed on a PR soon, after I've talked to people.

[kisg]

@zorbathut Another option would be to integrate the new GDExtension based C# support with libgodot the same way we did with godot-cpp, SwiftGodot, etc. That should require minimal (or hopefully zero) changes to Godot itself.

https://github.com/raulsntos/godot-dotnet

[Gnumaru]

I've got working libgodot C# integration going

@zorbathut In your branch, are you initiating dotnet from godot? Or are you taking advantage of libgodot to initiate godot from dotnet?

[zorbathut]

@zorbathut Another option would be to integrate the new GDExtension based C# support with libgodot the same way we did with godot-cpp, SwiftGodot, etc. That should require minimal (or hopefully zero) changes to Godot itself.

Yeah, it's unclear to me how solid this new functionality is - there's still a bunch of issues on that and it isn't even in preview yet. Maybe it's working? But maybe it's working well enough for libgodot, which is sort of the more important part.

That said, that would eliminate only two out of the seven changes; the others are legit "Godot does not like being started from within a .NET environment and then told to start up its own .NET environment" issues.

(This isn't just "start Godot from C#", this is "start a .NET Godot build from a .NET environment", and Godot currently kind of steps on its own feet if you try doing that. No offense meant to the original developers, this usage was unforseeable when that code was written, I wouldn't have done any better.)

[zorbathut]

I've got working libgodot C# integration going

@zorbathut In your branch, are you initiating dotnet from godot? Or are you taking advantage of libgodot to initiate godot from dotnet?

Start a standard C# program, start Godot from that. This specific example doesn't include "and then Godot is running C# code of its own, and your Program can communicate directly with your game code, bypassing Godot entirely, if you want", but it does work; I've got plain vanilla NUnit testing in my own project, and it's seamless with test discovery and debugging.

So, Dotnet -> LibGodot -> Godot -> the exact same Dotnet environment.

[zorbathut]

ping @kisg

I'm talking with @raulsntos about the C# stuff, and they pointed out - accurately - that the API is cleaner if we pass around object IDs instead of actual pointers. The object ID registration is finished during start(), not start2(), so that entire structure already exists. This prevents a bunch of invalid-pointer errors and also means we don't have to muck about with GDExtension-library internal functions; ObjectDB::get_instance is already part of the GDExtension API and is guaranteed to be working by the time we get the ID.

I implemented this and it honestly seems cleaner. Here's the Godot-side diff, here's what it looks like in the driver. This means I can avoid calling godot::internal::get_object_instance_binding() for the C++ binding, and avoid an entire pain point with exposing Godot.NativeInterop.InteropUtils.UnmanagedGetManaged() for the C# binding.

Any reason not to make this change? We're stuck with whatever goes live with 4.6, so if this is better (which I think it is!), it's important to get on it ASAP.

Unless you've got a serious objection I'll go throw up a PR, just want to make sure there's nothing I'm missing.

[kisg]

ping @kisg

I'm talking with @raulsntos about the C# stuff, and they pointed out - accurately - that the API is cleaner if we pass around object IDs instead of actual pointers. The object ID registration is finished during start(), not start2(), so that entire structure already exists. This prevents a bunch of invalid-pointer errors and also means we don't have to muck about with GDExtension-library internal functions; ObjectDB::get_instance is already part of the GDExtension API and is guaranteed to be working by the time we get the ID.

I implemented this and it honestly seems cleaner. Here's the Godot-side diff, here's what it looks like in the driver. This means I can avoid calling godot::internal::get_object_instance_binding() for the C++ binding, and avoid an entire pain point with exposing Godot.NativeInterop.InteropUtils.UnmanagedGetManaged() for the C# binding.

I have no specific objections, I also see the merit in this change, except that you should be using GDExtensionObjectID as the return value / parameter instead of uint64_t directly. I think it should also be documented that GDExtensionObjectID value 0 means an invalid object. What do you think @dsnopek?

Any reason not to make this change? We're stuck with whatever goes live with 4.6, so if this is better (which I think it is!), it's important to get on it ASAP.

Another question for @dsnopek regarding this point: Should we mark these APIs (defined in libgodot.h, godot_instance.h) as experimental, so we can more easily change them if new use cases / requirements pop up? I think in the early days of 4.x the whole GDExtension API was marked experimental.

[kisg]

@zorbathut Another option would be to integrate the new GDExtension based C# support with libgodot the same way we did with godot-cpp, SwiftGodot, etc. That should require minimal (or hopefully zero) changes to Godot itself.

Yeah, it's unclear to me how solid this new functionality is - there's still a bunch of issues on that and it isn't even in preview yet. Maybe it's working? But maybe it's working well enough for libgodot, which is sort of the more important part.

That said, that would eliminate only two out of the seven changes; the others are legit "Godot does not like being started from within a .NET environment and then told to start up its own .NET environment" issues.

(This isn't just "start Godot from C#", this is "start a .NET Godot build from a .NET environment", and Godot currently kind of steps on its own feet if you try doing that. No offense meant to the original developers, this usage was unforseeable when that code was written, I wouldn't have done any better.)

I understand these issues, that is why my proposal was to use godot-dotnet + a minimal LibGodot C# binding (like your driver-cs), which bridges the libgodot C API to .NET, and then initializes godot-dotnet with those APIs during the initial GDExtension callback, like we do with godot-cpp, SwiftGodot / SwiftGodotKit and other languages.

This way by the time you call GodotInstance::start() from godot-dotnet in the host process, .NET in Godot will be already initialized, and all the scripts and GDExtension classes written in .NET can be loaded. This way Godot does not have to be compiled with the legacy .NET support, and thus it won't try to start a new .NET VM.

This might need some small initialization changes in godot-dotnet, so it does not try to load a .NET VM itself, but it should not require changes to Godot Core.

Regarding the stability of godot-dotnet, it would be great if @raulsntos could weigh in about the planned timeline, and any specific blockers.

In general, since this is definitely the future of .NET support in Godot, I think it would be great if LibGodot could help with steering developers towards it, because it already offers benefits to them, thanks to your .NET integration work.

That said, I am not opposed to supporting the legacy .NET APIs with LibGodot, and your changes to Godot seem to be pretty straightforward to me, but I don't like the addition of libgodot_create_godot_instance_and_start in your patch. The whole point of LibGodot is to return control to the host process as early in the initialization process as possible.

In a next iteration my plan is to refactor Godot init, so we can return the GodotInstance object even before the current Main::setup() ends:

• just after ClassDB, ObjectDB and GDExtension core initialization
• but before the current parsing of command line parameters and project loading or even touching any filesystem resources.

This way the Godot startup process can be more tightly controlled by the host process, and any potential platform-specific configuration (looking at you Android...) can be done using standard GDExtension APIs, without increasing the libgodot_* API surface.

I just did not want to open this discussion before at least the core of LibGodot is merged.

[zorbathut]

except that you should be using GDExtensionObjectID as the return value / parameter instead of uint64_t directly.

As near as I can tell, there isn't actually any such thing; unless I'm screwing up my search, that string doesn't show up in the codebase, and the C# API and the C++ GDExtension bindings have Object::get_instance_id() return a raw ulong/uint64_t respectively.

that is why my proposal was to use godot-dotnet + a minimal LibGodot C# binding (like your driver-cs), which bridges the libgodot C API to .NET, and then initializes godot-dotnet with those APIs during the initial GDExtension callback, like we do with godot-cpp, SwiftGodot / SwiftGodotKit and other languages.

While I completely agree with you, this is the "reimplement C# under GDExtension" project that's been going on for a year and a half now. I hope it happens; I'm not gonna be the one to do it, and certainly not before the 4.6 release :)

In a next iteration my plan is to refactor Godot init, so we can return the GodotInstance object even before the current Main::setup() ends:

just after ClassDB, ObjectDB and GDExtension core initialization

but before the current parsing of command line parameters and project loading or even touching any filesystem resources.

This sounds like a reasonable goal, but is going to be a lot of work! Note that on most platforms, GDExtension intrinsically touches the filesystem because many (most? all, except specifically for libgodot?) GDExtensions exist in dynamic-library form.

(also please add "logger initialization" before all of that :V)

---

#111553 pull request set up for the Instance ID thing

[kisg]

except that you should be using GDExtensionObjectID as the return value / parameter instead of uint64_t directly.

As near as I can tell, there isn't actually any such thing; unless I'm screwing up my search, that string doesn't show up in the codebase, and the C# API and the C++ GDExtension bindings have Object::get_instance_id() return a raw ulong/uint64_t respectively.

Sorry, I wrote it from memory, and (of course :)) it has a different name: GDObjectInstanceID https://github.com/godotengine/godot/blob/cb7cd815eeeb11aaa1f68453a515c76bec5ba73d/core/extension/gdextension_interface.h#L177C18-L177C36

[...]

In a next iteration my plan is to refactor Godot init, so we can return the GodotInstance object even before the current Main::setup() ends:

just after ClassDB, ObjectDB and GDExtension core initialization

but before the current parsing of command line parameters and project loading or even touching any filesystem resources.

This sounds like a reasonable goal, but is going to be a lot of work! Note that on most platforms, GDExtension intrinsically touches the filesystem because many (most? all, except specifically for libgodot?) GDExtensions exist in dynamic-library form.

The GDExtension core itself does not have to touch the filesystem, only when it starts to load the extensions themselves, which in this case will happen after the main extension is initialized, the project is loaded ... etc.

(also please add "logger initialization" before all of that :V)

Sure. :)

#111553 pull request set up for the Instance ID thing

[dsnopek]

I have no specific objections, I also see the merit in this change, except that you should be using GDExtensionObjectID as the return value / parameter instead of uint64_t directly. I think it should also be documented that GDExtensionObjectID value 0 means an invalid object. What do you think @dsnopek?

I gave my thoughts about this on #111553 (review)

Another question for @dsnopek regarding this point: Should we mark these APIs (defined in libgodot.h, godot_instance.h) as experimental, so we can more easily change them if new use cases / requirements pop up? I think in the early days of 4.x the whole GDExtension API was marked experimental.

We could certainly say these APIs are "experimental" which usually means we reserve the right to break compatibility. However, even if we reserve that right, I would strongly prefer not to exercise it, if we can!