CI: runs tests on 4.x versions of Blender, drops tests on unsupported versions

[DougReeder]

What?

CI: runs tests on 4.x versions of Blender, drops tests on unsupported versions

Why?

We need to test against all supported versions, but not against unsupported versions

How to test

1. Navigate to Actions tab of this repo
2. Observe that all tests pass

Documentation of functionality

The docs don't state what versions of Blender this works with, so no changes are necessary

Limitations

Doesn't test 5.0, which is on the verge of being released

[Exairnous]

Before we can drop unsupported versions and add the supported versions we need to decide what versions we want to support (and then we should note it down as our policy).

The initial informal policy for the add-on was to only support the latest Blender release, but I have always thought that was too inflexible and it relaxed as time went by, though no formal policy was ever adopted.

Some possible policies we could adopt (in no particular order).

1. Support the currently maintained LTS versions of Blender, along with the latest version of Blender.
2. Support all the versions from the oldest currently maintained LTS to the latest version of Blender.
3. Support all the versions of a series, i.e. what you have here.
4. Support all the versions from a team decided starting version to the latest version of Blender.
5. Support several versions that are team decided.

For the options that contain team decided versions, what team should decide is another question. I would say that probably the art team should decide, but arguments can be made for the programming and governance teams too.

My preference would probably be for option 2 or option 4. Fun fact, I still predominantly use Blender 3.6 because the EEVEE shaders compile exceptionally quicker than future LTS releases. I just took some rough numbers and this is how long it takes for the EEVEE shaders to compile on the same, fairly simple, scene for the different versions:

4 seconds for Blender 3.6.14 LTS
4 seconds for Blender 4.0.1
4 seconds for Blender 4.1.0
107 seconds for Blender 4.2.1 LTS
42 seconds for Blender 4.5.1 LTS
42 seconds for Blender 5.0 beta

So even though it would normally make sense, I'm not sure whether I really want to drop support for Blender 3.6 yet.

Another thing to think about is what we do when we drop support for a version, i.e. do we just not test on it anymore, or do we remove any version specific support for it that was introduced?

@jua360 @hobbs-Hobbler @Spiderguy-F @GottfriedHofmann @aelainelong @theanine3D Feel free to weigh in on this as well (and feel free to tag anyone else who you think would be appropriate).

[hobbs-Hobbler]

I definitely have opinions on needing the flexibility to jump between versions of Blender.

Versions

In my Tron build, as I advertised, the heavy lifting was done by the textures. The essential 3D objects were 3 cubes and 1 cylinder which looked like busy, computer-like buildings and roads because of only 5 textures. Following a tutorial, I built the textures first in Blender 4.4.3 (and GIMP) simply because the drawing tools are more visible and I'm still learning how to draw in Blender (grease pencil) and I don't know how to easily find the same tools in some of the earlier versions. (Note: the textures, even though I "drew" them, are primarily straight lines, dots, and dashes. It is only clever artwork that changes them to look like roads, RAM, and motherboard components.)

I managed to make the Blender version of the build quickly and the visuals. I'll attach 2 Work-In-Progress images that matched the tutorial well (props to that teacher!). But initially what I saw in Blender translated to Hubs ZERO (Read: it was completely black b/c it was, essentially, unlit.)

I've been having terrible alpha sorting errors in Blender 4.4 and I simply cannot reliably build an immersive space (via a glb file) in anything higher than Blender 4.0 right now. Alpha sorting errors in a space where I was depending on transparency is bad news. Blender tutorials are happy to show you all of the new features of these 4.x versions, but in order to be sure things like lighting, fog, and the feel of the space work, I MUST use the Hubs Add-On to test spaces. So I went out of my way to open and use Blender 4.0 anytime I was working on the entire space, even though components like the textures, Light Cycle, and Solar Sailer were built in 4.4.

So my first point is that artists and creators NEED a reliable version to depend on when the stakes are high. I'd already promised a Hubs build for Halloween before I moved the glb into Hubs. Finding a finished and lit scene in Blender go totally black in Hubs will make you realize that there is a LOT more work to do.

Testing

For the Tron build, I spent easily 2 weeks daily testing small tweaks with the Hubs Add-On. It took 3 days to get the lighting right including considering that Flynn's Hideout has LOTS of silver/reflective textures so I had to decide what I could accept being seen in those reflections (like the dining room table).

For the larger city part, I had to fight between alpha vertex painting (which I eventually abandoned even though I thought I had a perfect and easy test case) and went with black fog to get the main building light beam to fade into the sky (a VERY important art feature for the Tron feeling that I was not willing to compromise on. I did NOT want the "top" of the beam to be easily seen. The beam could not look "tinker toy", it had to look like a power light beaming into space).

Further, I worked hard on the glowing tail texture that follows the Light Cycle (it glows from the top and bottom). So again, I had to hop between GIMP, Blender 4.0, and Blender 4.4 to get the transparency and color drop off just right. Repeat again since I re-did the same Light Cycle in purple. Finally, testing with the Hubs Add-on was critical to making sure that I got the art effect of "light propelled/light speed" that I was looking for.

So my second point is that I could not have made the Tron build (which was finished with Blender 4.0) without the Hubs Add-On. Without testing, I would have had a cute few seconds of Blender video zooming down a digital street....and that's it.

Notice that I'm testing "how much" bloom intensity and that the textures in the back are still their original Blender white (b/c Hubs did not understand the blue color ramp).

[DougReeder]

I think we should support (test on) all the currently supported versions of Blender (including LTS versions). I don't think we should remove code unless it makes non-trivial difficulties.

So, the plugin should continue to work with old versions of Blender, but we don't put any effort into doing so. If a new version of the plugin loses compatibility with an old version of Blender, users can still use an old version of the plugin.

Of course, we could make a special exception for a particular unsupported version of Blender, if the Art and Programming teams agreed.

[theanine3D]

I have successfully been maintaining compatibility with as far back as Blender 3.0 in my own Blender addons. So it's doable - but it does require some mindfulness and extra code to check for versions. Since we have a very limited amount of people, I would say being practical has to be the winning option.

I would go with option 1:
Support the currently maintained LTS versions of Blender, along with the latest version of Blender.

We don't need to outright block older versions - they should still be able to install and try running the addon, and much of the addon will likely still work. But we won't provide support for anything broken if they're running an unsupported version.

[DougReeder]

It looks like @Exairnous 's option 2

Support all the versions from the oldest currently maintained LTS to the latest version of Blender.

...with the possibility of adding specific releases, would satisfy everyone.

That makes the current list:
"4.2.15", "4.3.2", "4.4.3", "4.5.4", "5.0.0"

[Exairnous]

Thank you to everyone so far who has provided opinions on the policy for what Blender versions to support. I want to mull things over a bit before responding further, but in the mean time, I wanted to note a few things I found out about the tests failing in Blender 4.5 and 5.0.

It seems that our support for HDR images may be what's causing the issue. The uri for the image isn't being set when the tests are run, which causes a python error in the glTF add-on (this seems to be the specific line here: https://github.com/KhronosGroup/glTF-Blender-IO/pull/2560/files). It seems that this is actually a problem with exporting the glTF in separate parts (a .gltf file, .a bin file, and any images) with the glTF Separate format option since the glTF Binary and glTF Embedded format options don't encounter this error. If I export the environment-settings test file using either the binary or embedded formats, everything works fine, but if I use the separate format I get the same error the tests encounter (which also export using the separate format).

Since the embedded format is text based, we could potentially switch to using the embedded format for the tests, but this has the downside of sightly larger file sizes and potentially just sweeps the problem under the rug/out of sight.

I'll try to keep this PR updated with anything additional I find out about this issue.