Skip to content

Many fixes, come inside for a better explanation#383

Open
Lord-Kamina wants to merge 24 commits into
AppImageCrafters:mainfrom
Lord-Kamina:fix-deps-final
Open

Many fixes, come inside for a better explanation#383
Lord-Kamina wants to merge 24 commits into
AppImageCrafters:mainfrom
Lord-Kamina:fix-deps-final

Conversation

@Lord-Kamina
Copy link
Copy Markdown

First, sorry for an idiotically big PR... I began working on this when our workflows at https://www.github.com/performous/performous broke recently.

This is part of a larger effort to give whoever steps up as maintainer (I'm not it!) an all-around easier time. It partly relies on other PRs I have submitted at https://github.com/AppImageCrafters/AppRun and https://github.com/AppImageCrafters/tests-env

Here, I have done a number of changes:

  1. I have fixed the testing suite.
  2. I have added two environment variables:
  • APPIMAGE_BUILDER_PY_VERSION: This will be set automatically when running the script, and it's populated from sys.version_info. The idea is to use this in python recipes, and then you don't need to update them when there are updates to python.
  • APPIMAGE_BUILDER_REMOTE_REPO_OWNER: This will hold the repository-owner for any parts of the code that download or reference things from other repos. The idea is if somebody forks all three repos (appimage-builder, AppRun and tests-env) they can still have CI working on their forks. It's populated from the environment in __main__.py (and the CI workflows by default set it to ${{ github.repository_owner }}, with appimagecrafters as a default fallback.
  1. Modernize the python project by updating all requirements and getting rid entirely of the setup.py, migrating completely to pyproject.toml
  2. Fixes a few things in the appimage-builder and bash recipes.
  3. Many fixes and refactoring to the actions workflows, which will result in you having a saner and working CI pipeline again.
  4. I was working on fixing the compression thing in parallel to what got merged, it seems. I am making a change to that as well by defaulting to zstd because, according to the AppImage team, that should be the standard, as per XZ compression is broken #378

Below I will paste my explanation to a teammate of why I had to change some things in AppRun, as well as the structure of symlinks.

[20:45]Lord Kamina: The reason my builds were failing and the old ones are working is because the "compatibility" runtime is broken
[20:51]Lord Kamina: The way the compatibility thing works is, it bundles the glibc from the build machine in the AppImage
[20:52]Lord Kamina: And at runtime it compares the bundled and host glibc versions
[20:53]Lord Kamina: If the system's is newer, it uses that and if not it uses the bundled or "compatibility"
[20:53]Lord Kamina: But the compat version is broken for ubuntu at least
[20:54]Lord Kamina: Because ubuntu expects the ld-linux linker to be in /lib64
[20:55]Lord Kamina: And that symlink is not created by appimage-builder because it's a pretty non-standard location
[20:56]Lord Kamina: So you try to fire it, it can't find it and it fails with the "no such file error", which is just strerr(ENOENT)
[20:58]Lord Kamina: The reason the old appimages work is because they have ancient glibc versions so it just selects the normal os runtime
[20:58]Lord Kamina: But there's a bug in the comparison, it compares if system > bundled
[20:59]Lord Kamina: So when they're the same, it tries to use the broken bundled runtime, and that's why our CI was failing

This will probably take a while to digest and be reviewed properly but I'm willing to answer any questions.
In the meantime, you can go over to my fork and see how the CI works and download what should be a working appimage-builder AppImage made with python 3.12

@james-lawrence
Copy link
Copy Markdown

@Lord-Kamina would you be willing to refresh this off main? Its fine if not, but it'd be helpful, I plan on taking care of this on saturday for review/integrating.

@Lord-Kamina
Copy link
Copy Markdown
Author

Will do when I have some time.

Signed-off-by: Gregorio Litenstein <g.litenstein@gmail.com>
Signed-off-by: Gregorio Litenstein <g.litenstein@gmail.com>
Signed-off-by: Gregorio Litenstein <g.litenstein@gmail.com>
Seems I was working on this in parallel to what got merged.

My reasoning is this: Use the expanded `-noX` commands that mean the same as `—no-compresion` according to the documentation.

By default use zstd compression instead of xz, because the AppImage team has said on at least one occassion that’s what’s going in the standard spec.

Signed-off-by: Gregorio Litenstein <g.litenstein@gmail.com>
Signed-off-by: Gregorio Litenstein <g.litenstein@gmail.com>
@Lord-Kamina
Copy link
Copy Markdown
Author

There's something here that actually needs some discussion @james-lawrence.

re: The pydpkg update that enforces numeric versions... What I did was just add logic to side-step the bug by not trying to parse invalid versions. It works on my fork, and it allows to keep using the "continuous" tag. By contrast, #346 just changed "continuous" to 2.0.0, but I'm not sure that's the best idea.

P.S. that PR also hard-coded gzip compression, which is no longer supported.

Lord-Kamina added 11 commits May 7, 2026 15:30
Signed-off-by: Gregorio Litenstein <g.litenstein@gmail.com>
Signed-off-by: Gregorio Litenstein <g.litenstein@gmail.com>
Signed-off-by: Gregorio Litenstein <g.litenstein@gmail.com>
This doesn't change the default behavior, but makes the whole workflow much more friendly for people forking this and other AppImageCrafters repos.
Signed-off-by: Gregorio Litenstein <g.litenstein@gmail.com>
Signed-off-by: Gregorio Litenstein <g.litenstein@gmail.com>
Signed-off-by: Gregorio Litenstein <g.litenstein@gmail.com>
debug?

Signed-off-by: Gregorio Litenstein <g.litenstein@gmail.com>

don't need debug anymore
Signed-off-by: Gregorio Litenstein <g.litenstein@gmail.com>
Signed-off-by: Gregorio Litenstein <g.litenstein@gmail.com>
@james-lawrence
Copy link
Copy Markdown

I think combining both is fine. having an escape hatch for a condition that can hard block the release in that location is reasonable imo.

So the prelease doesn't begin hoarding files.

Signed-off-by: Gregorio Litenstein <g.litenstein@gmail.com>
@Lord-Kamina
Copy link
Copy Markdown
Author

I'm gonna have to fix the CI again... god, I hate pip.

Copy link
Copy Markdown

@james-lawrence james-lawrence left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only the CI comment is important the rest are questions that'd it be helpful to know and notes for me post merge.

update_string=recipe.AppImage["update-information"]() or "guess",
runtime_arch=recipe.AppImage.arch(),
compression=recipe.AppImage.comp() or "xz",
sign_key=recipe.AppImage["sign-key"]() or None,
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what did the sign_key do that caused you to remove it?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I honestly cannot remember. It might well be an oversight.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can leave it removed for now. I'll address post merge.


def setup(self):
self.move_glibc_to_compat_runtime()
ubuntu_compat_linker_folder = self.compat_runtime_path / "lib64"
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you remember what caused this code to be written?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is explained in the copypasta from a discussion I was having with someone on Discord.

Copy link
Copy Markdown

@james-lawrence james-lawrence May 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah right, sorry didnt register I read the description last week and didnt refresh when reviewing code this morning. =) note to self: document inline with the code post merge.

fetch-depth: 0
- name: Set up a virtualenv
run: |
python3 -m venv .venv
Copy link
Copy Markdown

@james-lawrence james-lawrence May 9, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

backout the sudo change and use git config --global --add safe.directory $PWD you dropped that command which is why you're seeing the ci errors about dubious ownership. I know its not optimal/feels dirty to have, but will get us over the initial merge hump and then it'll becomes my problem. =)

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm trying out changes in the main branch of my fork. I am also going to update the canned action versions but I want to get this fixed first. I am now having another issue on arch, which seems stem from it trying to download non-existing signatures. I'm not really familiar enough with Arch to understand how to fix this:

https://github.com/Lord-Kamina/appimage-builder/actions/runs/25603560688/job/75161484580

error: failed retrieving file 'core.db.sig' from mirror.rackspace.com : The requested URL returned error: 404
error: failed to synchronize all databases (failed to retrieve some files)
------------------------------ Captured log setup ------------------------------
ERROR    root:shell.py:30 "['/usr/sbin/fakeroot', '/usr/sbin/pacman', '--config', '/tmp/pacman-venv/pacman.conf', '-Sy', '--quiet']" execution failed
______________ ERROR at setup of TestVenv.test_read_package_data _______________

cls = <class 'tests.modules.deploy.pacman.test_venv.TestVenv'>

    @classmethod
    def setUpClass(cls):
        cls.appdir_path = Path("/tmp/AppDir")
        cls.venv_path = Path("/tmp/pacman-venv")
        cls.pacman_venv = Venv(
            cls.venv_path,
            repositories={
                "core": ["https://mirror.rackspace.com/archlinux/$repo/os/$arch"]
            },
            architecture="auto",
        )
>       cls.pacman_venv.update()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oi. interesting. most likely that mirror is out of sync/bad. since I actually run arch here is my current mirror list on my system. jsut replace it with some from here for now. that'll likely fix it.

################################################################################
################# Arch Linux mirrorlist generated by Reflector #################
################################################################################

# With:       reflector @/etc/xdg/reflector/reflector.conf
# When:       2026-05-09 13:49:03 UTC
# From:       https://archlinux.org/mirrors/status/json/
# Retrieved:  2026-05-09 13:49:03 UTC
# Last Check: 2026-05-09 13:27:16 UTC

Server = https://frankfurt.mirror.pkgbuild.com/$repo/os/$arch
Server = https://mirror.moson.org/arch/$repo/os/$arch
Server = https://ftp.halifax.rwth-aachen.de/archlinux/$repo/os/$arch
Server = https://arch.mirror.constant.com/$repo/os/$arch
Server = https://mirror.pseudoform.org/$repo/os/$arch
Server = https://mirror.kumi.systems/archlinux/$repo/os/$arch
Server = https://berlin.mirror.pkgbuild.com/$repo/os/$arch
Server = https://losangeles.mirror.pkgbuild.com/$repo/os/$arch
Server = https://de.arch.niranjan.co/$repo/os/$arch
Server = https://us.arch.niranjan.co/$repo/os/$arch

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From what I was reading, it's a config issue because those sig files do not actually exist on mirrors.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh dear. well I'm more interested in ubuntu passing atm than arch. I'll happily cleanup arch after merging this.

@Lord-Kamina
Copy link
Copy Markdown
Author

BTW and utterly random, I just noticed a folder has a typo, appimage-builder/appimagebuilder/modules/analisys, you might want to fix that eventually.

Signed-off-by: Gregorio Litenstein <g.litenstein@gmail.com>
Signed-off-by: Gregorio Litenstein <g.litenstein@gmail.com>
This reverts commit 27ca7b3.

Signed-off-by: Gregorio Litenstein <g.litenstein@gmail.com>
Signed-off-by: Gregorio Litenstein <g.litenstein@gmail.com>
@Lord-Kamina
Copy link
Copy Markdown
Author

@james-lawrence this should be good now. We should merge the changes in the other repos first though, as these depend on them.

@james-lawrence
Copy link
Copy Markdown

@azubieta going to need help getting the other repo's merged...

AppImageCrafters/tests-env#1
AppImageCrafters/AppRun#66

@Lord-Kamina
Copy link
Copy Markdown
Author

@azubieta going to need help getting the other repo's merged...

AppImageCrafters/tests-env#1 AppImageCrafters/AppRun#66

Org members only have write access on this repo?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants