Inconsistent DAG Import Errors Display in Airflow UI

[swapnilpatil0905]

Apache Airflow version

2.10.4

If "Other Airflow 2 version" selected, which one?

No response

What happened?

When refreshing the Airflow UI, the displayed count of DAG import errors is inconsistent. The following issues are observed:

1. The number of DAG import errors changes with each refresh. For example, errors may fluctuate between 13, 14, and 17.
2. Some old import errors disappear after a refresh but reappear during subsequent refreshes.

Impact:
This inconsistency makes it difficult to accurately identify and address DAG import issues, leading to potential confusion and inefficiencies in debugging workflows.

What you think should happen instead?

Expected Behavior:
The count and details of DAG import errors should remain consistent across UI refreshes until the underlying errors are resolved.

How to reproduce

Steps to Reproduce:

1. Open the Airflow UI.
2. Note the number of DAG import errors displayed.
3. Refresh the UI multiple times and observe the changes in the count of errors.
4. Compare the errors to verify if previously resolved issues reappear.

Operating System

NAME="Red Hat Enterprise Linux" VERSION="9.4 (Plow)" ID="rhel" ID_LIKE="fedora" VERSION_ID="9.4" PLATFORM_ID="platform:el9" PRETTY_NAME="Red Hat Enterprise Linux 9.4 (Plow)" ANSI_COLOR="0;31" LOGO="fedora-logo-icon" CPE_NAME="cpe:/o:redhat:enterprise_linux:9::baseos" HOME_URL="https://www.redhat.com/" DOCUMENTATION_URL="https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/9" BUG_REPORT_URL="https://issues.redhat.com/" REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 9" REDHAT_BUGZILLA_PRODUCT_VERSION=9.4 REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" REDHAT_SUPPORT_PRODUCT_VERSION="9.4"

Versions of Apache Airflow Providers

2.9.2

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg[bot]]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[ephraimbuddy]

DAGs are constantly being parsed, and if some are due to import timeouts during parsing, then those DAGs could pass at times and fail at other times. I think that's what you are experiencing, and it's not a bug

[tirkarthi]

Are the dag files in the dags folder python files or zip files?

[swapnilpatil0905]

@tirkarthi Few are in dags folder whereas few are zipped. Note: We are running on multipod webserver mode.

[rwitzel]

@ephraimbuddy How can one find whether this was due to an import timeout? The actually shown import error indicate real issues, not timeouts.

[potiuk]

@ephraimbuddy How can one find whether this was due to an import timeout? The actually shown import error indicate real issues, not timeouts.

You need to look at the errors. It does not have to be timeouts, it can be ANYTHING that changes between one parsing and second parsing. This basically means that parsing of some of your Dags is simply unstable and sometimes works, sometimes does not.

You need to look at those import errors (also they should be logged by dag file processor I think, and for sure they are stored in the database of Airflow (see ERD schema on our docs) and you can compare the errors when they differ. The actual extra error you see when you have 13 of them is what you should figure out and find out why it might sometimes cause import error and sometimes not.

It can be anything and it 100% depends on how your Dags are written - so only you and your team can investigate the root causes and fix them. There might be many, many different reasons - depending on the environmental changes - database timeouts, reading external files that might impact what happens during parsing, or maybe even potentially inconsistent environment your dag file processor(s) run - maybe some of your environment does not have some libraries installed. Also a number of users (completely against our recommendations) dynamically install dependencies in their environment rather than for example bake all dependencies in their images or have a read-only non-modifiable virtual env, and due to racing conditions that might cause all kinds of weirdness when parsing done in one environment (with some deps) might result with different results than the one done in effectively different environment (for example when you install or remove dependencies dynamicall without redeploying airflow in a "frozen" environment.

So .. all in your hands to analyse and find out what's wrong with your Dags and your environment rather than look for issue in Airflow.

I am converting that one to a discussion, would be nice to hear back what you find out after your investigations - that might help people who will have similar issues like you in the future when they see what mistakes could be avoided were done when deploying airflow or authoring Dags.

Also looking at "Best practices" in our docs might help - we explain there a number of things you should avoid when authoring Dags.