Airflow 3.1.1 UI extremely slow when loading DAGs (running on Docker Compose)

[jun-roh]

I’m running Apache Airflow 3.1.1 using Docker Compose, and I’ve noticed that the Web UI is extremely slow when loading data — especially the main DAG list and DAG details pages.

We currently have about 90 DAGs, and everything else (scheduler, workers, etc.) seems to be functioning normally. However, the UI becomes noticeably sluggish, sometimes taking 20–30 seconds to render pages.

Here’s a simplified version of my setup:

version: "3.8"

networks:
  airflow-net:
    driver: bridge

volumes:
  airflow-git:
    driver: local

x-airflow-common: &airflow-common
  image: ${AIRFLOW_IMAGE_NAME}
  user: "${AIRFLOW_UID:-50000}:0"
  restart: unless-stopped
  environment:
    # ---- Core ----
    AIRFLOW__CORE__EXECUTOR: CeleryExecutor
    AIRFLOW__CORE__STORE_SERIALIZED_DAGS: True
    AIRFLOW__CORE__LOAD_EXAMPLES: False

    # ---- Database ----
    AIRFLOW__DATABASE__SQL_ALCHEMY_CONN: "mysql+pymysql://${AIRFLOW_DB_USER}:${AIRFLOW_DB_PASSWORD_URL}@${AIRFLOW_DB_HOST}/${AIRFLOW_DB_NAME}"
    AIRFLOW__CELERY__RESULT_BACKEND: "db+mysql+pymysql://${AIRFLOW_DB_USER}:${AIRFLOW_DB_PASSWORD_URL}@${AIRFLOW_DB_HOST}/${AIRFLOW_DB_NAME}"

    # ---- Pooling ----
    AIRFLOW__DATABASE__SQL_ALCHEMY_POOL_SIZE: 30
    AIRFLOW__DATABASE__SQL_ALCHEMY_MAX_OVERFLOW: 60
    AIRFLOW__DATABASE__SQL_ALCHEMY_POOL_TIMEOUT: 60
    AIRFLOW__DATABASE__SQL_ALCHEMY_POOL_RECYCLE: 1800
    AIRFLOW__DATABASE__SQL_ALCHEMY_POOL_PRE_PING: True

    # ---- Scheduler ----
    AIRFLOW__SCHEDULER__PARSING_PROCESSES: 1
    AIRFLOW__SCHEDULER__DAG_DIR_LIST_INTERVAL: 120
    AIRFLOW__SCHEDULER__MIN_FILE_PROCESS_INTERVAL: 300
    AIRFLOW__SCHEDULER__MAX_TIS_PER_QUERY: 64

    # ---- Logging ----
    AIRFLOW__LOGGING__REMOTE_LOGGING: True
    AIRFLOW__LOGGING__REMOTE_BASE_LOG_FOLDER: s3://example-bucket
    AIRFLOW__LOGGING__REMOTE_LOG_CONN_ID: aws_default

    # ---- Celery ----
    AIRFLOW__CELERY__BROKER_URL: redis://redis.example.net:6379/2
    AIRFLOW__CELERY__TASK_TRACK_STARTED: True
    AIRFLOW__CELERY__PREFETCH_MULTIPLIER: 1

    TZ: Asia/Seoul
    volumes:
    - airflow-git:/opt/airflow/git
    - ./airflow_local_settings.py:/opt/airflow/airflow_local_settings.py:ro
  networks: [airflow-net]

services:
  airflow-apiserver:
    <<: *airflow-common
    command: >
      bash -c 'exec airflow api-server --host 0.0.0.0 --port 8080 --workers 1 --apps all'
    ports: ["8080:8080"]
    depends_on:
      airflow-gitsync: { condition: service_healthy }
      airflow-init: { condition: service_completed_successfully }

Environment summary
• Airflow version: 3.1.1
• Executor: CeleryExecutor
• DB: MySQL on AWS RDS (db.r7g.large)
• Redis: hosted Redis cluster (for Celery broker)
• DAG count: ~90
• UI: AWS loadbalancer using HTTPS
• Serialized DAGs: enabled (STORE_SERIALIZED_DAGS=True)

The Airflow UI is slow:
• DAG list takes 10–20 seconds to load
• Clicking into a DAG detail or “Graph View” is also very sluggish
• Scheduler and workers seem healthy, tasks execute fine
• Database CPU/memory usage is normal

[potiuk]

Try 3.1.2 released yesterday. There were number of n+1 (many database queries when querying for dags) issues solved in 3.1.2 - see the changelog.