[libc++] Reorganize and fix the libc++ CI dockerfiles (#167530)

Instead of having one large Dockerfile building multiple images with
relatively confusing inheritance, explicitly have three standalone
Dockerfiles each building one image. Then, tie the three images together
using the docker-compose file which explicitly versions the base image
used by the Android and the Github Actions images.

Author: ldionne

.github/workflows/libcxx-build-containers.yml

@@ -15,11 +15,11 @@ on:
     branches:
       - main
     paths:
-      - 'libcxx/utils/ci/**'
+      - 'libcxx/utils/ci/docker/**'
       - '.github/workflows/libcxx-build-containers.yml'
   pull_request:
     paths:
-      - 'libcxx/utils/ci/**'
+      - 'libcxx/utils/ci/docker/**'
       - '.github/workflows/libcxx-build-containers.yml'
 
 jobs:
@@ -40,17 +40,18 @@ jobs:
         echo '{ "data-root": "/mnt/docker" }' | sudo tee /etc/docker/daemon.json
         sudo systemctl restart docker
 
-    - name: Build the Linux builder image
-      working-directory: libcxx/utils/ci
-      run: |
-        docker compose build builder-base
-        docker compose build actions-builder
+    - name: Build the base image
+      run: docker compose --file libcxx/utils/ci/docker/docker-compose.yml build libcxx-linux-builder-base
+      env:
+        TAG: ${{ github.sha }}
+
+    - name: Build the Linux Github Actions image
+      run: docker compose --file libcxx/utils/ci/docker/docker-compose.yml build libcxx-linux-builder
       env:
         TAG: ${{ github.sha }}
 
     - name: Build the Android builder image
-      working-directory: libcxx/utils/ci
-      run: docker compose build android-buildkite-builder
+      run: docker compose --file libcxx/utils/ci/docker/docker-compose.yml build libcxx-android-builder
       env:
         TAG: ${{ github.sha }}
 
@@ -61,19 +62,21 @@ jobs:
         username: ${{ github.actor }}
         password: ${{ secrets.GITHUB_TOKEN }}
 
-    - name: Push the Linux builder image
+    - name: Push the images
       if: github.event_name == 'push'
-      working-directory: libcxx/utils/ci
-      run: |
-        docker compose push builder-base
-        docker compose push actions-builder
-      env:
-        TAG: ${{ github.sha }}
+      run: docker compose push libcxx-linux-builder-base libcxx-linux-builder libcxx-android-builder
 
-    - name: Push the Android builder image
-      if: github.event_name == 'push'
-      working-directory: libcxx/utils/ci
+    # We create tarballs with the images and upload them as artifacts, since that's useful for testing
+    # the images when making changes.
+    - name: Create image tarballs
       run: |
-        docker compose push android-buildkite-builder
-      env:
-        TAG: ${{ github.sha }}
+        docker image save ghcr.io/llvm/libcxx-linux-builder-base:${{ github.sha }} | gzip > libcxx-linux-builder-base.tar.gz
+        docker image save ghcr.io/llvm/libcxx-linux-builder:${{ github.sha }} | gzip > libcxx-linux-builder.tar.gz
+        docker image save ghcr.io/llvm/libcxx-android-builder:${{ github.sha }} | gzip > libcxx-android-builder.tar.gz
+    - uses: actions/upload-artifact@330a01c490aca151604b8cf639adc76d48f6c5d4 # v5.0.0
+      with:
+          name: libcxx-docker-images
+          path: |
+            libcxx-linux-builder-base.tar.gz
+            libcxx-linux-builder.tar.gz
+            libcxx-android-builder.tar.gz

libcxx/docs/Contributing.rst

@@ -269,20 +269,20 @@ Updating the CI testing container images
 ----------------------------------------
 
 The libcxx linux premerge testing can run on one of three sets of runner
-groups. The three runner group names are "llvm-premerge-libcxx-runners",
-"llvm-premerge-libcxx-release-runners" and "llvm-premerge-libcxx-next-runners".
-Which runner set to use is controlled by the contents of
+groups. The three runner group names are ``llvm-premerge-libcxx-runners``,
+``llvm-premerge-libcxx-release-runners`` and ``llvm-premerge-libcxx-next-runners``.
+The runner set to use is controlled by the contents of
 https://github.com/llvm/llvm-project/blob/main/.github/workflows/libcxx-build-and-test.yaml.
-By default, it uses "llvm-premerge-libcxx-runners". To switch to one of the
-other runner sets, just replace all uses of "llvm-premerge-libcxx-runners" in
+By default, it uses ``llvm-premerge-libcxx-runners``. To switch to one of the
+other runner sets, just replace all uses of ``llvm-premerge-libcxx-runners`` in
 the yaml file with the desired runner set.
 
 Which container image is used by these three runner sets is controlled
 and set by the variable values in
 https://github.com/llvm/llvm-zorg/blob/main/premerge/premerge_resources/variables.tf.
 The table below shows the variable names and
 the runner sets to which they correspond. To see their values, follow the
-link above (to variables.tf in llvm-zorg).
+link above (to ``variables.tf`` in llvm-zorg).
 
 +------------------------------------+---------------------------+
 |Runner Set                          |Variable                   |
@@ -295,39 +295,21 @@ link above (to variables.tf in llvm-zorg).
 +------------------------------------+---------------------------+
 
 
-When updating the container image you can either update just the
-runner binary (the part the connects to Github), or you can update
-everything (tools, etc.). Whether to update just the runner or to update
-everything is controlled by the value of ``ACTIONS_BASE_IMAGE``, under
-``actions-builder`` in ``libcxx/utils/ci/docker-compose.yml``.
-
-To update just the runner binary, change the value of ``ACTIONS_BASE_IMAGE``
-to be a modified version of one of the libcxx runner variable images from
-https://github.com/llvm/llvm-zorg/blob/main/premerge/premerge_resources/variables.tf,
-as follows: Find the libcxx runner image name you want to use from the
-variables.tf file. The name will be something like
-``ghcr.io/llvm/libcxx-linux-builder:<some-commit-SHA>``. Replace
-``libcxx-linux-builder`` with ``libcxx-linux-builder-base``. Use this new image
-name as the value you assign to ``ACTIONS_BASE_IMAGE``.
-
-To update the entire container image, set the value of ``ACTIONS_BASE_IMAGE``
-to ``builder-base``. If the value is already ``builder-base`` (there
-have been no just-the-runner updates since the last complete update), then you
-need to find the line containing ``RUN echo "Last forced update executed on``
-in ``libcxx/utils/ci/Dockerfile`` and update the date to be the current date.
-
-Once you have created and merged a PR with those changes, a new image
-will be created, and a link to it can be found at
-https://github.com/llvm/llvm-project/pkgs/container/libcxx-linux-builder,
-where the actual image name should be
-``ghcr.io/llvm/libcxx-linux-builder:<SHA-of-committed-change-from-PR>``.
-
-Lastly you need to create a PR in the llvm-zorg repository,
-updating the the value of the appropriate libcxx runner variable in
-the variables.tf file mentioned above to the name of your newly created
-image (see above paragraph about finding the image name). Once that change
-has been merged, an LLVM premerge maintainer (a Google employee) must use
-terraform to apply the change to the running GKE cluster.
+When updating the container image you can either update just the runner binary (the part
+that connects to Github), or you can update everything (tools, etc.). To update the runner
+binary, bump the value of ``GITHUB_RUNNER_VERSION`` in ``libcxx/utils/ci/docker/docker-compose.yml``.
+To update all of the tools, bump ``BASE_IMAGE_VERSION`` to a newer version of the ``libcxx-linux-builder-base``
+image. You can see all versions of that image at https://github.com/llvm/llvm-project/pkgs/container/libcxx-linux-builder-base.
+
+On push to ``main``, a new version of both the ``libcxx-linux-builder`` and the ``libcxx-android-builder``
+images will be built and pushed to https://github.com/llvm/llvm-project/packages.
+
+You can then update the image used by the actual runners by changing the sha associated
+to ``libcxx_runner_image``, ``libcxx_release_runner_image`` or ``libcxx_next_runner_image``
+in `the Terraform configuration file <https://github.com/llvm/llvm-zorg/blob/main/premerge/premerge_resources/variables.tf>`_.
+To do so, you will need to create a PR in the llvm-zorg repository and wait for it to be
+merged. Once that change has been merged, an LLVM premerge maintainer (a Google employee)
+must use terraform to apply the change to the running GKE cluster.
 
 
 Monitoring premerge testing performance