Restructuring container docs #886

ChristinaLK · 2025-04-29T22:03:59Z

https://chtc.github.io/web-preview/preview-container-adjust

MASSIVE changes to the container docs. 🙃 Lots of shuffling, TBH, not a lot of deletion.

I still need to go through and make sure all the links work. Also we could consider splitting apart some of the longer guides. But I'm happy with this for now!

And videos would be next!!

ChristinaLK · 2025-04-29T22:06:58Z

i ALSO do include Andrew's new tool, so we should, uh, make sure that's live before we merge.

xamberl · 2025-11-18T20:35:52Z

_uw-research-computing/alphafold.md

Not sure how this fits in this container PR. A leftover file from a previous branch?

xamberl · 2025-11-18T20:41:46Z

_uw-research-computing/apptainer-build.md


 This guide describes the general process for creating an Apptainer container. 
-Specifically, we discuss the components of the "definition file" and how that file is used to construct or "build" the container itself. 
+Specifically, we discuss the components of the "definition file" and how that file is used to construct or "build" the container itself. For a more step-by-step description of 


Suggested change

Specifically, we discuss the components of the "definition file" and how that file is used to construct or "build" the container itself. For a more step-by-step description of

Specifically, we discuss the components of the "definition file" and how that file is used to construct or "build" the container itself. **Use this guide as a reference to help customize the contents of your container.** For a step-by-step tutorial of

add `chmod 777 /tmp` in guide

xamberl · 2025-11-18T20:52:56Z

_uw-research-computing/apptainer-htc.md

-condor_submit -i build.sub
-```
-{:.term}
+TBD: graphic


We'll need a graphic here before we merge.

xamberl · 2025-11-18T20:53:18Z

_uw-research-computing/apptainer-htc.md

+1. **Start an interactive job for building.** We require that you build containers while in an interactive build job.
+1. **Build the container.** To build a container, Apptainer uses the instructions in the `.def` file to create a `.sif` file. The `.sif` file is the compressed collection of all the files that comprise the container.
+1. **(Optional): Test the container.** Once the image (`.sif` file) is created, it is important to test it to make sure you have all software, packages, and libraries installed correctly.
+1. **Move the container to a persistent location.** We recommend placing the image file into your `/staging` folder


Suggested change

1. **Move the container to a persistent location.** We recommend placing the image file into your `/staging` folder

1. **Move the container to a persistent location.** We recommend placing the image file into your `/staging` folder.

xamberl · 2025-11-18T21:12:31Z

_uw-research-computing/apptainer-htc.md


 ```
-exit
+chtc-submit-apptainer-build -build image.def


This command currently does not exist in $PATH

This is my winter break project, I hope.

For now, we should provide a build.sub file.

xamberl · 2025-11-18T21:13:29Z

_uw-research-computing/apptainer-htc.md

+{:.tip-header}
+> 
+> Apptainer `.sif` files can be fairly large, especially if you have a complex software stack. 
+> If your interactive job abruptly fails during the build step, you may need to increase the value of `request_disk` in the submit file generated by `chtc-submit-apptainer-build`


Suggested change

> If your interactive job abruptly fails during the build step, you may need to increase the value of `request_disk` in the submit file generated by `chtc-submit-apptainer-build`

> If your interactive job abruptly fails during the build step, you may need to increase the value of `request_disk` in the submit file generated by `chtc-submit-apptainer-build`.

xamberl · 2025-11-18T21:18:18Z

_uw-research-computing/apptainer-htc.md

+> If the build command fails, examine the output for error messages that may explain why the build was unsuccessful.
+Typically there is an issue with a package installation, such as a typo or a missing but required dependency.
+Sometimes there will be an error during an earlier package installation that doesn't immediately cause the container build to fail.
+But, when you test the container, you may notice an issue with the package.
+> 
+> If you are having trouble finding the error message, edit the definition file and remove (or comment out) the installation commands that come after the package in question.
+Then rebuild the image, and now the relevant error messages should be near the end of the build output.


Suggested change

> If the build command fails, examine the output for error messages that may explain why the build was unsuccessful.

Typically there is an issue with a package installation, such as a typo or a missing but required dependency.

Sometimes there will be an error during an earlier package installation that doesn't immediately cause the container build to fail.

But, when you test the container, you may notice an issue with the package.

>

> If you are having trouble finding the error message, edit the definition file and remove (or comment out) the installation commands that come after the package in question.

Then rebuild the image, and now the relevant error messages should be near the end of the build output.

> If the build command fails, examine the output for error messages that may explain why the build was unsuccessful.

> Common errors include:

> * Typos

> * Missing dependencies

> Search the internet for the error message as a starting point for troubleshooting software installation. You can also [reach out](get-help) to us for help.

xamberl · 2025-11-18T21:19:56Z

_uw-research-computing/conda-installation.md

-1. Follow the instructions in [our guide](conda-installation.html#option-b-create-your-own-portable-copy)
+Create your own portable copy of your Conda packages:

 > This approach may be sensitive to the operating system of the execution point. 


Suggested change

> This approach may be sensitive to the operating system of the execution point.

> This approach is sensitive to the operating system of the execution point.

xamberl · 2025-11-18T21:23:57Z

_uw-research-computing/conda-installation.md

I suggest deleting the "More information" section (up to the "executable" section)

Honestly this page is super long... may need a trim in another PR later

xamberl · 2025-11-18T21:31:48Z

_uw-research-computing/docker-jobs.md

 highlighter: none
 layout: guide
-title: Running HTC Jobs Using Docker Containers
+title: Use Custom Software in Jobs Using Docker


I think the previous title is more informative - the changed title is vague and could possibly confuse a reader into thinking this guide is about building Docker containers

xamberl

The general structure LGTM! There's a couple comments/suggestions I added, but I think it should be good to merge. We can then go in with a fine-tooth comb on each software-related page.

aowen-uwmad

Overall, I like the changes you've made. Once the specific comments in mine & Amber's reviews are addressed, this is good to go.

aowen-uwmad · 2025-12-04T22:27:39Z

_uw-research-computing/apptainer-build.md

    apt install -y gcc make wget
 ```

+> The `chmod 777 /tmp` is a specific workaround for building containers on the HTC system. Do not use this line if you are using Apptainer to build containers on a different system.


Suggested change

> The `chmod 777 /tmp` is a specific workaround for building containers on the HTC system. Do not use this line if you are using Apptainer to build containers on a different system.

> The `chmod 777 /tmp` is a specific workaround for building containers on the HTC system at CHTC. **Do not use this** line if you are using Apptainer to build containers **on a different system**.

aowen-uwmad · 2025-12-04T22:30:08Z

_uw-research-computing/apptainer-htc.md


 ```
-exit
+chtc-submit-apptainer-build -build image.def


This is my winter break project, I hope.

For now, we should provide a build.sub file.

aowen-uwmad · 2025-12-04T22:35:08Z

_uw-research-computing/apptainer-htc.md

+> Apptainer `.sif` files can be fairly large, especially if you have a complex software stack. 
+> If your interactive job abruptly fails during the build step, you may need to increase the value of `request_disk` in the submit file generated by `chtc-submit-apptainer-build`
+> In this case, the `.log` file should have a message about the reason the interactive job was interrupted.
+{:.tip}


Suggested change

> Apptainer `.sif` files can be fairly large, especially if you have a complex software stack.

> If your interactive job abruptly fails during the build step, you may need to increase the value of `request_disk` in the submit file generated by `chtc-submit-apptainer-build`

> In this case, the `.log` file should have a message about the reason the interactive job was interrupted.

{:.tip}

> Your interactive job may fail abruptly during the build step.

> The most common reason is that your interactive job ran out of disk space.

> **Try increasing the value of `request_disk` in the submit file.**

>

> You can see the exact reason the job was interrupted by running `condor_q -hold` or by looking in the `.log`.

{:.tip}

aowen-uwmad · 2025-12-04T22:39:53Z

_uw-research-computing/docker-jobs.md

+To run a job using a Docker container, you will need access to a Docker container
+image that has been built and placed onto the
+[DockerHub](https://hub.docker.com/) website or another Docker registry service 
+like [https://quay.io/] or a GitLab registry. 


Suggested change

like [https://quay.io/] or a GitLab registry.

like [https://quay.io/](https://quay.io/) or a GitLab registry.

aowen-uwmad · 2025-12-04T22:42:59Z

_uw-research-computing/python-jobs.md

 ## Quickstart: Python

-### Option A (recommended)
+### Option A - Own Container (recommended)


Suggested change

### Option A - Own Container (recommended)

### Option A - Your Container (recommended)

Doesn't match the bit in the R card.

aowen-uwmad · 2025-12-04T22:46:32Z

_uw-research-computing/software-overview-htc.md

+1. Find an existing container 
+1. Build your own container image
+
+If building your own container, the process will include: 


Suggested change

If building your own container, the process will include:

#### Find an existing container image

TBD: guidance on finding useful, trusted container images online.

#### Build your own container image

To build your own container, the process will include:

aowen-uwmad · 2025-12-04T22:48:44Z

Once the updated guides are live on the website, I think we should ask Patricia Tran to do a once over to look for any additional improvements. But this has been lingering long enough that it shouldn't be blocker.

ChristinaLK added 8 commits April 29, 2025 13:39

initial docker updates, toc incomplete

6f63641

later updates ot docker jobs guide

c1b0fb5

reformatting apptainer guide

2e68230

rearranging container guides

3d06ec4

more software menu tweaks

eacff0b

rearranging apptainer build

05830bc

apptainer final adds

328f061

big software overview rewrite

952c08d

ChristinaLK requested review from aowen-uwmad and xamberl April 29, 2025 22:04

ChristinaLK added 6 commits July 18, 2025 09:42

merging recent changes

b110c5b

container changes, maybe

acd89ff

updating quickstart pages a bit

487785b

moving things on the overview page

2733806

final apptainer changes

6f984e4

docker guide cleanup

e33f86a

xamberl reviewed Nov 18, 2025

View reviewed changes

_uw-research-computing/alphafold.md

Copy link

Contributor

xamberl Nov 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure how this fits in this container PR. A leftover file from a previous branch?

xamberl reviewed Nov 18, 2025

View reviewed changes

Update apptainer-build.md

7a8152a

add `chmod 777 /tmp` in guide

xamberl reviewed Nov 18, 2025

View reviewed changes

xamberl requested changes Nov 18, 2025

View reviewed changes

aowen-uwmad self-assigned this Dec 4, 2025

aowen-uwmad requested changes Dec 4, 2025

View reviewed changes

	Specifically, we discuss the components of the "definition file" and how that file is used to construct or "build" the container itself. For a more step-by-step description of
	Specifically, we discuss the components of the "definition file" and how that file is used to construct or "build" the container itself. Use this guide as a reference to help customize the contents of your container. For a step-by-step tutorial of

	1. Move the container to a persistent location. We recommend placing the image file into your `/staging` folder
	1. Move the container to a persistent location. We recommend placing the image file into your `/staging` folder.

	> If your interactive job abruptly fails during the build step, you may need to increase the value of `request_disk` in the submit file generated by `chtc-submit-apptainer-build`
	> If your interactive job abruptly fails during the build step, you may need to increase the value of `request_disk` in the submit file generated by `chtc-submit-apptainer-build`.

	> This approach may be sensitive to the operating system of the execution point.
	> This approach is sensitive to the operating system of the execution point.

	> The `chmod 777 /tmp` is a specific workaround for building containers on the HTC system. Do not use this line if you are using Apptainer to build containers on a different system.
	> The `chmod 777 /tmp` is a specific workaround for building containers on the HTC system at CHTC. Do not use this line if you are using Apptainer to build containers on a different system.

-> Apptainer `.sif` files can be fairly large, especially if you have a complex software stack.
-> If your interactive job abruptly fails during the build step, you may need to increase the value of `request_disk` in the submit file generated by `chtc-submit-apptainer-build`
-> In this case, the `.log` file should have a message about the reason the interactive job was interrupted.
-{:.tip}
+> Your interactive job may fail abruptly during the build step.
+> The most common reason is that your interactive job ran out of disk space.
+> **Try increasing the value of `request_disk` in the submit file.**
+>
+> You can see the exact reason the job was interrupted by running `condor_q -hold` or by looking in the `.log`.
+{:.tip}

	like [https://quay.io/] or a GitLab registry.
	like [https://quay.io/](https://quay.io/) or a GitLab registry.

	### Option A - Own Container (recommended)
	### Option A - Your Container (recommended)

-If building your own container, the process will include:
+#### Find an existing container image
+TBD: guidance on finding useful, trusted container images online.
+#### Build your own container image
+To build your own container, the process will include:

Restructuring container docs #886

Are you sure you want to change the base?

Restructuring container docs #886

Uh oh!

Conversation

ChristinaLK commented Apr 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ChristinaLK commented Apr 29, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

xamberl left a comment

Choose a reason for hiding this comment

Uh oh!

aowen-uwmad left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

aowen-uwmad commented Dec 4, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

ChristinaLK commented Apr 29, 2025 •

edited

Loading