Upgrade pants from 2.15 to 2.16.0a0

[cognifloyd]

Background

This is another part of introducing pants, as discussed in various TSC meetings.

Related PRs can be found in:

• pantsbuild milestone (which includes PRs since Initialize pantsbuild #5713)
• pantsbuild label (which also includes preparatory PRs that came before adding pantsbuild)

Overview of this PR

I recommend stepping through the commits to review this PR. Ignore the "fix typo in st2actions/BUILD" commit, as it is part of #5931, which should be merged first.

This updates to pants to 2.16.0a0 which only requires:

• regenerate the pants-plugins lockfile
• regenerate the st2 lockfile

Unlike the upgrade to 2.15 (#5899), we do not need to update our pants-plugins because they don't use anything that changed (see the Plugin upgrade guide).

Upgrading to 2.16 gives us access to some new features and allows us to delete some of our BUILD metadata:

• Several module_mapping definitions were upstreamed, so we no longer need to define them locally.
• pants.backend.experimental.python.framework.stevedore: The plugin I wrote to tell pants how to manage including runners and other stevedore plugins when the tests need them to be available.
• Uncomment env("ST2_PUBLISH_REPO") in pants-plugins/macros.py (added in pants: Add st2_*_python_distribution() macros to pants-plugins/macros.py #5901) so that we can use the ST2_PUBLISH_REPO env var to determine where we want to publish the wheels (eg to use @testpypi instead of @pypi).
• The python-dump-source-analysis goal makes debugging dependency inference much easier. And, it was the final piece we needed to be able to use [python-infer].string_imports so that pants can infer python dependencies from strings, not just the actual imports. That made it possible to delete some of our BUILD metadata.
• The [python-infer].ambiguity_resolution = "by_source_root" option made dependency inference a little smarter. This way, when something imports tests.base, it will use the closest tests.base instead of complaining about all of the different directories that define that. That made it possible to delete even more of the BUILD metadata, as those deps are no longer ambiguous.
• Turn on diff output when using generate-lockfiles (does not display when our git deps change, but it helps for everything else). This is what it looked like when I regenerated the st2 lockfile (see the screenshots in Print diff report when generating new lockfiles. pantsbuild/pants#17347 as well):
  

In a follow-up I will add support for:

• pants.backend.experimental.visibility which gives us access to __dependency_rules__ and __dependents_rules__. This is a much more intelligent way to avoid circular imports (eg st2common does not import st2api) than the regex-lint based approach. This needs a few more bug fixes before we can use it.
  • Validating dependencies covers __dependencies_rules__ and __dependents_rules__

Pants documentation

• Plugin upgrade guide
• 2.16 Changelog
• python-dump-source-analysis goal
• python-infer subsystem
  • string_imports option
  • ambiguity_resolution option