[WIP] POC Opts (Args) concept

[lefou]

This PR is still Work in progress, mostly because not all examples are migrated and the API may change. Nevertheless the concept is open for discussion. Please engage!

Summary

BINARY BREAKING CHANGES!

Related to:

• Make out/ folder contents (more) reproducible and filesystem layout agnostic #3660 - Tracker issue
• Dynamically map known root paths in cache files #6031 - Stores relocatable paths
• new `Args` class for relocateable configurations #6057 - Idea that drives this PR about how to store relocatable configuration options

This PR contains:

1. The new mill.api.opt API containing the classes Opts, Opt, OptGroup and OptMap.

2. A refactoring of various modules that contain configuration data potentially containing paths - this is a API breaking change to just proof the concept

In contrast to the proposal (#6057) the class is named Opts, not Args, since we already have a mill.api.Args class used for other purposes.

While this PR can be applied as-is, it's real effect only emerges when also PR #6031 is merged, since otherwise, the paths stored as part of the configuration data will not be relocatable.

Opt API

Opt is a holder of configuration data. Opt can be constructed from one or multiple Strings or os.Paths. When rendered as string, all parts will be concatenated.

Opts is a Seq-like container to be used instead of Seq[String]. OptMap is a type alias to Map[String, String] to provide some convenience. Both container types can be easily converted to a String-based container with .toStringSeq or .toStringMap.

OptGroup is a group of Opts that are tightly coupled, and should never be split. This is useful to model semantically bound CLI args with multi-arity like --file <file> or -classpath jar2:jar2:...:jarN.

API changes in detail

All tasks containing configuration data that also may contain file or directory paths should use the Opt class or the container type Opts or OptMap.

Most prominent tasks are:

• javacOptions: T[Seq[String]] --> T[Opts]
• scalacOptions: T[Seq[String]] --> T[Opts]
• kotlincOptions: T[Seq[String]] --> T[Opts]
• forkArgs: T[Seq[String]] --> T[Opts]
• forkEnv: T[OptMap] --> T[OptMap]

JSON serialization

The JSON sturcture is more complex that the previous Seq[String] equivalent, because it needs to support options, option groups and multi-parts options, which may itself consist of many strings and paths.

A rather complex Opts JSON serialization may look like this:

Opts(
  "-deprecated", 
  OptGroup("-classpath", Opt.mkPath(classpath, sep = ":"))
)

[
  "-deprecated",
  [
    "-classpath",
    [
      { "path": "/path/to/jar1.jar" },
      ":",
      { "path": "/path/to/jar2.jar" },
      ":",
      { "path": "/path/to/jar3.jar" }
    ]
  ]
]

Since the JSON serialization for paths re-use the respective upickle-serializers, the root path mapping of PR #6031 may also work here (Resulting is paths like $MILL_OUT/task/to/jar.dest/out.jar).

YAML convenience

In addition, the json reader supports reading of flat json arrays, which means, that all Seq[String] can easily be read as Opts. This was done to make working with Opts as convenient as Seq[String] not only in Scala code but also in YAML configuration code.

This means, build.mill YAML frontmatter and build.mill.yaml don't need any changes.

mill-build:
  scalacOptions: ["-verbose"]
  forkEnv: { "MY_CUSOTM_ENV": "my-env-value" }

Although the full encoding is also supported but seldomly required.

mill-build:
  forkArgs: [[["-javaagent:", {"path": "$WORKSPACE/agent.jar"}]]]
  forkEnv: { "MY_WORK_DIR": [{"path": "$HOME/workDir"}] }

Caveats

In this current POC, we need to add import mill.api.opt.*. I'm glad for ideas, how to improve this and make it a single import.

[lefou]

Merged in latest changes from main branch, fixed conflicts and added a OptMap type alias to make working with maps containing config data more convenient.