Review hashicorp license change impact

[killianmuldoon]

Recently hashicorp announced changes to some of its licenses. This issue is a place to assess the impact on Cluster API.

CNCF issue: cncf/foundation#617

None of the dependencies CAPI imports have been updated to the Business source license.

Note: The original finding in this issue are irrelevant at this point. See below for the current state and plan-of-action.

Initial findings

Generate list of all modules which are importing a hashicorp module. This must be done for each of the three go modules in the CAPI repo.

go mod graph | grep "\s.*hashicorp" > hashicorp_modules
cd test ; go mod graph | grep "\s.*hashicorp" >> ../hashicorp_modules
cd ../hack/tools ; go mod graph | grep "\s.*hashicorp" >> ../../hashicorp_modules
cd ../..

Get a unique sorted list of hashicorp modules that are being imported

cat hashicorp_modules | cut -d ' ' -f 2 | sort -u -o hashicorp_modules

The end result of the above is a list of 27 modules. Some are the same module with a different version.

• MPL: https://github.com/hashicorp/consul/blob/main/api/LICENSE
  github.com/hashicorp/consul/api@v1.1.0
  github.com/hashicorp/consul/api@v1.20.0

• MPL: https://github.com/hashicorp/consul/blob/main/sdk/LICENSE
  github.com/hashicorp/consul/sdk@v0.1.1

• MPL: https://github.com/hashicorp/errwrap/blob/master/LICENSE
  github.com/hashicorp/errwrap@v1.0.0

• MPL: https://github.com/hashicorp/go-cleanhttp/blob/master/LICENSE
  github.com/hashicorp/go-cleanhttp@v0.5.1
  github.com/hashicorp/go-cleanhttp@v0.5.2

• MIT: https://github.com/hashicorp/go-hclog/blob/main/LICENSE
  github.com/hashicorp/go-hclog@v1.2.0

• MPL: https://github.com/hashicorp/go-immutable-radix/blob/master/LICENSE
  github.com/hashicorp/go-immutable-radix@v1.0.0
  github.com/hashicorp/go-immutable-radix@v1.3.1

• MPL: https://github.com/hashicorp/golang-lru/blob/master/LICENSE
  github.com/hashicorp/golang-lru@v0.5.0
  github.com/hashicorp/golang-lru@v0.5.1
  github.com/hashicorp/golang-lru@v0.5.4

• MIT: https://github.com/hashicorp/go-msgpack/blob/main/LICENSE
  github.com/hashicorp/go-msgpack@v0.5.3

• MPL: https://github.com/hashicorp/go-multierror/blob/main/LICENSE
  github.com/hashicorp/go-multierror@v1.0.0

• BSD: (forked from go) https://github.com/hashicorp/go.net/blob/master/LICENSE
  github.com/hashicorp/go.net@v0.0.1

• MPL https://github.com/hashicorp/go-rootcerts/blob/master/LICENSE
  github.com/hashicorp/go-rootcerts@v1.0.0
  github.com/hashicorp/go-rootcerts@v1.0.2

• MPL https://github.com/hashicorp/go-sockaddr/blob/master/LICENSE
  github.com/hashicorp/go-sockaddr@v1.0.0

• MIT https://github.com/hashicorp/go-syslog/blob/master/LICENSE
  github.com/hashicorp/go-syslog@v1.0.0

• MPL https://github.com/hashicorp/go-uuid/blob/master/LICENSE
  github.com/hashicorp/go-uuid@v1.0.0
  github.com/hashicorp/go-uuid@v1.0.1

• MPL https://github.com/hashicorp/hcl/blob/master/LICENSE
  github.com/hashicorp/hcl@v1.0.0

• MPL https://github.com/hashicorp/logutils/blob/master/LICENSE
  github.com/hashicorp/logutils@v1.0.0

• MIT https://github.com/hashicorp/mdns/blob/main/LICENSE
  github.com/hashicorp/mdns@v1.0.0

• MPL https://github.com/hashicorp/memberlist/blob/master/LICENSE
  github.com/hashicorp/memberlist@v0.1.3

• MPL https://github.com/hashicorp/serf/blob/master/LICENSE
  github.com/hashicorp/serf@v0.10.1
  github.com/hashicorp/serf@v0.8.2

Outstanding tasks:

• 🌱 Add licence-scan for pull requests #9184
• Remove the indirect dependency on hashicorp HCL. Current exception is implemented here: https://github.com/killianmuldoon/cluster-api/blob/c173312047ca2497cae8e4617fcea050d29e670e/hack/verify-licenses.sh#L38C1-L38C1
• Consider adding an additional periodic license scan for published artifacts including images and binaries. Do periodic license scanning on images and build artifacts #9436

[killianmuldoon]

This list reflects the current state of the master branches. These can change in future versions.

It could be good idea to add license scanning on PRs so that if a version updates to a non-compatible license we can catch it. Not sure what k/k is doing in this area right now.

[killianmuldoon]

Looks like trivvy has this functionality - will take a look at how it works and whether it's useful.

[sbueringer]

Thx for looking into this!

/triage accepted

[vincepri]

Thanks @killianmuldoon ! We might want to broadcast the news at this week community meeting, there is usage in other Cluster API repositories as well.

[sbueringer]

@killianmuldoon
https://github.com/kubernetes/kubernetes/blob/master/hack/unwanted-dependencies.json#L30
from https://kubernetes.slack.com/archives/C5P3FE08M/p1691699636105219?thread_ts=1691699636.105219&cid=C5P3FE08M

Just in case it's relevant

[sbueringer]

Looks like they are blocking dependencies because of MPL (because "MPL license not in CNCF allowlist") Would be interesting to know how they enforce this config (and eg if they check only go.mod or also go.sum)

Also would be good to know why we have those dependencies

[killianmuldoon]

Looks like they are blocking dependencies because of MPL (because "MPL license not in CNCF allowlist") Would be interesting to know how they enforce this config (and eg if they check only go.mod or also go.sum)
Also would be good to know why we have those dependencies

Their tool only checks go.mod AFAIU, and they have those dependencies in the go.sum. I'm still doing some research on how the CNCF sees go.sum dependencies, but I haven't found anything concrete yet.

[furkatgofurov7]

Thanks @killianmuldoon looking into this!

[sbueringer]

I recently implemented a feature in CR for authorized metrics. I ended up implementing it in a way that you only actually get dependencies like k8s.io/apiserver in your go.mod file when you are using it (i.e. importing a certain package). If you don't use it, it will only end up in go.sum.

So my rough guess is that dependencies which are only in go.sum are not actually getting compiled into your binary because you don't need any of their packages. They are probably just part of "what if you include all dependencies of all dependencies without looking at specific packages". But really just a guess.

Probably we should just ask in some Kubernetes Slack channel about the background of only checking go.mod and they can tell us (it seems k/k is aligning to CNCF recommendations)