Image SHA keeps changing

[medyagh]

There is a Kong Addon for minikube, and to keep our users secure we verify Image SHA at the time of the minikube release, Image SHa is not supposed to be changed after you publish a version,

unfortunately KONG project keeps pushing a Different Image to a Same Image Version after release, which is not only a bad fo rsecurity and integrity it is adding a burden on minikube mtainerers,

I already created multiple issues in the past without any actions to prevent this from happening

#14261
#14433

and it keeps happening multiple time a months, (see example PR https://github.com/kubernetes/minikube/pull/21698/files)
unfortuntely if Kong project can not meet this requriement, we will have to remove Kong addon from minikube.

To avoid being remove from minikube addon system I suggest creating processes that once an image is published with a tag it should not be overwritten, if there is a change in the Image that is essential that needs to be rebuilt, that means there should be a new Patch version for that image (for example instead of overwritting image 3.9.1 after it was released and published it should be a new image patch version 3.9.2) this is more transparent to the open source community why there was a new change

and will not cause the tools that verify the SHA to fail and make possible security issues trackable and auditable.

[Water-Melon]

Hi @medyagh ,

Thank you for raising this issue.

However, I checked the execution dates of Kong's workflow and the modification time of the tag on Docker Hub, and both are from over four months ago. Therefore, this tag has not been modified from June until now.

Could you please confirm whether your image was pulled from Docker Hub?

[medyagh]

hi @Water-Melon

as seen in this Automated PR
https://github.com/kubernetes/minikube/pull/21698/files
minikube-bot found a NEW SHA (as seen in the bellow diff) and tried get the new SHA and yes it is pulling form docker hub:

"kong:3.9.1@sha256:14c689c0caf1b8da1403a742016ec64d2f5b5b12ecdec2989f36b2c2c4aa1ac0",
"kong:3.9.1@sha256:293fb8c293b4920023131558ef314e52086bb4c92cbb3296f60295811aa006ad",

@Water-Melon if there has been no new build and the SHA has changed, that is a security incident and I suggest you report that as such and find who pushed the image.

when a new image is added to minikube addons we record the SHA in the code and when it changes it makes a Diff in the code, so the sha for kong:3.9.1 has clearly been changed.

and also I suggest you publish the image shas in the release notes so when it Magically Changes you could audit and trace it has changed.

btw I checked the docker hub UI the 3.9.1 image was pushed 4 days ago by doijanky

[medyagh]

btw since.I created this issue the image sha was changed even one more time kubernetes/minikube#21730

Kong Project changes SHAs very freqently after release, this is not right !

[kikito]

Hello @medyagh , thank you for opening this issue.

We (Kong) have no direct control over the _ docker organization, where the kong image is hosted:

• https://hub.docker.com/_/kong

That is under Docker's control - doijanky is their user name. When we want to release a new version, we open a pull request against their official-images repository and request them to make a new release. We don't have a way to trigger a new release ourselves. Docker can choose to update the base image and do a re-release (usually in order to fix some underlying security issue) but that is also not under our control. As far as we know they do this with all the projects inside the _ org, not just with Kong.

I don't know what the best course of action is for minikube maintainers.

[medyagh]

@kikito thats interesting, why would docker wanna control the image that is for this repo ? is there any communication channel between your team and docker ?

[kikito]

@medyagh the main communication channel is the repo I linked. The communication happens in the form of PRs, comments in PRs and issues.