Broker access to the Kubernetes API

[griffindvs]

In versions of Strimzi prior to the upcoming 0.49.0, the only case where broker pods interact directly with the Kubernetes API is for rack awareness. When rack awareness is enabled, an init container is added to the broker pods to fetch the rack ID from the Kubernetes node.

My team embeds Strimzi into a product that our clients deploy onto their own Kubernetes clusters. Many of these clients have very strict security requirements including a requirement that only operators interact with the Kubernetes API. The operands that manage the client data then do not have any access to the Kubernetes API.

These requirements were part of the motivation for opening strimzi/proposals#184 which suggests an alternative approach to rack awareness which removes the broker usage of the Kubernetes API.

With the changes delivered in #11447 in support of #11294, it looks like Strimzi 0.49.0 will be extending the usage of the Kubernetes API by the broker pods.

Previously, a container startup script populated PKCS12 trust and key stores using the PEM format certificates mounted as volumes into the broker pods. The changes in 0.49.0 look to be moving away from this script by instead fetching the secrets from the Kubernetes API and then building the trust and key stores as a part of the KafkaAgent.

Is there a reason for fetching the secrets from the Kubernetes API instead of continuing to mount them as volumes and having the KafkaAgent read from those mounted files? Also, would there be an opportunity to allow adopters to opt back into the old approach of mounting secrets as volumes?

Thank you for the help with these questions!

[scholzj]

There are several reasons for the changes. For example:

• Reducing the security risk from having certificate processing tools, such as OpenSSL, installed in the images
• Allowing for future dynamic updates of the certificates
• Simpifying the logic to in the container images and reducing the dependency between the operator and the images
• Improving the test quality and coverage by having the logic Java-based intsead of being done in shell

While some of them are not achieved immediately, these changes are a key part of the path towards it.

There is no plan to support the old approaches.

---

Please also keep in mind that you are the first one raising this as a concern. While there were people in the past who preferred to avoid giving the operator any rights to cluster-wide resources, nobody raised any concerns about the API access. (While they actually asked for the features listed above because of how they lead to improved security)

[griffindvs]

Thank you for the reply. I apologize if I am missing something basic here, but I think that those goals could be achieved without fetching the secrets from the Kubernetes API. Could the Java-based logic in the KafkaAgent just read the PEM format certificates from the file system? This would allow you to remove the certificate processing tools and scripts from the container image. Secrets mounted as volumes are refreshed in the pod by the kubelet when the secret changes, so I don't think mounting as volumes would preclude dynamic certificate updates either.

[scholzj]

Almost anything can be done in many different ways. But I think this is the one that makes the most sense and has the least complexity. In general, using volumes for reloading anything is pretty complicated, given their eventually consistent nature.

[ppatierno]

I see the KafkaAgent suggestion as a different approach but as pointed by Jakub it's much more complex.
It would need re-implementing the script shell logic (to fill the trust/key stores) but in Java within that component. Why should have we picked that way when the KubernetesSecretConfigProvider provides a really immediate and simpler one?
Also while I can get the reasoning for the proposal about rack-awareness, so admins restrict the access to cluster-wide resources (i.e. nodes) in Kubernetes, in this case the Secret is a namespaced resource. Is it really that dangerous for some users/companies/customers having access to a Secret, created by the operator, from operands related to that operator?

[griffindvs]

Thank you both for the feedback. I think those are reasonable explanations, and I agree cluster-wide resource access is much more of a concern than namespaced access to specific secrets.