RBAC Rules for users

[kfox1111]

The documentation here: https://gateway-api.sigs.k8s.io/concepts/security-model/#roles-and-personas references personas that can use the gateway api

But the chart does not deploy any RBAC rules enabling any users but a cluster-admin to use the gateway api. This makes it very hard to use.

#4532 was a first stab at some rbac rules, but seems to have stalled, and did not use the personas or support all the modes defined by the gateway api.

We should add an option to the chart to select between no user rbac (existing behavior), 3-tier and 4-tier setups as described in:

• https://gateway-api.sigs.k8s.io/concepts/security-model/#write-permissions-for-simple-3-tier-model
• https://gateway-api.sigs.k8s.io/concepts/security-model/#write-permissions-for-advanced-4-tier-model

[jukie]

Thanks for raising this. Could you share a bit more about the use case and why you’d prefer the chart to manage human-oriented RBAC instead of handling that in your own cluster policy?

I'm open to hearing input from other maintainers or the community but I personally don't think we should ship aggregated roles or really any human-oriented RBAC rules.

• RBAC models vary a lot between orgs so I don't think we can build a common set that would match everyone's expectations in a safe way.
• The blast radius is large - if we get an aggregated rule wrong, it can grant cluster-wide privileges and is hard to unwind across upgrades.

In my opinion we should follow least-privilege and limit RBAC to controller-scoped service accounts required for Envoy Gateway to function. Managing human access typically belongs to the platform or security layer.

The Gateway API docs you linked read to me as guidance/examples rather than a mandate. To my knowledge the official deployments also don’t package aggregated human roles, probably for similar reasons.

[kfox1111]

Kubernetes provides some prebuilt rbac rules like cluster-admin and admin and such that have largely been usable by the majority of clusters without users having to write custom rules.

Ingress works that way too. If you are a namespace admin, you can create ingresses. Most charts assume you helm install ingress-nginx (as a cluster-admin) and then helm install fooapp --set ingress.enabled=true and set a couple of other values (as namespace admin), and things work.

new versions of helm can do the same thing with httpRoute objects for charts. so looking for the equivalent at minimum, to make it easy to follow the same workflow.

helm install envoy-gateway --set rbac.mode=simple (as cluster admin) + helm install fooapp --set httpRoute.enabled=true (as a namespace admin) and you have something working.

While the gateway api has provided several different models of rbac systems, we could start with something simple, where cluster-admin assumes ownership of all the gateway stuff, and just httpRoutes is extended to users (namespace admins) by default? Others can be added as common patterns found.

[jukie]

I'm not aware of any ingress or gateway api controller that ships Roles/ClusterRoles in that way.

Ingress does happen to make RBAC management easier by way of already having the roles aggreated like you mentioned but that's because it's a core Kubernetes API whereas Gateway API requires an explicit installation.

I'm still interested in hearing from a few more people on this but could you please elaborate on the challenges for users in setting up RBAC themselves?

[kfox1111]

They don't have to, because Ingress object type is built into k8s and its in the admin ClusterRole by default. So, things "just work" out of the box.

If we want to help gain adoption we need to make it as easy as possible to replace ingress with gateway IMO. Having to figure out rbac rules for every cluster from scratch is kind of a huge pain and will hinder adoption. :/

If not profiles, we could just do a structure where we allow some individual configs in the chart like:

namespaceAdminRBAC:
  enable:
    httpRoutes: true
    tcpRoutes: true
    gateways: false

and then the end user could at least pick a few to enable quickly without having to write the explicit policies.

[arkodg]

+1 for opt in Role creation in the EG helm chart for the personas we have

[jukie]

@kfox1111 Right, for ingress controllers that works like you mentioned and that's because it's a core K8s API. To my knowledge, no Gateway API controllers (nor the official Gateway API installation manifests) are shipping human targeted RBAC rules/roles and in my opinion it's not a good idea to start doing it.

The same is true across many other k8s operators/controllers that ship CRDs outside the K8s core APIs and isn't limited to Gateway API controllers. It's recommended to leave RBAC management up to the users and is widely accepted as a security best practice.

Looking to hear from other @envoyproxy/envoy-maintainers on this

[kfox1111]

If its not easy to use gateway api, people will just stick to ingress. Its been years and adoption is a major problem :/

I'm having a real hard time adopting it myself due to lots of little issues like this.

It should not be the end users problem if the k8s apiserver developers decided that they wanted to have ingress in tree and gateway api out of tree. Or, if you make it their problem, you run the risk of them going elsewhere.

I don't think it is a good plan not to have this feature, conditionally. It costs practicality nothing for the maintainers, and lowers the bar to adoption. something you want.

[jukie]

If other maintainers agree I'm willing to accept that and move forward but I don't buy that this is hindering adoption. There's still plenty of challenges with Gateway API but I don't think default RBAC is adding friction.

Again, this is a common stance across many Kubernetes addons/operators/etc. Are there any examples you can point that handle this differently?

From a quick browse other Gateway API controllers such as Istio, Contour, Traefik, and Linkerd don't do this.

Argo Rollouts does do this though! https://github.com/argoproj/argo-helm/blob/01b1d84f5fe5785b6292e6f47c96a9b1f3099b25/charts/argo-rollouts/templates/aggregate-roles.yaml

Kubeflow creates dedicated roles and then uses rbac.authorization.kubeflow.org/aggregate-to-kubeflow-katib-admin: "true" which would still require users to setup RoleBindings. Would that be acceptable for your use case?

Have the Gateway maintainers made any suggestions about this?